Re: st: Mixed model degrees of freedom and Stata presentation

[ymarchenko@stata.com]

Jordan Silberman <silberman.stata@gmail.com> asked a few questions about
small-sample adjustments for multilevel models.  Let me address some of them:

> ...
>
> So, a few questions:
> 
> 1. Are there plans to provide more extensive options for df estimation (eg,
> Kenward-Roger, Satterthwaite, etc.) with xtmixed/xtmelogit in the future?
> This feature would be extremely helpful, even if just one estimation method
> is provided.

The implementation of small-sample adjustments using Satterthwaite and
Kenward-Roger approximations is very high on our development list, but we do
not anticipate it being added in the nearest future.


> 2. I have read that Stata statisticians believe there's no defensible way to
> estimate df for mixed models. Can anyone explain why this is so, preferably
> in language a non-statistician can understand?

There is no theoretical justification for the proposed adjustments when they
are applied to unbalanced designs and general mixed models.  This is why they
were not added to Stata initially.  

You may find the following talk by Phil Ender useful:

  www.stata.com/meeting/chicago11/materials/chi11_ender.pdf

You may also find interesting the response from Douglas Bates, the author of
R's "lmer" package, who feels even stronger about this issue than us:

  https://stat.ethz.ch/pipermail/r-help/2006-May/094765.html

Even so, we are very sympathetic to those who have very small samples and who
are loath to rely on large-sample properties that produce demonstrably
anticonservative tests and confidence intervals.  In some cases, a
small-sample approximation and its associated assumptions may be preferred to
the large sample statistics with asymptotically provable distributions.


> 3. The solution to this problem offered in Stata is to assume infinite
> degrees of freedom. It seems to me, from a statistically naive perspective,
> that it is literally mathematically impossible to use a less defensible
> solution. It's not possible to provide a df estimate that is further from
> the true df value than infinite. But I suspect that there's more to it than
> this.  Can anyone explain why assuming that df = infinite is more defensible
> than other df estimation methods, even though other methods are
> mathematically guaranteed to provide more accurate df estimates?

The inference currently provided by Stata for mixed models requires large
samples and should not be used with small samples.  We agree that this is
restrictive for small samples in practice and are looking into adding the
commonly used degrees-of-freedom adjustments for linear mixed models.


-- Yulia
ymarchenko@stata.com
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/