Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
Jordan Silberman <silberman.stata@gmail.com> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: st: Re: Degrees of freedom for xtmixed |

Date |
Sat, 22 Jun 2013 15:31:38 -0400 |

At least one reference (Hox's Multilevel Analysis: Techniques and Applications, p. 46) states that the Raudenbush and Bryk approach to degrees of freedom is conservative, particularly when compared to asymptotic significance tests like those of xtmixed. This of course makes sense because the p value associated with a z ratio is always smaller than the p associated with the same t ratio, though the difference btwn the 2 of course becomes negligible with large samples. I certainly agree that it would be nice to have Kenward-Roger approximations with xtmixed! Hopefully Stata will get around to this soon. It seems to me that with Stata 13 they provide many advanced options in terms of types of mixed effects models, but they neglect the very basic issue of providing a df approximation that works for small sample sizes. The Monte Carlo approach you suggest sounds interesting, but for me it would be prohibitively time consuming. And, whether or not it's fair, I think many reviewers would not understand the Monte Carlo approach you're suggesting. Thanks for your response. Jordan On Sat, Jun 22, 2013 at 4:50 AM, Joseph Coveney <stajc2@gmail.com> wrote: > Jordan Silberman wrote: > > The following URL summarizes the simple formulas used to compute > degrees of freedom in the HLM 7.0 software: > http://www.ssicentral.com/hlm/help7/faq/FAQ_Degrees_of_freedom_in_HLM_models.pdf > > Is there any reason why one can't use these formulas to assign df > values to the fixed effects > estimated by xtmixed? I realize that this approach might be > conservative, which is fine for my purposes. > > -------------------------------------------------------------------------------- > > Are they conservative? For some reason, I had always believed that these kinds of simple degrees-of-freedom formulas tended to yield anti-conservative p-value estimates for fixed-effects contrasts. > > While we're waiting for, say, Kenward-Roger approximations, can't we simulate under the null to get an estimate of the quantile of the test statistic with reasonable precision? How far astray would we be led using the observed values of the variance components (REML) as first-pass estimates in the simulation? (And if it's worrisome, aren't there formulas that you can use to compute the worst-case bound for downward bias in these estimates for both REML and MLE, at least for common models?) > > With the small samples where this whole matter arises, and with Stata MP, -xtmixed- shouldn't be so prohibitively slow as to rule out Monte Carlo simulation to get a handle on the distribution of the test statistic under the null hypothesis for a given fixed-effects contrast, as long as you're not interested in something way out in the tails of the distribution. Is there something that makes this approach impractical or invalid? > > Joseph Coveney > > > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/faqs/resources/statalist-faq/ > * http://www.ats.ucla.edu/stat/stata/ * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/faqs/resources/statalist-faq/ * http://www.ats.ucla.edu/stat/stata/

**References**:**st: Degrees of freedom for xtmixed***From:*Jordan Silberman <silberman.stata@gmail.com>

**st: Re: Degrees of freedom for xtmixed***From:*"Joseph Coveney" <stajc2@gmail.com>

- Prev by Date:
**st: piecewise regression that uses panel data with entity and time fixed effects** - Next by Date:
**Re: st: Beta values in QREG** - Previous by thread:
**st: Re: Degrees of freedom for xtmixed** - Next by thread:
**st: Blanchard & Quah decomposition** - Index(es):