Notice: On March 31, it was **announced** that Statalist is moving from an email list to a **forum**. The old list will shut down on April 23, and its replacement, **statalist.org** is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
Rebecca Pope <rebecca.a.pope@gmail.com> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: st: odd simulation results for ICC with binary outcomes |

Date |
Sat, 16 Feb 2013 13:09:48 -0600 |

Thanks Austin. With 1,000 reps, I would expect the same general shape by the CLT, correct? What I would expect to differ would be the mean around which they converge. With 1,000 reps, we can't reject equality between the simulated beta-bin & 1/11; with 10,000 we can. The MM estimator similarly "flip-flops", but the ANOVA doesn't. Just looking at the graphs, I agree that it is difficult to argue that the plotted distributions are different (regardless of # of reps chosen); the lines are nearly indistinguishable from each other. Indicating, I think, that despite the tests they are probably heading in the same direction. I've done some more experimenting with beta distributions that may be more realistic. The literature indicates that often individuals do not get recommended care, so I tried some distributions that put more weight on the lower probabilities of success. It takes 10,000 repetitions to achieve the normal shape from the sample means with the skewed distributions. Once this happens, all of the modes (and hence means) look pretty close to the true ICC. The t-tests however indicate that they haven't converged to the theoretically true value, except for the ANOVA. Finally, the beta-bin does even worse with unequally sized groups. The ANOVA still does just fine. I might be focusing over-much on the tests, but this seems completely counterintuitive to me. Am I just misunderstanding how the simulations should work? Or is it time to just accept that regardless of intuition, the ANOVA is going to do just fine irrespective of the underlying distribution? The Donner article seemed to suggest that this is the case, at least for some non-normal distributions. There are also several references at the end regarding binomial data that I will try to find. Thanks also for cleaning up some of the finer points of my code. I haven't used -simulate- often, but of course -tempvar- is unnecessary in that context. I didn't think about that. It also (obviously) didn't occur to me to use -scalar- in place of -local-. Regards, Rebecca On Fri, Feb 15, 2013 at 2:43 PM, Austin Nichols <austinnichols@gmail.com> wrote: > Rebecca Pope <rebecca.a.pope@gmail.com>: > I don't see much difference across estimators below. > You might also want to read > http://www.jstor.org/stable/1403259 > for more options. > > set seed 72114 > set more off > clear all > program define betabinsim, rclass > version 12 > syntax [, a(real 5) b(real 5) groups(int 135) n(real 100)] > drop _all > set obs `groups' > gen int group = _n > gen y1 = rbinomial(`n', rbeta(`a',`b')) > gen y0 = `n'-y1 > reshape long y, i(group) j(event) > xtnbreg y event, i(group) fe nolog > scalar alph=exp(_b[_cons]+_b[event]) > scalar beta=exp(_b[_cons]) > return scalar n = `n' > return scalar a = alph > return scalar b = beta > return scalar mu = alph/(alph+beta) > return scalar icc = 1/(alph+beta+1) > expandcl y, cluster(group event) generate(foo) > loneway event group > return scalar icc_anova = r(rho) > egen double mn=mean(event), by(group) > gen double se2=mn*(1-mn)/`n' > su se2, mean > scalar corrct=r(mean) > su mn > scalar Var=r(Var)-corrct > ret scalar iccm=Var/(r(mean)*(1-r(mean))) > eret clear > end > simul, r(1000): betabinsim > local truerho = 1/(1+5+5) > ttest icc==icc_anova, unpaired > ttest icc ==`truerho' > ttest icc_anova == `truerho' > ttest iccm == `truerho' > la var icc_an "ICC from ANOVA" > la var icc "ICC from xtnbreg" > la var iccm "ICC from MM" > tw kdensity icc_an||kdensity iccm||kdensity icc, xli(`=1/11') > su > * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/faqs/resources/statalist-faq/ * http://www.ats.ucla.edu/stat/stata/

**References**:**st: odd simulation results for ICC with binary outcomes***From:*Rebecca Pope <rebecca.a.pope@gmail.com>

**Re: st: odd simulation results for ICC with binary outcomes***From:*Austin Nichols <austinnichols@gmail.com>

- Prev by Date:
**Re: st: STATA error r(603) File could not be opened - Dropbox** - Next by Date:
**Re: st: using macro lists** - Previous by thread:
**Re: st: odd simulation results for ICC with binary outcomes** - Next by thread:
**st: t test in stata** - Index(es):