Re: st: Comparing Chi2/L2 in different samples using bootstrap

[Steven Samuels <sjsamuels@gmail.com>]

Dmitry:

Contrary to your belief, it is very likely that the data sets can be  
pooled: just incorporate survey year into the stratum definition.  
Write a command like "gen new_stratum = group(year stratum)". Then - 
svyset- the combined samnple with the new stratum variable, but with  
PSUs and weights from from the individual years.

Differing yearly design effects and sample sizes have no bearing on  
the validity of this approach.  There might be some difficulty if the  
types and sizes of PSUs changed greatly between years.  Also, you will  
have to take special steps if the surveys were rotating panels.

Steve

Steven J. Samuels
sjsamuels@gmail.com
18 Cantine's Island
Saugerties NY 12477
USA
Voice: 845-246-0774
Fax:    206-202-4783




On Dec 6, 2010, at 8:44 AM, Dmitriy Poznyak wrote:

Hello all,

I am estimating three identical multinomial models with bootstrap for  
the different years of survey data, for instance. 1991, 1999 and 2007.  
Aside from comparing predicted probabilities, which I assume shouldn't  
pose any problem, I need to compare Chi2/L2 coefficients for the  
different variables in the model. The rationale for doing this, is  
that the fit of the individual predictors (e.g. social-demographic  
stuff) declines through time. Here's where the question arises.  
Clearly, samples in different years have different size, and perhaps  
different design effects, and so on.

In order to possibly address these issues I ran the bootstrapped  
models with the same number of iterations in each case:
bootstrap, reps(2000) force: mlogit vote5  x y z ... ,base(1) cl(zip),  
[pweight=weight1], rrr
Next, I test the effect of the predictors:  test x;  test z, etc.  
Again, the models' specification is identical for all years; what  
differs is the sample size and design.

Considering the bootstrap method being used, will it be possible to  
compare Chi2/L2 and perhaps pseudo R2 coefficients for different  
samples in this case, and, if not, what would be my strategy. Note  
that pooling datasets is not feasible due to several reasons, like  
weighting, etc.

Thanks for your suggestions,
Dmitriy
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/