Re: st: Resampling and compare full sample with subsamples

[Steve Samuels <sjsamuels@gmail.com>]

Johannes-

I don't get the logic of your approach. Assuming that prevalence rates
of responders and non-responders differ, i.e. show "response bias", a
comparison of random samples of responders to all responders will
provide *no* information on the degree of bias.

There are accepted reweighting techniques for evaluating and reducing
response bias. See the  downloadable references below.

If, in fact, you know descriptive statistics for the entire population
of schools and for the responding schools, you can make the responding
schools more closely resemble not just the sample, but the population.
See Stas Kolenikov's -ipfraking- (-findit-) and John D'Souza's
-calibrate- (SSC).


References:

Burns, Shelley, Xiaolei Wang, and Alexandra Henning. 2011. NCES Handbook
of Survey Methods. NCES 2011-609. National Center for Education
Statistics http://eric.ed.gov/?id=ED521154

Carlson, BL, and Williams, S. 2001. A comparison of two methods to
adjust weights for non-response: propensity modeling and weighting class
adjustments. Proceedings of the Annual Meeting of the American
Statistical Association
http://www.amstat.org/sections/SRMS/proceedings/y2001/Proceed/00111.pdf

Kreuter, Frauke, Kristen Olson, James Wagner, Ting Yan, Trena M
Ezzati-Rice, Carolina Casas-Cordero, Michael Lemay, Andy Peytchev,
Robert M Groves, and Trivellore E Raghunathan. 2010. Using proxy
measures and other correlates of survey outcomes to adjust for
non‚Äêresponse: examples from multiple surveys. Journal of the Royal
Statistical Society: Series A (Statistics in Society) 173, no. 2:
389-407.
http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1139&context=
sociologyfacpub


Little, RJ, and S Vartivarian. 2003. On weighting the rates in
non-response weights. Stat Med 22, no. 9: 1589-1599, available at
http://deepblue.lib.umich.edu/bitstream/2027.42/34860/1/1513_ftp.pdf


Wun, L-M, and Ezzati-Rice, T. 2007. Assessment of the impact of health variables on nonresponse adjustment in the medical expenditure panel survey (MEPS). Proc. Surv. Res. Meth. Sect. Am. Statist. Ass 2857-2864.
http://www.amstat.org/sections/SRMS/Proceedings/y2007/Files/JSM2007-000336.pdf

Steve


Steve Samuels
Consultant in Statistics
18 Cantine's Island
Saugerties NY 12477 USA
845-246-0774


> On Mar 3, 2014, at 10:58 AM, Johannes Thrul <Thrul@ift.de> wrote:
> 
> Dear list,
> I am working on a large survey dataset (12,000 individuals clustered in 600 schools) and want to examine, how non-response/non-participation of schools affects prevalence estimates (e.g., alcohol use). My plan is to reduce the sample of schools randomly and systematically (e.g., only exclude large schools) and compare the resulting estimates from the subsamples with the estimates from the full sample. I thought of an approach like this: Reduce the sample size in 10% increments, draw a number of subsamples at every step and compare the estimates. However, I have 2 questions about how to best approach this in Stata: 
> 1. Drawing subsamples: Should I use a jackknife, bootstrap, or even something entirely different for drawing the subsamples?
> 2. Testing: How should I go about testing the results from the subsamples against the full sample?
> Any help is greatly appreciated!
> Thanks and kind regards, Johannes
> 
> --
> Dr. Johannes Thrul, Dipl.-Psych.
> 
> Wissenschaftlicher Mitarbeiter / Researcher
> Präventionsforschung / Prevention Research
> 
> IFT Institut für Therapieforschung / Parzivalstr. 25, D-80804 München / www.ift.de
> phone +49 (0) 89 360804 86 / fax +49 (0) 89 360804 69 / e-mail thrul@ift.de
> 
> IFT Institut für Therapieforschung gem. Gesellschaft mbH / Registergericht München HRB 46395 Geschäftsführung: Prof. Dr. Gerhard Bühringer
> 
> 
*
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/