Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Bootstrap sampling for evaluating hypothesis tests


From   Margaret MacDougall <Margaret.MacDougall@ed.ac.uk>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: Bootstrap sampling for evaluating hypothesis tests
Date   Thu, 14 Mar 2013 10:49:54 +0000

Dear Maareten

Very many thanks for the most interesting details you have provided by means of your various emails. It would be great to be able to use your examples below as exemplars to guide a student who is getting to grips with bootstrapping. Are you able to provide me with story lines that could helo me demonstrate the illustrations in context, identify the particular hypothesis test being assessed and know what the variables stand for?

Thank you very much indeed.

Best wishes

Margaret

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dr Margaret MacDougall
Medical Statistician and Researcher in Education
Centre for Population Health Sciences
University of Edinburgh Medical School
Teviot Place
Edinburgh EH8 9AG

Tel:  +44 (0) 131 650 3211
Fax:  +44 (0) 131 650 6909
E-mail: Margaret.MacDougall@ed.ac.uk
http://www.chs.med.ed.ac.uk/cphs/people/staffProfile.php?profile=mmacdoug

On 13/03/2013 15:45, Maarten Buis wrote:
On Wed, Mar 13, 2013 at 4:04 PM, Margaret MacDougall wrote:
I would value receiving recommendations on literature explaining the
application of bootstrap sampling to assess robustness to Type I errors of a
proposed new hypothesis test. Better still, if the recommended references
contain corresponding computer syntax!

Rich Williams and I are currently working on such a project. In
general I would not say that a test is "robust" against Type I errors
but that the Type I error rate corresponds to your prespecified level
of significance. Type I errors will occur, but the chance of it
occuring should be the same as the level of significance you have
chosen. This means that if we change the data such that the null
hypothesis is true and bootstrap from that changed dataset the
p-values should follow a uniform distribution. This changing the data
is inevitable when assessing the Type I error rate: in order to assess
the probability of rejecting a true null hypothesis you first need to
make sure that the null hypothesis is true.

Here are two examples of how to do this in Stata:

*------------------ begin example ------------------
clear all
sysuse auto
recode rep78 1/2=3
logit foreign price
predict double pr
gen byte ysim = .
keep foreign price rep78 pr ysim
keep if !missing(foreign,price,rep78)
program define sim
	replace ysim = runiform()<  pr
	logit ysim  price ib3.rep78
	test 4.rep78 = 5.rep78 = 0
end
simulate chi2=r(chi2) p=r(p), reps(1000) : sim
simpplot p
qchi chi2, df(2) name(q)
*------------------- end example -------------------
(For more on examples I sent to the Statalist see:
http://www.maartenbuis.nl/example_faq )

*------------------ begin example ------------------
clear all
sysuse auto
gen lnprice = ln(price)
reg turn mpg i.rep78 foreign
predict double mu1

reg turn mpg i.rep78 foreign weight lnprice
predict double mu2
gen double ysim = turn - mu2 + mu1

keep ysim mpg rep78 foreign weight lnprice
keep if !missing(ysim, lnprice, mpg, rep78, foreign, weight)
tempfile temp
save `temp'

  program define qenv_sim_F
     use `1', clear
     bsample
     reg ysim mpg i.rep78 foreign weight lnprice
     test weight lnprice
end

simulate F=r(F) p=r(p), reps(1000): qenv_sim_F `temp'

simpplot p
*------------------- end example -------------------
(For more on examples I sent to the Statalist see:
http://www.maartenbuis.nl/example_faq )

Hope this helps,
Maarten

---------------------------------
Maarten L. Buis
WZB
Reichpietschufer 50
10785 Berlin
Germany

http://www.maartenbuis.nl
---------------------------------
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


--
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index