Re: st: R: R: bootstrapped p-values

 From "Richard Harvey" To statalist@hsphsun2.harvard.edu Subject Re: st: R: R: bootstrapped p-values Date Sun, 7 Dec 2008 15:26:23 +0000

```hi,

I am still confused by your example..
especially this bit of the code..

generate indicator =abs(r(t))>=abs(scalar(tobs))
sum indicator, mean
display "ASLboot="r(mean)

i understand what it is doing is strong the original value of t-stats
as a scalar and then after the bootrap process creating a count of the
number of bootstrapped t-values greater than the original t and taking
the mean of that number

However, this always turns out to be 1 when i run the code even when i
run it on different samples ..so what does that mean??? how is that
related to the p-value.. sorry if tis is really dumb but i have got
myself into a mental knot with this.

thanks
rich

2008/12/6 Carlo Lazzaro <carlo.lazzaro@tiscalinet.it>:
> Dear Richard,
> I suppose that confusion arose because you are implicitly referring to two
> different things:
>
> - the meaning of p-value reported in bootstrap printout (and Nick's
>
> - the bootstrap p-value vs the original one in searching for the
> significance of the difference of the mean with skewed data.
> As widely reported in literature (please see for instance, Desgagne A,
> Castilloux A, Angers J, LeLorier J. The use of the bootstrap statistical
> method for the pharmacoeconomic cost analysis of skewed data.
> Pharmacoeconomics 1998; 13:487-497.Barber JA and Thompson SG. Analysis of
> cost data in randomized trials: an application of the non-parametric
> bootstrap. Statistics in Medicine 2000; 19:3219-3236), ttest procedure needs
> the fulfilments of some prerequisite before being invoked, which are
> systematically violated whenever data are skewed (please, take a look at the
> following example sketched via Stata 9.2/SE):
> ---------------------------begin example---------------------------
> sysuse auto
> save "C:\Documents and
> Settings\carlo\Documenti\Statistiche\Stata\Richard_1.dta", replace
> ttest mpg, by(foreign) unequal
> scalar tobs=r(t)
> sum mpg, mean
> scalar omean=r(mean)
> sum mpg if foreign==0, mean
> replace mpg = mpg-r(mean) + scalar(omean)  if foreign==0
> sum mpg if foreign==1, mean
> replace mpg = mpg-r(mean) + scalar(omean)  if foreign==1
> sort foreign
> by foreign: sum mpg
> keep mpg foreign
> set seed 1
> bootstrap r(t), reps(1000) nodots strata(foreign) saving(C:\Documents and
> Settings\carlo\Documenti\Statistiche\Stata\Richard_2.dta, every(1)
> replace)verbose : ttest mpg, by(foreign) unequal
> save "C:\Documents and
> Settings\carlo\Documenti\Statistiche\Stata\Richard_1.dta", replace
> use "C:\Documents and
> Settings\carlo\Documenti\Statistiche\Stata\Richard_2.dta", clear
> generate indicator =abs(r(t))>=abs(scalar(tobs))
> sum indicator, mean
> display "ASLboot="r(mean)
> ------------------------------end example----------------------------
>
> HTH, Kind Regards and enjoy your W-E,
>
> Carlo
> -----Messaggio originale-----
> Inviato: venerdì 5 dicembre 2008 23.37
> A: Carlo Lazzaro
> Oggetto: Re: R: bootstrapped p-values
>
> hi,
>
> thanks nick and carlo..
>
> I understand your point about the p value in the statout now. I am a
>
> for example  the following
>
> http://compdiag.molgen.mpg.de/ngfn/docs/2003/nov/Exercises_Bootstrap.pdf
>
> discusses non parametric bootstrap on page 4 and on page 5 it talks about
>
> "The two-sided p-values are now calculated by determining the
> percentage of situation where the
> absolute value of the test statistics calculated from the groups with
> permutated values is at least
> as large as the absolute value of the test statistics calculated from
> the original groups."..
>
> and the following page
>
> westfall.ba.ttu.edu/Finance%20Pres.ppt
>
> talks about something similar on page 11 and page 12
>
> I am not what that means and how that is related to the stata p-value?
> and the following page
>
> westfall.ba.ttu.edu/Finance%20Pres.ppt
>
> talks about something similar on page 11 and page 12
>
> I am not what that means and how that is related to the stata p-value?
>
>
> thanks
> rich
>
> 2008/12/5 Carlo Lazzaro <carlo.lazzaro@tiscalinet.it>:
>> Dear Richard,
>> I would refer you to Stata Archives and recommend to take a look at
> Maarten
>> Buis' and Martin Weiss' kind replies to one thread I have posted on 2008
> Sep
>> 6 (which was not so different from yours, as it would seem):
>>
>> st: z statistics in bootstrap output
>>
>> Kind Regards and enjoy your W-E,
>> Carlo
>> -----Messaggio originale-----
>> Da: owner-statalist@hsphsun2.harvard.edu
>> [mailto:owner-statalist@hsphsun2.harvard.edu] Per conto di Richard Harvey
>> Inviato: venerdì 5 dicembre 2008 18.40
>> A: statalist@hsphsun2.harvard.edu
>> Oggetto: st: bootstrapped p-values
>>
>> Hi,
>>
>> Could someone please explain how stata computes the bootstrap p-values?
>>
>> suppose i issue the following commands
>>
>> sysuse auto
>> bootstrap t=r(t), rep(1000)  saving(bsauto, replace): ttest mpg==0
>>
>> i get
>>
>>
>> Bootstrap results                                               Number
>> of obs=        74
>> Replications       =      1000
>>
>>
>> Observed   Bootstrap            Normal-based
>> Coef.   Std. Err.       z       P>z     [95% Conf. Interval]
>>
>> t    31.66644   2.900212        10.92   0.000     25.98213    37.35075
>>
>>
>> how does stata compute the P>z?
>>
>> does it...
>>
>> look at the proportion of bootstrapped t-values which are less than or
>> equal to the original t-value ( from
>> ttest mpg==0) say x and the proportion of bootstrapped t-values which
>> are greater than or equal to the original t-value ( from ttest mpg==0)
>> say y then computes p as 2 min(x,y) ?
>>
>> --
>> rich
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>>
>>
>
>
>
> --
> rich
>
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>

--