[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

From |
Joseph Coveney <jcoveney@bigplanet.com> |

To |
Statalist <statalist@hsphsun2.harvard.edu> |

Subject |
Re: st: -sampsi- command and exact tests |

Date |
Wed, 02 Feb 2005 17:29:29 +0900 |

David Miller wrote: I understand from several previous posts that the command -sampsi- uses an approximate large sample test on proportion. power, and sample size calculations. Specifically, it uses the normal approximation (with correction) as opposed to an exact test. The advice in a previous post was that the following equalities must hold in order for sampsi to work adequately: n1p1>=10 n1(1-p1)>=10 n2p2>=10 n2(1-p2)>=10 (see post from ymarchenko@stata.com entitled st:RE: calculation of sample size, dated 8 Oct 2004) I am trying to use sampsi to estimate the required number of samples as follows: sampsi 0.4 0.46, alpha(0.05) power(0.90) onesample Stata indicates that 711 samples are required as indicated in the Stata output below: sampsi 0.4 0.46, alpha(0.05) power(0.90) onesample Estimated sample size for one-sample comparison of proportion to hypothesized value Test Ho: p = 0.4000, where p is the proportion in the population Assumptions: alpha = 0.0500 (two-sided) power = 0.9000 alternative p = 0.4600 Estimated required sample size: n = 711 This seems to meet the n1p1>=10 etc. requirements listed above to use the -sampsi- command. However, I am told that the right answer using NQuery Advisor and its exact test for single proportions is 610 observations. S-Plus gives 613 as an answer. StatXact also gives a similar answer to NQuery Advisor and S-Plus. Does anyone know if Stata's use of the normal approximation (with continuity correction) is indeed what is causing the 100+ discrepancy here? Is there is an exact test in Stata that can be used instead of -sampsi-? And are there additional criteria beyond the n1p1>=10 etc. criteria listed in the referenced previous post that should be checked before using -sampsi- ? I have used the -findit- command to see if there is an exact test available and looked at both Roger Newson's -powercal- command described in the most recent Stata Journal (4th Quarter 2004) as well as Al Feiveson's article entitled "Power by Simulation" (Stata Journal, 2nd Quarter 2002) and wasn't able to find the answer to this question. Is the -sampncti- command appropriate here? ---------------------------------------------------------------------------- In simulations, using both Wilson's score method in -ciw- and the exact binomial test in -bitest- (if I understand its output correctly), it seems that Stata is on the mark and NQuery Advisor, S-Plus or StatXact are not. (See do-file below.) A sample size of 600 give about 80 to 85% power for a two-sided test; 700 gives around 90%. Output from the do-file below: Sample size: 600, Power: Wilson = 0.85 Exact = 0.83 Sample size: 650, Power: Wilson = 0.88 Exact = 0.88 Sample size: 700, Power: Wilson = 0.90 Exact = 0.90 Sample size: 750, Power: Wilson = 0.91 Exact = 0.91 So: 1. It doesn't appear that Stata's use of the normal approximation accounts for the discrepancy. 2. Yes; there is an exact test available in Stata. 3. I don't know of any additional criteria to check first, other than whether the hypothesis pair is directional. 4. -samplncti- wouldn't be appropriate here. (But I've read where power calculations for binomial data have been done with the Student's t test statistic in simulations, and it appears to be pretty good for that purpose.) Joseph Coveney clear set more off set seed `=date("2005-02-02", "ymd")' set seed0 `=date("2005-02-02", "ymd")' local reps = 10000 set obs `reps' generate byte Wilson = . generate byte Exact = . generate float pi1 = 0.46 forvalues den = 600(50)750 { generate int den = `den' quietly rndbinx pi1 den quietly compress forvalues i = 1/`reps' { local successes = bnlx[`i'] quietly ciwi `den' `successes' quietly replace Wilson = (r(lb) > 0.4) in `i' quietly bitesti `den' `successes' 0.4 quietly replace Exact = (r(p) < 0.05) in `i' } summarize Wilson, meanonly local Wilson = r(mean) summarize Exact, meanonly local Exact = r(mean) display display in smcl as result "Sample size: `den', Power: Wilson = " /// %4.2f `Wilson', "Exact = " %4.2f `Exact' drop den bnlx } exit * * For searches and help try: * http://www.stata.com/support/faqs/res/findit.html * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

- Prev by Date:
**st: Teething troubles in using -gen, mod()-** - Next by Date:
**RE: st: Confidence limits on survival probabilities from stcox** - Previous by thread:
**st: RE: -sampsi- command and exact tests** - Next by thread:
**st: ipshin** - Index(es):

© Copyright 1996–2016 StataCorp LP | Terms of use | Privacy | Contact us | What's new | Site index |