Re: st: Binomial confidence intervals

[Constantine Daskalakis <C_Daskalakis@mail.jci.tju.edu>]

At 08:03 AM 9/8/2004, Richard Williams wrote:
> At 11:47 AM 9/8/2004 +0100, Paul Seed wrote:
> > Dear all,
> >
> > As I don't have access to a decent Stats library here, I tried to obtain
> > the recommended paper (Brown, Cai, & DasGupta.
> > Interval Estimation for a Binomial Proportion. Statistical Science, 2001,
> > 16, pp. 101-133.)  over the internet; but it is currently behind a "rolling
> > firewall", until 2005.
> A few key quotes:
>
> p. 115 - "Based on this analysis, we recommend the Wilson or the 
> equal-tailed Jeffreys prior interval for small n (n is less than or equal 
> to 40). These two intervals are comparable in both absolute error and 
> length for n is less than or equal to 40, and we believe that either could 
> be used, depending on taste."
>
> p. 115 - "For larger n, the Wilson, the Jeffreys and the Agresti�Coull 
> intervals are all comparable, and the Agresti�Coull interval is the 
> simplest to present....we recommend the Agresti�Coull interval for 
> practical use when n is greater than or equal to 40. Even for small sample 
> sizes, the easy-to-present Agresti�Coull interval is much preferable to 
> the standard one."
>
> p. 113 - "The Clopper�Pearson interval is wastefully conservative and is 
> not a good choice for practical use."
The controversy can be summarized like this:

A. Suppose I have a true probability of success P that I am trying to 
estimate with a sample of size N. If I draw 1 million samples and compute 
my interval, I want at least 95% of those to cover the true P. I want this 
to happen, irrespective of the true value of P and the sample size N (ie, 
100% of the time). So, if you have a different situation with probability 
P* and sample size N*, you should also have coverage of 95% or better. In 
other words, this means that I want an interval with coverage AT LEAST 95% 
ALL THE TIME (for some situations, it will have more).

B. In contrast, I may want the 95% coverage to be "on average," across 
different situations. In other words, in some situation of P and N, I am 
willing to accept coverage less than 95%, while for some other situation P* 
and N*, I will have coverage better than 95%. All I want is my interval to 
give at least 95% coverage ON AVERAGE.

(A) leads us to conservative intervals such as Clopper-Pearson. They have 
to cover the truth 95% of the time no matter what the situation. Even a 
single situation where they fall to 94.9% is not acceptable. However, they 
need not be as conservative as Clopper-Pearson. The Blyth-Still-Casella is 
much better (ie, it stays at 95% or better no matter what, but does not 
become as conservative as Clopper-Pearson) and is very competitive with 
those advocated in Agresti & Coull and Brown et al.

(B) leads us to generally shorter intervals, but in some situations they 
fall below 95% (sometimes way below that). Brown et al. propose certain 
"corrections" to remove the most severe dips in coverage probability. 
Still, none of those intervals is guaranteed to have 95% coverage or better in
a particular situation. And the problem is we cannot be sure how much below 
95% they can fall. All we know is that they will have 95% coverage on average.

I go with A. I think that B is a bit ad hoc and a dip below 95% may be 
acceptable to me but unacceptable to you. But either approach seems quite 
defensible.





The documents accompanying this transmission may contain confidential 
health or business information. This information is intended for the use of 
the individual or entity named above. If you have received this information 
in error, please notify the sender immediately and arrange for the return 
or destruction of these documents.

________________________________________________________________
Constantine Daskalakis, ScD
Assistant Professor,
Biostatistics Section, Thomas Jefferson University,
211 S. 9th St. #602, Philadelphia, PA 19107
   Tel: 215-955-5695
   Fax: 215-503-3804
   Email: c_daskalakis@mail.jci.tju.edu
   Webpage: http://www.jefferson.edu/medicine/pharmacology/bio/  

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/