Notice: On March 31, it was **announced** that Statalist is moving from an email list to a **forum**. The old list will shut down at the end of May, and its replacement, **statalist.org** is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
"Ilian, Henry (ACS)" <Henry.Ilian@dfa.state.ny.us> |

To |
"'statalist@hsphsun2.harvard.edu'" <statalist@hsphsun2.harvard.edu> |

Subject |
RE: st: Bootstrapping question |

Date |
Fri, 8 Feb 2013 11:08:06 -0500 |

Maarten, This helps a great deal. Bootstrapping is a new idea for me. I saw it as a way to solve the problem of the fuzziness around the frequencies we are reporting, since people are making decisions based on those frequencies. If the proportion we report could be nearly 20 percentage points in either direction, then it doesn't tell us very much about what's going on in the real world. People might be doing poorly in an area reflected by the proportion, but they might also be doing well. Fortunately, we're supplementing the case-reading frequencies with focus groups among the people who work on the cases. If there is some agreement, then we can feel better about the numbers we're reporting, and the decision makers can feel more confident that they're making the right decisions. Right now, the process has the quality of fumbling around in the dark. I'm aware that people use small samples for many purposes, and if you didn't have small samples, you'd have no samples. I was lo! oking for a better way of saying what's going on. This may not be it. Henry -----Original Message----- From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Maarten Buis Sent: Friday, February 08, 2013 3:49 AM To: statalist@hsphsun2.harvard.edu Subject: Re: st: Bootstrapping question On Thu, Feb 7, 2013 at 10:28 PM, Ilian, Henry (ACS) wrote: > I looked at the table of contents. The book is clearly worth having, but it doesn't seem to cover the sample-size problem--which actually may not be a problem, since the sample size is what it is, and there isn't a way to make it any larger. By improved, I meant narrower, although that's such an obvious answer I don't think it was what you were asking me. If bootstrapping won't result in narrower confidence intervals, then I'll have to live with the confidence intervals as they are. It is not obvious that smaller confidence intervals represent an improvement. A confidence interval is based on a thought experiment: what if I could draw many new sample of the same size from my population and compute my statistic in each of these samples. Each of these statistics would be slightly different, as they are based on a different random sample from the population. The 95% confidence interval is an estimate of the interval within which 95% of these hypothetical statistics will be. This is an estimate of the uncertainty you have about your estimate, and the source of that uncertainty is the fact that you don't have the entire population but only a sample from that population. If you are unhappy about the size of that interval than the obvious way to reduce that is to increase the sample size. There are other cute ways of improving the precision of your estimate, e.g. stratified sampling, but don't expect too much from that: there is no way around the fact that a sample size of 27 is small and any estimate based on that sample size will be uncertain. If you say "improving the confidence interval", than that would mean to me making sure that the probability that the statistic computed on a random draw from the population falls within the 95% confidence interval is indeed 95%. This may seem trivial, but for many estimates of the confidence intervals this is not strictly true. Some confidence intervals are based on a computation that assumes an infinitely large sample and than the question becomes how large does the sample has to be before this approximation becomes reasonable. Improving the confidence interval would in that case mean some sort of adjustment that takes into account that you have a sample of finite size (which would typically increase the confidence interval rather than decrease it). For other problems the problem of computing confidence intervals is just very very hard and all existing estimates are approximate. The estimate of a proportion is a good example of that: if we have N observations, that our estimate of the proportion can only take one of N+1 possible values: 0/N, 1/N, 2/N, ..., or N/N. This discreteness makes the computation of an interval with exactly 95% coverage very hard. Paradoxically, the estimates of that interval that are called "exact" have far worse coverage than many approximate methods. The bootstrap confidence intervals can be said to be better at dealing with small samples in the sense that it tends to make fewer assumptions. It is not better in the sense that it will lead to smaller confidence intervals, that might or might not be the case depending on the type of violation of assumptions in the method with which you compared the bootstrap estimate. Hope this helps, Maarten --------------------------------- Maarten L. Buis WZB Reichpietschufer 50 10785 Berlin Germany http://www.maartenbuis.nl --------------------------------- * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/faqs/resources/statalist-faq/ * http://www.ats.ucla.edu/stat/stata/ Confidentiality Notice: This e-mail communication, and any attachments, contains confidential and privileged information for the exclusive use of the recipient(s) named above. If you are not an intended recipient, or the employee or agent responsible to deliver it to an intended recipient, you are hereby notified that you have received this communication in error and that any review, disclosure, dissemination, distribution or copying of it or its contents is prohibited. If you have received this communication in error, please notify me immediately by replying to this message and delete this communication from your computer. Thank you. * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/faqs/resources/statalist-faq/ * http://www.ats.ucla.edu/stat/stata/

**References**:**Re: st: Problem with standard errors using eststo***From:*Johan Hellström <johan.hellstrom@pol.umu.se>

**st: Bootstrapping question***From:*"Ilian, Henry (ACS)" <Henry.Ilian@dfa.state.ny.us>

**Re: st: Bootstrapping question***From:*Maarten Buis <maartenlbuis@gmail.com>

**Re: st: Bootstrapping question***From:*Nick Cox <njcoxstata@gmail.com>

**RE: st: Bootstrapping question***From:*"Ilian, Henry (ACS)" <Henry.Ilian@dfa.state.ny.us>

**Re: st: Bootstrapping question***From:*Nick Cox <njcoxstata@gmail.com>

**Re: st: Bootstrapping question***From:*Steve Samuels <sjsamuels@gmail.com>

**Re: st: Bootstrapping question***From:*Nick Cox <njcoxstata@gmail.com>

**RE: st: Bootstrapping question***From:*"Ilian, Henry (ACS)" <Henry.Ilian@dfa.state.ny.us>

**Re: st: Bootstrapping question***From:*Nick Cox <njcoxstata@gmail.com>

**RE: st: Bootstrapping question***From:*"Ilian, Henry (ACS)" <Henry.Ilian@dfa.state.ny.us>

**Re: st: Bootstrapping question***From:*Maarten Buis <maartenlbuis@gmail.com>

- Prev by Date:
**Re: st: Bootstrapping question** - Next by Date:
**st: Fwd: xtile: how to make it fast** - Previous by thread:
**Re: st: Bootstrapping question** - Next by thread:
**Re: st: Bootstrapping question** - Index(es):