[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

From |
Tim Wade <wade_tj@yahoo.com> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: st: general statistical reasoning question in biomedical statistics (no Stata content) |

Date |
Thu, 11 Dec 2003 10:40:10 -0800 (PST) |

Here are my 2 cents: It is my understanding that now p-values are often discouraged from "Table 1" at least in part for the reason that you describe. I think in part this is a confusion of validity vs. precision. What do you mean by your randomization is "solid"? Randomization *does not* (and can never) guarantee a completely balanced sample. So if there is convincing evidence, qualitiative or quantitaitve, that the two arms are not the same then it does call into question the equivalence of the two groups. A perfectly randomized study could produce by chance two very different groups (although it is not likely to do so) and thus call into question the validity of the results, but would not necessarily invalidate the results. Especially if numerous factors were compared in Table 1, there may have been a difference simply by chance, in two groups that are actually equivalent. Tim --- "Christopher W. Ryan" <cryan@binghamton.edu> wrote: > Having read the Statalist FAQ, and previous > correspondence about general > statistical questions, I hope no one minds . . . . > > Among my teaching duties in my medical school and > family practice > residency is "critical appraisal of the medical > literature." I try to > go over principles of good design and valid > analysis. A question > frequently comes up when we discuss randomized > controlled trials. In > these articles, there is almost always a "Table 1," > that describes the > baseline demographic and clinical variables of the > two arms (say, > placebo and active drug, for example.) There are > usually *a lot* of > baseline measurements. Each one is usually listed > with a "P value," > indicating whether the placebo and active drug > subjects differed on that > measurement. > > Then the manuscript goes on to describe the rest of > the study, and the > results . . . > > If the results show an advantage for the active > drug, readers (including > my students and residents) will often go back to > "Table 1" and say, "Oh > but look, the samples were not identical. Blah-blah > was significantly > higher in the placebo arm to begin with. Therefore > I can't accept these > results as valid." > > I've never agreed with that. So I want to outline > my chain of reasoning > here and see if I've got it straight. > > There are two premises in a randomized controlled > trial with two arms: > > 1. The two samples are drawn randomly from the same > population > 2. The active drug actually has no effect (the null > hypothesis) > > And then there are the results (R). > > If 1 and 2 are both true, we can look at R and > calculate how likely we > were to see results that "extreme" or more so. > That's the P value. If > P < the conventional 0.05, we say, "Gee, if 1 and 2 > are both true, we > *might* have seen results R, but only 5% of the time > or less, and that's > pretty unlikely. But we *did* see R. Therefore > either 1 or 2 must be > untrue. And I'm confident my randomization was > solid. Therefore 2 must > be untrue, and the drug really does have an effect." > > There is nothing this chain of reasoning that > requires the samples to be > indentical/indistinguishable. And for every 20 > baseline variables > compared, you'd *expect* about 1 of those baseline > variables to have a P > of < 0.05 The statistical techniques have > "built-in" accomodation for > this. This does not invalidate the conclusions. > > It is a difficult concept for my learners to grasp. > Or maybe I've got > it wrong? > > Thanks. > > --Chris Ryan > > * > * For searches and help try: > * > http://www.stata.com/support/faqs/res/findit.html > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ ===== wade_tj@yahoo.com __________________________________ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree * * For searches and help try: * http://www.stata.com/support/faqs/res/findit.html * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**Re: st: Problem with --update ado--***From:*"R. Allan Reese" <R.A.Reese@hull.ac.uk>

**st: Problem with --update ado--***From:*"Jeffrey Quon [UBC]" <quon@interchange.ubc.ca>

**References**:**st: general statistical reasoning question in biomedical statistics (noStata content)***From:*"Christopher W. Ryan" <cryan@binghamton.edu>

- Prev by Date:
**Re: st: general statistical reasoning question in biomedical statistics(no Stata content)** - Next by Date:
**Re: st: general statistical reasoning question in biomedical statistics(no Stata content)** - Previous by thread:
**st: RE: Ordered logit - Fixed effects** - Next by thread:
**st: Problem with --update ado--** - Index(es):

© Copyright 1996–2023 StataCorp LLC | Terms of use | Privacy | Contact us | What's new | Site index |