Re: st: Power calculation for Beta/Odds Ratios in logistic regressionmodels

[Nico Hutter <Nico.Hutter@psychologie.uni-freiburg.de>]

Thanks Roger and Thanks Neil for the very helpful comments to our question!

Indeed we had two intentions:
First, we wanted to use post hoc power analysis for a further 
interpretation of non-significant odds ratios in an existing study. This 
approach is useless, what we have learned from your notes and references.
Second, we also tried to calculate the sample size for a "hypothetical" 
new study on the basis of the data of the existing study. This still 
seems to be reasonable.

Best wishes
Nico

Newson, Roger B schrieb:

> I would argue that it depends what you mean by "post hoc power
> calculations".
>
> If Nico is proposing to calculate the power to detect a POPULATION odds
> ratio of the size of the observed SAMPLE odds ratio, then that would
> indeed be misleading and uninformative. The Central Limit ?Theorem
> implies that the power to detect a POPULATION odds ratio the size of the
> observed SAMPLE odds ratio, using the observed SAMPLE P-value as the
> threshold, will always be 0.5 (or 50 percent). Therefore, it will not be
> surprising if the study was underpowered to detect a POPULATION odds
> ratio as large as the observed SAMPLE odds ratio, even though the study
> may have been adequately powered to detect a larger POPULATION odds
> ratio, which was considered to be clinically interesting by the study
> designers at the time of the study design.
>
> HOWEVER, if Nico intends to use an existing study to inform the design
> of a possible larger (and therefore more sensitive) study, using the
> existing study as a pilot, then Nico is doing nothing wrong. In fact,
> Nico is probably doing the best that anybody can do, given that the
> proposed analysis method appears to be confounder-adjusted, implying
> that doing power calculations from first principles will be very
> complicated. And there may be a genuine lack of consensus regarding what
> effect size is "clinically interesting".
>
> I hope this helps.
>
> Best wishes
>
> Roger
>
>
> Roger Newson
> Lecturer in Medical Statistics
> Respiratory Epidemiology and Public Health Group
> National Heart and Lung Institute
> Imperial College London
> Royal Brompton campus
> Room 33, Emmanuel Kaye Building
> 1B Manresa Road
> London SW3 6LR
> UNITED KINGDOM
> Tel: +44 (0)20 7352 8121 ext 3381
> Fax: +44 (0)20 7351 8322
> Email: r.newson@imperial.ac.uk 
> Web page: www.imperial.ac.uk/nhli/r.newson/
> Departmental Web page:
> http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/pop
> genetics/reph/
>
> Opinions expressed are those of the author, not of the institution.
>
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu
> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of uri goldbourt
> Sent: 15 August 2007 20:30
> To: statalist@hsphsun2.harvard.edu
> Subject: RE: st: Power calculation for Beta/Odds Ratios in logistic
> regression models
>
> Well and importantly said!
>
> UG
> ---------------------
>
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu
> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Neil Shephard
> Sent: Wednesday, August 15, 2007 9:08 AM
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: Power calculation for Beta/Odds Ratios in logistic
> regression models
>
> On 8/15/07, Nico Hutter <Nico.Hutter@psychologie.uni-freiburg.de> wrote:
>   
> > Hi everyone,
> >
> > we would like to run a power calculation for beta-coefficients / odds
> > ratios in logistic regression models with covariates.
> > Is such a procedure implemented in SATA?
> >
> > Used command for the logistic regression model:
> > svy: logit depression age sex comorbidity, or
> >
> > Data stems from a national representative survey. Dependent variable
> >     
> is
>   
> > depression. Comorbidity is dichotomised. Now, we are interested in the
> > "post hoc" power of the odds ratio of comorbidity.
> >
> > Can anybody give us some advice, please? Thanks in advance!
> >     
> Roger Newson has pointed to an appropriate solution, but I would
> question why you wish to do this?
>
> In my opinion "post hoc" power is a meaningless measurement that is
> completely useless in interpreting results, and a practice that needs
> to discouraged.
>
> You have already done your test (logistic regression) and got your
> results.  Knowing the 'power' your data had of detecting this size of
> effect (or more likely lack of) will tell you nothing more informative
> about the association.  Its like trying to tell someone who's just one
> the lottery that they shouldn't buy lottery tickets because the chance
> of winning is so low, they're not going to care as they've already got
> their answer.
>
> There are a few papers around that discuss this in greater detail (I'm
> sure there are more).
>
> Neil
>
> References
>
> Goodman SN, Berlin JA (1994) The Use of Predicted Confidence Intervals
> when Planning Experiments and the Misuse of Power When Interpreting
> Results. Annals of Internal Medicine 121.3:200-206
>
> Hoenig J.M., Heisey D.M. (2001) The Abuse of Power: The Pervasive
> Fallacy of Power Calculations for Data Analysis. The American
> Statistician  55:19-24
>
> Levine M, Ensom MH (2001) Post hoc power analysis: an idea whose time
> has passed? Pharmacotherapy 21.4:405-409
>
>   

--
Nico Hutter, Dipl. Psych.
Abt. f�r Rehabilitationspsychologie und Psychotherapie
Institut f�r Psychologie 
Universit�t Freiburg
D-79085 Freiburg
Tel: +49(0)761/203-9172
Fax: +49(0)761/203-3040
http://www.psychologie.uni-freiburg.de/Members/hutter

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/