Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

st: RE: interim analysis in epidemiologic studies


From   "Steichen, Thomas" <STEICHT@rjrt.com>
To   <statalist@hsphsun2.harvard.edu>
Subject   st: RE: interim analysis in epidemiologic studies
Date   Thu, 7 Aug 2003 10:53:25 -0400

Ricardo Ovaldia writes:
 
> I know that interim analysis are used in clinical
> trials often with a "stopping rule" so that a trial
> can be stopped if participants are being put at risk
> unnecessarily. However, I am uncertain about the
> effect of multiple looks of the data on the p-values
> of the final analysis in epidemiologic studies such as
> case-control studies. Two scenarios come to mind:
> 
> 1. A case control study in which we are enrolling a
> fix number of subjects (say 100 cases and 100
> controls). If I compute odds ratio when I have
> enrolled 25%, 50% and 75% of subjects does that affect
> my p-value when I have 100% of the subjects? I do not
> think it does because I set my final sample size to a
> given value at the beginning.
> 
> 2. Assume that now my sample size is not fixed but I
> decide to enroll patients for a fixed amount of time,
> say a year. How do the interim analysis affect the
> final p-values?
> 
> Note that I am not using a stopping rule in either
> case. Your thoughts will be much appreciated.

In my opinion for the scenarios you defined, I do not believe a
preliminary look at the results would bias or invalidate either
the interpretation or the apparent p-value "level" of the final 
statistics.  In both cases, you have a predefined limit
(on # of cases & controls in 1. and on the case ascertainment
time period in 2.) that is unaffected by your peek at the 
data, and you suggest that you will not change any of the 
enrollment rules based on this look at the data.

On the other hand, I have seen situations where researchers
have used a preliminary look at the data to justify increasing
their desired case count or ascertainment time period (with the 
intent of increasing the total case count) because they did not 
believe they would obtain "significance" with the original plan. 
Such a procedure, as we all know, totally invalids the final
statistics, since any difference will eventually become 
"significant" if we simply add enough data.  This was a study
where the researchers had performed a power analysis during the 
planning stage to determine the needed sample size to have 
reasonable power to detect the odds ratio they believed would
be important to find (1.75) if it existed; they eventually reported
a "significant" 1.29 based on a much-expanded study.  

I have also seen a preliminary look used to modify the eligibility
requirements for participants in order to increase the number
of cases during the specified enrollment period.  Unfortunately
the eligibility modifications differentially affected cases vs. 
controls, which led to a bias that was difficult to characterize 
but probably caused an increased likelihood of the results being 
declared "significant."

Both of the above situations occurred in large studies funded by 
the U.S. Public Health Service, were led by researchers supposedly 
above reproach, and involved health endpoints where "big brother" 
knew what the "answer" was supposed to be (for our own good!).

Tom


-----------------------------------------
CONFIDENTIALITY NOTE:  This e-mail message, including any
attachment(s), contains information that may be confidential,
protected by the attorney-client or other legal privileges, and/or
proprietary non-public information.  If you are not an intended
recipient of this message or an authorized assistant to an intended
recipient, please notify the sender by replying to this message and
then delete it from your system.  Use, dissemination, distribution,
or reproduction of this message and/or any of its attachments (if
any) by unintended recipients is not authorized and may be unlawful.


*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index