Ricardo Ovaldia writes:
> I know that interim analysis are used in clinical
> trials often with a "stopping rule" so that a trial
> can be stopped if participants are being put at risk
> unnecessarily. However, I am uncertain about the
> effect of multiple looks of the data on the p-values
> of the final analysis in epidemiologic studies such as
> case-control studies. Two scenarios come to mind:
>
> 1. A case control study in which we are enrolling a
> fix number of subjects (say 100 cases and 100
> controls). If I compute odds ratio when I have
> enrolled 25%, 50% and 75% of subjects does that affect
> my p-value when I have 100% of the subjects? I do not
> think it does because I set my final sample size to a
> given value at the beginning.
>
> 2. Assume that now my sample size is not fixed but I
> decide to enroll patients for a fixed amount of time,
> say a year. How do the interim analysis affect the
> final p-values?
>
> Note that I am not using a stopping rule in either
> case. Your thoughts will be much appreciated.
In my opinion for the scenarios you defined, I do not believe a
preliminary look at the results would bias or invalidate either
the interpretation or the apparent p-value "level" of the final
statistics. In both cases, you have a predefined limit
(on # of cases & controls in 1. and on the case ascertainment
time period in 2.) that is unaffected by your peek at the
data, and you suggest that you will not change any of the
enrollment rules based on this look at the data.
On the other hand, I have seen situations where researchers
have used a preliminary look at the data to justify increasing
their desired case count or ascertainment time period (with the
intent of increasing the total case count) because they did not
believe they would obtain "significance" with the original plan.
Such a procedure, as we all know, totally invalids the final
statistics, since any difference will eventually become
"significant" if we simply add enough data. This was a study
where the researchers had performed a power analysis during the
planning stage to determine the needed sample size to have
reasonable power to detect the odds ratio they believed would
be important to find (1.75) if it existed; they eventually reported
a "significant" 1.29 based on a much-expanded study.
I have also seen a preliminary look used to modify the eligibility
requirements for participants in order to increase the number
of cases during the specified enrollment period. Unfortunately
the eligibility modifications differentially affected cases vs.
controls, which led to a bias that was difficult to characterize
but probably caused an increased likelihood of the results being
declared "significant."
Both of the above situations occurred in large studies funded by
the U.S. Public Health Service, were led by researchers supposedly
above reproach, and involved health endpoints where "big brother"
knew what the "answer" was supposed to be (for our own good!).
Tom
-----------------------------------------
CONFIDENTIALITY NOTE: This e-mail message, including any
attachment(s), contains information that may be confidential,
protected by the attorney-client or other legal privileges, and/or
proprietary non-public information. If you are not an intended
recipient of this message or an authorized assistant to an intended
recipient, please notify the sender by replying to this message and
then delete it from your system. Use, dissemination, distribution,
or reproduction of this message and/or any of its attachments (if
any) by unintended recipients is not authorized and may be unlawful.
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
