Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: Comparing change in rates - frustrating problem, please help


From   Joseph Coveney <jcoveney@bigplanet.com>
To   Statalist <statalist@hsphsun2.harvard.edu>
Subject   Re: st: Comparing change in rates - frustrating problem, please help
Date   Sun, 01 Feb 2004 21:31:04 +0900

Kieran McCaul posted results from a randomized parallel-group design study to 
illustrate the use of conditional logistic regression.  The study randomized 
households to an intervention designed to promote banning of smoking in the 
home.  Policy in the home was measured before and after intervention.  Kieran 
invited Ricardo and I to respond with what we think of advocating conditional 
logistic regression to assess the efficacy of the intervention for before-and-
after studies based upon the results posted for that study.

I don't claim to speak for Ricardo, but his original question related to 
imbalances in the baseline rates of the outcome between the two parallel 
intervention groups.  It appears that Kieran's study was successful in its 
randomization (or used stratified randomization and didn't lose too many 
households to dropout), because the proportions of households banning smoking 
at baseline were nearly identical between the intervention groups.  With 
essentially identical rates of baseline, there would be little or no cause for 
concern about confounding due to it and little statistical difference in 
including baseline as a covariate.  And, in fact, both conditional logistic 
regression approach and the so-called ANCOVA-like multiple logistic regression 
approach give essentially similar results in this balanced study.  (I think the 
same would have obtained for Ricardo's study had the baseline rates of seatbelt 
use been similar between the two intervention groups.)

But, let's look at the issue of which approach is more suitable when the 
concern is, as it was for Ricardo, to analyze an intervention effect _in the 
face of an imbalance in the baseline rates of an outcome_.

If Kieran will indulge me one more time to use a fictional dataset to 
illustrate a point, let's say that Kieran's randomization method did not 
stratify on baseline household smoking policy, and suffered an unfortunate 
imbalance due to chance, for instance a 50 : 50 ratio of households banning 
smoking at baseline in the nonintervention group, but a 75 : 25 ratio in the 
intervention group.  Let's say that 2 of the 50 households that previously 
banned smoking in the nonintervention group now permit it, a worsening of 4% 
(if your health policy is to ban smoking), and that only 1 of the 50 households 
that didn't ban smoking now do so in the nonintervention group, a meager 
improvement of 2%.  Let's say that 4 of the 75 households that banned smoking 
at baseline switched and permitted smoking in the home after the intervention, 
and 2 of the 25 households that didn't ban smoking switched as a result of the 
intervention.  The results of the intervention are a slightly greater 5.3% 
worsening (compare to 4%) in the former nonbanning household population, but a 
much greater 8% (compare to 2%) improvement among the formerly permissive 
households.  

Now, the effects of intervention are no great shakes, but I think that it would 
be safe to say that it's not *nothing*, especially if you somehow take into 
account the possible confounding effect of the chance unfortunate imbalance in 
baseline policy between treatment groups.

But, by the conditional logistic regression approach, it *is* nothing--the odds 
ratio for both nonintervention and intervention groups is 0.5 (McNemar's test 
uses only the off-diagonal values and ignores the diagonal values) so the ratio 
of the two odds ratios is 1.0, and this is what the conditional logistic 
regression dutifully reports:  the period term is 0.5 and the interaction 
term's odds ratio is 1.0 with a Z-statistic of 0.00 and a p-value of 1.00.  
Granted, the confidence interval encompasses a lot, but the point estimate and 
hypothesis test for the interaction term (which is ostensibly the effect of 
intervention) just don't give the same take-home message as inspection of the 
data.  So, my conclusion differs from Kieran's on this; I don't think that 
conditional logistic regression is valid to test for differences between 
treatment effects (differences between treatment differences, which are between-
subject effects) in parallel-group designs with a repeated binary outcome 
measure, especially in the presence of baseline differences in the outcome 
measure, which are ignored in the conditional logistic model.

In contrast, the ANCOVA-like, baseline-as-covariate multiple regression 
approach does provide a separate, and I think competent, handling of baseline 
differences and their potential for confounding.  In the fictitious example, 
this approach shows the pronounced effect of baseline smoking policy as 
expected, and it shows that the odds ratio for intervention isn't 1.0 given 
baseline differences between intervention groups.  The saturated model (with 
the interaction term) also helps to put the potential for confounding into 
perspective.  (The do-file for all of this is below for anyone interested.)

It seems that at least some of the discrepancy between the two approaches 
reflects Simpson's paradox.  This is the same underlying phenomenon that 
results in bias in logistic regression coefficients (and in nonlinear 
regression, in general) when important covariates are left out of the model.  
This is what Frank E. Harrell Jr.'s lecture dealt with in the URL given in my 
last posting.  And it relates to the "noncollapsibility of odds ratios" that 
epidemiologists sometimes refer to.

In fairness to us all (Kieran, Ricardo and me), it seems that the matter of 
which approach is better isn't completely settled even for *linear* models, 
where this incollapsibility-of-odds-ratios phenomenon and the incidental 
parameters problem don't apply:  there is a thread ("Repeated measures and 
including time zero response as baseline covariate") on sci.stat.consult that 
was started on May 7 of last year by Frank Harrell.  Professor Harrell wrote a 
well received book on regression modeling and is now chairman of a department 
of biostatistics, yet even he asks, "Has anyone come across some practical 
guidance for when to include the first measured response (at time zero) as a 
baseline covariate as opposed to the first repeated measurement in a 
longitudinal data analysis?"

Joseph Coveney

-------------------------------------------------------------------------------

clear
tempfile tmp
set obs 100
generate byte ban0 = _n > _N / 4
generate byte ban1 = ban0
replace ban1 = !ban1 in 50/53
replace ban1 = !ban1 in 1/2
*
* Intervention group
*
display 4 / 75  // switching by banners
display 2 / 25  // switching by permitters
mcc ban1 ban0
generate byte intervention = 1
save `tmp'
clear
set obs 100
generate byte ban0 = _n > _N / 2
generate byte ban1 = ban0
replace ban1 = !ban1 in 50/52
*
* Nonintervention group
*
display 2/50  // switching by banners
display 1/50  // switching by permitters
mcc ban1 ban0
generate byte intervention = 0
append using `tmp'
erase `tmp'
generate byte iac = ban0 * intervention
generate int id = _n
logistic ban1 ban0 intervention iac, or nolog
estimates store A
logistic ban1 ban0 intervention, or nolog
estimates store B
lrtest A B
logistic ban1 ban0, or nolog
lrtest A .
lrtest B .
quietly {
    reshape long ban, i(id) j(period)
    replace iac = period * intervention
}
clogit ban period intervention iac, group(id) or nolog
xtgee ban period intervention iac, i(id) family(binomial) link(logit) ///
  corr(exchangeable) nmp eform nolog
exit


*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index