Re: st: MANCOVA versus Zellner's SUR

[Phil Schumm <pschumm@uchicago.edu>]

On Nov 4, 2007, at 8:56 PM, Joseph Coveney wrote:
> Thanks for your reply, Phil.  The objective of the study is to  
> determine whether there is a difference between treatments with  
> respect to outcome as measured by the response vector.
>
> There is one other covariate, a categorical one with two levels, on  
> which randomization is stratified.  Values for that variable, too,  
> are expected to correlate, albeit weakly, with values of the  
> response variables, hence, the stratified randomization.  Both - 
> manova- and -sureg- can readily accommodate the stratification  
> variable:
>
> manova L1 R1 = trt L0 R0 Stratum, category(trt Stratum) sureg (L1 =  
> trt L0 Stratum) (R1 = trt R0 Stratum), isure small dfk
>
> The model will not contain any interaction terms.  I didn't think  
> that this stratification covariate is relevent to the matter of  
> using baseline values of the two response variables as covariates,  
> and so didn't mention it.
>
> The option that you mention, pre-to-post differences in Hotelling's  
> T-squared test, is identical to the MANOVA at the bottom of my post  
> (the MANOVA model for which time-by-treatment interaction was  
> mentioned).  It suffers from a fairly large decrease in efficiency  
> (statistical power) vis-à-vis the corresponding MANCOVA and SUR  
> models described earlier in the post.

Sorry -- I didn't look at your last set of commands carefully enough  
to see that they were equivalent to a multivariate comparison of  
change scores (i.e., the same as I suggested).

Focusing for a moment on just one outcome, your options are  
essentially (1) an analysis of the change scores, or (2) regression  
of the outcome on the treatment indicator and the baseline value.   
Option (2) can give inconsistent estimates of the treatment effect  
due to measurement error in y_1 unless your treatment assignment is  
randomized (e.g., Allison 1990), which you indicated is the case  
here.  The advantage of (2) is that it is more efficient (as you  
point out), and yields a result that is often of direct interest  
(i.e., the difference between treatment groups in the mean value of  
y_2 for a given *observed* value of y_1).  There was a thread in the  
American Statistician on these issues a while back; Laird (1983) is a  
good entry point.

Once you have a model for each outcome that you feel comfortable  
with, you can then think about estimating them jointly either to  
increase efficiency and/or to permit joint tests (e.g., to construct  
a single test of treatment effect for both outcomes).  Certainly - 
sureg- provides one reasonable approach for doing this.  Your other  
approach -- multivariate regression in which both outcomes are  
regressed on both sets of baseline values -- strikes me as  
unjustifiable, unless you are really interested in the effects of the  
baseline value of one measure on the post-treatment value of the  
other.  Of course, if you are really interested in this, then you  
also need to consider the effect that measurement error in the  
baseline values may have on your analysis.

Personally, whenever I've been faced with an analysis of pre/post  
data, I've always started by considering several specific models for  
the measurement process and for the effect(s) of the treatment (e.g.,  
homogeneous versus heterogeneous, dependent on the baseline value of  
the outcome, etc.), and tried to figure out what the implications of  
these were for different analyses.  There's a limit to what you can  
do in terms of estimation with only a single pre and post  
measurement, of course, but I have still found this exercise to be  
helpful.  The papers cited here (and their references) provide  
several good examples of what I am talking about.


-- Phil

P. D. Allison. Change scores as dependent variables in regression  
analysis. Sociological Methodology, 20:93–114, 1990.

N. Laird. Further comparative analyses of pretest-posttest research  
designs. The American Statistician, 37(4):329–330, 1983.


*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/