I am doing an interrupted time-series analysis. A simplified
description of my model is as follows:
An intervention occurs on a specific date. The data fits two simple
linear models of response vs. date both before and after the
intervention, (with the slope and intercept coefficients before the
intervention being different from these coefficients after the
intervention). A first order autoregressive-moving average
(arima(1,0,1)) model fits the data quite well.
My questions have to do with dispersion. There is an abrupt, large and
persistent drop in the response variable that starts at the time of the
intervention. The dispersion after the intervention is substantially
less then in the baseline interval. My colleagues are interested in
assessing both the drop in the response variable as well as the
reduction in dispersion. They are used to using the coefficient of
variation to assess such changes.
Simple coefficients of variation are not ideal in this example because
there is some systematic variation in response with time in both the
baseline and post intervention intervals. What I have done so far is
the following:
1. Fit separate time series models to the baseline and post
intervention data.
2. Calculate the mean squared error (MSE) for both models using the
structural option of the predict command.
3. Divide the square root of the MSE by the mean response in either the
baseline or post intervention period. This is a coefficient of
variation like statistic that is adjusted for temporal trends within the
interval of analysis (i.e. trends within either the baseline or post
intervention interval). Doing this gives a 30% reduction in my
"adjusted" coefficient of variation.
4. The next issue is whether this reduction is statistically
significant. I have calculated res2, the squared residual for each day
divided by the squared mean for the interval (either baseline or post
intervention). I then entered res2 into a separate arima model. The
outcome shows a significant drop in res2, which I would like to
interpret as evidence that there is a significant drop in my "adjusted"
coefficient of variation after the intervention.
Questions
My fundamental question is does the preceding make sense and is there a
better way of doing things? My specific questions are
1. Is there a better way of fitting my original model? My original
model does a good job at dealing with the temporal correlation of my
response variable and produces an expected response curve that fits the
data well. However, it also assumes a constant white noise error term
that is clearly not correct. I could, of course, transform the data to
obtain more homoscedastic errors but I am concerned that this would
result is a less satisfactory estimate of the expected response. Is
there a way that I can allow for separate residual errors before and
after the intervention without giving up a linear model of the response
as a function of time?
2. A problem with my approach to assessing the significance of the drop
in adjusted coefficient of variation is that my res2 variable is both
highly skewed and temporally correlated. If I use an arima model to
deal with the temporal correlation, I am assuming a white noise term
that is normally distributed. This assumption is clearly incorrect.
However, I do not know of a nonparametric method that can account for
the autoregressive-moving average nature of my data. What do you
recommend? (This problem is made more acute by the fact that my P value
comparing adjusted coefficients of variation is only 0.03, and its
statistical significance might easily vanish if a more appropriate model
was used.)
3. Can anyone suggest references that might be helpful? In particular,
can anyone tell me of a reference for calculating a coefficient of
variation adjusted for within-group systematic trends?
***************
I apologize for writing such a long query. I would be most grateful for
any insights on how to analyze this sort of problem.
Bill Dupont
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
