Re: st: Statistically significant difference in R Squared

[Richard Williams <richardwilliams.ndu@gmail.com>]

At 08:15 AM 8/24/2012, Christopher Baum wrote:
> <>
> On Aug 24, 2012, at 2:33 AM, Ani wrote:
>
> > I am running a time-series regression model over a 10 year period. I would
> > be interested in splitting the sample into two five year periods 
> and finding
> > out whether the model has a statistically significantly higher R^2 during
> > the second period as compared to the first. Is there a test for 
> this, and if
> > so is it possible to implement in Stata?
>
> I don't think this is a very well posed question. In the example 
> below, -robvar- applied to the residuals of a single model
> shows that the forecast performance of the model deteriorates after 
> 1987q3. Running separate subperiod regressions confirms this:
> the model produces more accurate forecasts in the earlier period. 
> Yet the R^2 is higher in the later subperiod! I would be more
> interested in the model's accuracy, in terms of forecast confidence 
> intervals, than I would in R^2.
>
> use http://fmwww.bc.edu/cfb/data/usmacro1,clear
> reg d.cpi d.oilprice d.wage, robust
> predict double res if e(sample)
> g break = (tin(1987q3,))
> robvar res, by(break)
> reg d.cpi d.oilprice d.wage if !break, robust
> reg d.cpi d.oilprice d.wage if break, robust

I would add that R^2 can be affected by many things. A larger R^2 
could be due to a bigger structural coefficient in one group, but it 
could also be due to things like differences in the exogenous 
variances. A simple discussion of this can be found at

http://www.nd.edu/~rwilliam/xsoc63993/l72.pdf


-------------------------------------------
Richard Williams, Notre Dame Dept of Sociology
OFFICE: (574)631-6668, (574)631-6463
HOME:   (574)289-5227
EMAIL:  Richard.A.Williams.5@ND.Edu
WWW:    http://www.nd.edu/~rwilliam

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/