Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: Adjusted R-squared comparison


From   Panagiotis Manganaris <mangan@econ.auth.gr>
To   statalist@hsphsun2.harvard.edu
Subject   st: Adjusted R-squared comparison
Date   Wed, 06 Feb 2013 13:57:43 +0200

Unfortunately Nick and John, I must use adj r-squared because it represents a specific metric in the field of accounting. More specifically, I use a model where returns are the dependent variable and earnings, along with the change in earnings, are the independent variables. In this model the adjusted r-squared represents the value relevance of the earnings (this is what I am trying to gauge). Therefore, I am obliged to use r2. Thank you for the procedure you mention John, but I had already tried it in the past. It is helpful, but only in a vague way. It does not provide the mean and the variance of r2, so I could use them to test the significance. For instance, the intervals almost always overlap when I use this method. That does not provide concrete evidence of statistical significance or non-significance. If I don't prove that there is (or there is not) a statistically significant difference, I cannot show whether my metric (value relevance) has been altered between the two periods.



2013/2/6 John Antonakis <John.Antonakis@unil.ch>
Can't agree more with you Nick. We should care more about having consistent estimators than high r-squares (i.e., Panagiotis, what I mean here is that we can still estimate the slope consistently even if we don't have a tight fitting regression line). So, I don't know why you are interested in this comparison, Panagiotis. I would think you would be more interested in comparing estimates, as in a Chow test (Chow, G. C. (1960). Tests of equality between sets of coefficients in two linear regressions. Econometrica, 28(3), 591-605). If you are using fixed-effects models, you can model the fixed-effects with dummies and then do a Chow test via suest....see -help suest-.


Best,
J.

__________________________________________

John Antonakis
Professor of Organizational Behavior
Director, Ph.D. Program in Management

Faculty of Business and Economics
University of Lausanne
Internef #618
CH-1015 Lausanne-Dorigny
Switzerland
Tel ++41 (0)21 692-3438
Fax ++41 (0)21 692-3305
http://www.hec.unil.ch/people/jantonakis

Associate Editor
The Leadership Quarterly
__________________________________________

On 06.02.2013 11:40, Nick Cox wrote:
That's positive advice.

My own other idea is that adjusted R-squares are a lousy basis to
compare two models, even of the same kind. They leave out too much
information.

Nick

On Wed, Feb 6, 2013 at 10:37 AM, John Antonakis <John.Antonakis@unil.ch> wrote:
I think that the only think you can do is to bootstrap the r-squares and see
if their confidence intervals overlap.

To bootstrap you just do:

E.g.,

sysuse auto
bootstrap e(r2), seed(123) reps(1000) : reg price mpg weight

You will be interested in either:

       e(r2_w)             R-squared within model
       e(r2_o)             R-squared overall model
       e(r2_b)             R-squared between model

See help xtreg with respect to saved results.

Let's see if others have other ideas.
On 06.02.2013 10:22, Panagiotis Manganaris wrote:

I need to compare two adjusted r-squared of the same model for two
different periods of time (each one spans for a period of years). So far, I
have split my data in two groups, those that belong to the period 1998-2004
and those that belong to the period 2005-2011. Then I used xtreg on the same
model for each group of data. I've derived their adjusted r-squared and I
want to know if those two adjusted r-squared are significantly different
from each other.
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index