Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
David Hoaglin <[email protected]> |

To |
[email protected] |

Subject |
Re: st: Relative Importance of predictors in regression |

Date |
Tue, 5 Nov 2013 08:25:51 -0500 |

Hi, Sam. No "wordplay" is involved. Regression coefficients are a type of slope, so it is natural to talk about "increase" and "change," which most people would consider to be continuous, whereas "difference" often comes across as discrete. Earlier, when I mentioned subspaces and projections, I had in mind a description based on linear algebra. Since you teach regression, you should be familiar with partial-regression plots (also called added-variable plots), and I can use that setting to describe the main idea. Suppose Y = b1X1 + b2X2 + ... + bpXp. The observed values of Y usually include disturbances (or noise or errors), but I am using the true values of Y in order to describe the interpretation of the true regression coefficients. The description applies with minor modifications when Y includes noise. I have not numbered the predictors from 0, so X1 will often be the constant predictor, and I will focus on b2. Regress Y on X1, X3, ..., Xp, producing the fitted regression c1X1 + c3X3 + ... + cpXp and the residuals Y - (c1X1 + c3X3 + ... + cpXp). This step adjusts Y by removing the contributions of X1, X3, ..., Xp. Regress X2 on X1, X3, ..., Xp, producing the fitted regression a1X1 + a3X3 + ... + apXp and the residuals X2 - (a1X1 + a3X3 + ... + apXp). This step adjusts X2 for simultaneous linear change in X1, X3, ..., Xp. Regress the Y-residuals on the X2-residuals. The resulting regression line (through the origin) is Y - (c1X1 + c3X3 + ... + cpXp) = b2[X2 - (a1X1 + a3X3 + ... + apXp)]. That is, the slope of this regression line is the coefficient of X2 in the original model. And, because the values of Y do not include noise, the points (X2-residual, Y-residual) lie on that line through the origin. I hope this explanation will help you to explain the correct general interpretation in your teaching, and steer your students away from the "held constant" interpretation. I realize that it may be awkward if the textbook uses the "held constant" interpretation (students will wonder what else the book gets wrong), but the message is important. Some books do get multiple regression right. One example is Weisberg (1985). And Cook and Weisberg (1982, Section 2.3.2) give a straightforward proof for the usual partial-regression plot. Regards, David Hoaglin Cook, R. D., and S. Weisberg (1982). Residuals and Influence in Regression. New York: Chapman & Hall. Weisberg, S. (1985). Applied Linear Regression, second edition. New York: John Wiley & Sons. On Mon, Nov 4, 2013 at 9:13 PM, Lucas <[email protected]> wrote: > Well, just two things: > > 1)The word "change" means change (Xa becomes Xb); the word > "difference" means difference (Xa is not the same as Xb). Why > wordplay? If difference is always an accurate statement of the model, > and "change" is only accurate sometimes (as your response implicitly > admits), why not say, "Wow, my bad. Yes, difference is generally > accurate, and that of course is a better way to describe the model in > general, and especially in a context with people from dozens of > fields."? > > 2)You indicated that "Straightforward mathematics" proved the point > you have often articulated. Now you indicate is too difficult to > express in plain text, which certainly does not sound like > straightforward mathematics. But, okay. Still, I would love to see > this expression. I'm not joking or baiting--I teach this material and > if there's an expression that would make your point I'll integrate it > into my teaching. Others on the list might do so as well. That would > be a major service to the disciplines. Perhaps instead of trying to > write it in plain text, you can provide a citation to something that > clearly makes your point. That could be helpful. > > So, at this point I, and perhaps others, look forward to receiving > that citation. Thanks in advance. > > Take care. > Sam * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/faqs/resources/statalist-faq/ * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:

**References**:**st: Relative Importance of predictors in regression***From:*Nikos Kakouros <[email protected]>

**Re: st: Relative Importance of predictors in regression***From:*David Hoaglin <[email protected]>

**Re: st: Relative Importance of predictors in regression***From:*Lucas <[email protected]>

**Re: st: Relative Importance of predictors in regression***From:*David Hoaglin <[email protected]>

**Re: st: Relative Importance of predictors in regression***From:*Lucas <[email protected]>

**Re: st: Relative Importance of predictors in regression***From:*David Hoaglin <[email protected]>

**Re: st: Relative Importance of predictors in regression***From:*Lucas <[email protected]>

- Prev by Date:
**st: R: Checking Goodness of Fit for NBREG Model** - Next by Date:
**Re: st: Checking Goodness of Fit for NBREG Model** - Previous by thread:
**Re: st: Relative Importance of predictors in regression** - Next by thread:
**Re: st: Relative Importance of predictors in regression** - Index(es):