[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
"Schaffer, Mark E" <M.E.Schaffer@hw.ac.uk> |

To |
"Steven Archambault" <archstevej@gmail.com> |

Subject |
RE: st: RE: Hausman test for clustered random vs. fixed effects (again) |

Date |
Fri, 3 Jul 2009 01:17:21 +0100 |

Steve, > -----Original Message----- > From: Steven Archambault [mailto:archstevej@gmail.com] > Sent: 03 July 2009 00:42 > To: Schaffer, Mark E > Cc: statalist@hsphsun2.harvard.edu > Subject: Re: st: RE: Hausman test for clustered random vs. > fixed effects (again) > > Okay that makes sense. For a second there I thought I was not > understanding the test. The different model specifications I > use give p values (from the xtoverid test) of .1 to .25. Do > you think values over say 20% make you less nervous about > accepting RE results? My plan is to report both FE and RE > models, suggesting that RE results can be considered valid > given the p values. > > -Steve Well, like I said, it's really a matter of taste. I'm perhaps more nervous and less gung ho than your average applied economist. 20% makes me less nervous than 10%, of course. But if you want to pursue this seriously, you should consider going down the route of testing specifically the subset of coefficients of interest. --Mark > On Thu, Jul 2, 2009 at 5:13 PM, Schaffer, Mark > E<M.E.Schaffer@hw.ac.uk> wrote: > > Steve, > > > >> -----Original Message----- > >> From: Steven Archambault [mailto:archstevej@gmail.com] > >> Sent: 03 July 2009 00:01 > >> To: Schaffer, Mark E > >> Subject: Re: st: RE: Hausman test for clustered random vs. > >> fixed effects (again) > >> > >> Wait a second, I thought with a Chi sq test we reject the > null that > >> the FE and RE coefficients are different when the critical > value is > >> such that the p-value is greater or equal to .05. This > would give us > >> a 5% (or more) significance that the null is rejected. We get this > >> with a lower chi-sq value. > >> It was with this logic that I am saying RE is the preferred model. > > > > There's nothing sacred about the 5% level. Some people, > when constructing tables for their papers, put *s next to > coefficients that are significant at the 10% level ... which > happens to be your p-value. > > > > The bigger the contrasts, the smaller the p-value, and 10% > implies contrasts that are large enough to make me nervous. > Of course, de gustibus non est disputandum. > > > > If you want to take this further, you might consider > focusing on the coefficients of interest, whatever they are. > You may well find that the joint contrast between the RE and > FE coefficients of interest is significant at a still smaller > p-value (suggesting you dump RE), or is not at all > significant (suggesting RE is preferred on efficiency grounds). > > > > -xtoverid- doesn't support tests of subsets of coefficients > (I should consider adding this feature, I guess) but you can > do the test by hand. It's described in the Arellano paper in > the help file, and I think Vince Wiggins had a post on > Statalist some time ago that describes how to do it. > > > > Cheers, > > Mark > > > >> > >> -Steve > >> > >> > >> > >> On Thu, Jul 2, 2009 at 4:47 PM, Schaffer, Mark > >> E<M.E.Schaffer@hw.ac.uk> wrote: > >> > Steve, > >> > > >> >> -----Original Message----- > >> >> From: Steven Archambault [mailto:archstevej@gmail.com] > >> >> Sent: 02 July 2009 22:41 > >> >> To: statalist@hsphsun2.harvard.edu; Schaffer, Mark E > >> >> Cc: austinnichols@gmail.com; Alfred.Stiglbauer@oenb.at > >> >> Subject: Re: st: RE: Hausman test for clustered random vs. > >> >> fixed effects (again) > >> >> > >> >> Mark, > >> >> > >> >> I should have commented on this earlier, but when I eye the > >> >> coefficients for both the FE and RE results, I see that > >> some of them > >> >> are quite different from one another. However, the > xtoverid result > >> >> suggests RE is the one to use. Does anybody see this as > a problem? > >> >> The numerator of the Hausman wald test is the difference in > >> >> coefficients of the two models. Is this not missed in > the xtoverid > >> >> approach? > >> > > >> > A few things here: > >> > > >> > - The "xtoverid approach" in this case is **identical** to > >> the traditional Hausman test in concept. They are both > >> vector-of-contrast tests, the contrast being between the 9 > FE and RE > >> coefficients. The **only** difference in this case > between the GMM > >> stat reported by -xtoverid- and the traditional Hausman > stat is that > >> the former is cluster-robust. In addition to the > references on this > >> point that I cited in my previous posting, you should also > check out > >> Ruud's textbook, "An Introduction to Classical Econometric Theory". > >> > > >> > - The test has 9 degrees of freedom because 9 coefficients > >> are being contrasted jointly. This means that some can indeed be > >> quite different, but if the others are very similar then a test of > >> the joint contrasts can be statistically insignificant. > >> > > >> > - The p-value reported by -xtoverid- is 10%, which a little > >> worrisome. If you were to do a vector-of-contrast tests > focusing on > >> a subset of coefficients instead of all 9 (not supported by > >> -xtoverid- but do-able by hand), you could well find that > you reject > >> the null at 5% or 1% or whatever. I don't think it's > straightforward > >> to conclude that RE is the estimator of choice. > >> > > >> > Hope this helps. > >> > > >> > Cheers, > >> > Mark > >> > > >> >> > >> >> I am posting my regression results to show what I am > talking about > >> >> more clearly. > >> >> > >> >> Thanks for your input. > >> >> -Steve > >> >> > >> >> > >> >> Fixed-effects (within) regression Number of obs > >> >> = 404 > >> >> Group variable: id_code_id Number of > >> groups = > >> >> 88 > >> >> > >> >> R-sq: within = 0.2304 Obs per > >> >> group: min = 1 > >> >> between = 0.4730 > >> >> avg = 4.6 > >> >> overall = 0.4487 > >> >> max = 7 > >> >> > >> >> F(9,87) > >> >> = 2.47 > >> >> corr(u_i, Xb) = -0.9558 Prob > F > >> >> = 0.0148 > >> >> > >> >> (Std. Err. adjusted for 88 > clusters in > >> >> id_code_id) > >> >> -------------------------------------------------------------- > >> >> ---------------- > >> >> | Robust > >> >> lnfd | Coef. Std. Err. t P>|t| > >> [95% Conf. > >> >> Interval] > >> >> -------------+------------------------------------------------ > >> >> ---------- > >> >> -------------+------ > >> >> lags | -.0267991 .0185982 -1.44 0.153 -.063765 > >> >> .0101668 > >> >> lagk | .0964571 .0353269 2.73 0.008 > >> >> .0262411 .166673 > >> >> lagp | .2210296 .1206562 1.83 0.070 > >> >> -.0187875 .4608468 > >> >> lagdr | -.0000267 .0000251 -1.06 0.291 -.0000767 > >> >> .0000232 > >> >> laglurb | .3483909 .1234674 2.82 0.006 .102986 > >> >> .5937957 > >> >> lagtra | .1109513 .1267749 0.88 0.384 > >> >> -.1410275 .3629301 > >> >> lagte | .0067764 .004166 1.63 0.107 > >> >> -.0015039 .0150567 > >> >> lagcr | .0950221 .0683074 1.39 0.168 > >> >> -.0407463 .2307905 > >> >> lagp | .0343752 .1291378 0.27 0.791 > >> >> -.2223001 .2910506 > >> >> _cons | 4.316618 1.996618 2.16 0.033 > >> >> .348124 8.285112 > >> >> -------------+------------------------------------------------ > >> >> ---------- > >> >> -------------+------ > >> >> sigma_u | .44721909 > >> >> sigma_e | .0595116 > >> >> rho | .98260039 (fraction of variance due to u_i) > >> >> -------------------------------------------------------------- > >> >> ---------------- > >> >> > >> >> > >> >> > >> >> Random-effects GLS regression Number of obs > >> >> = 404 > >> >> Group variable: id_code_id Number of > >> groups = > >> >> 88 > >> >> > >> >> R-sq: within = 0.1792 Obs per > >> >> group: min = 1 > >> >> between = 0.5074 > >> >> avg = 4.6 > >> >> overall = 0.5017 > >> >> max = 7 > >> >> > >> >> Random effects u_i ~ Gaussian Wald chi2(9) > >> >> = 48.97 > >> >> corr(u_i, X) = 0 (assumed) Prob > chi2 > >> >> = 0.0000 > >> >> > >> >> (Std. Err. adjusted for > clustering on > >> >> id_code_id) > >> >> -------------------------------------------------------------- > >> >> ---------------- > >> >> | Robust > >> >> lnfd | Coef. Std. Err. z P>|z| > >> [95% Conf. > >> >> Interval] > >> >> -------------+------------------------------------------------ > >> >> ---------- > >> >> -------------+------ > >> >> lags | -.01138 .0135958 -0.84 0.403 -.0380274 > >> >> .0152673 > >> >> lagk | .0115314 .0180641 0.64 0.523 > >> >> -.0238735 .0469363 > >> >> lagp | .2551701 .119322 2.14 0.032 > >> >> .0213033 .4890369 > >> >> lagdr | -6.17e-06 .0000153 -0.40 0.686 -.0000361 > >> >> .0000238 > >> >> laglurb | .0657802 .0153923 4.27 0.000 .0356119 > >> >> .0959486 > >> >> lagtra | .0022183 .0579203 0.04 0.969 > >> >> -.1113034 .11574 > >> >> lagte | .0048012 .0016128 2.98 0.003 > >> >> .00164 .0079623 > >> >> lagcr | .1051833 .045994 2.29 0.022 > >> >> .0150368 .1953298 > >> >> lagp | .184373 .1191063 1.55 0.122 > >> >> -.0490711 .4178171 > >> >> _cons | 9.071133 .2322309 39.06 0.000 > >> >> 8.615968 9.526297 > >> >> -------------+------------------------------------------------ > >> >> ---------- > >> >> -------------+------ > >> >> sigma_u | .10617991 > >> >> sigma_e | .0595116 > >> >> rho | .76095591 (fraction of variance due to u_i) > >> >> -------------------------------------------------------------- > >> >> ---------------- > >> >> > >> >> . xtoverid; > >> >> > >> >> Test of overidentifying restrictions: fixed vs random effects > >> >> Cross-section time-series model: xtreg re robust Sargan-Hansen > >> >> statistic 14.684 Chi-sq(9) P-value = 0.1000 > >> >> > >> >> > >> >> > >> >> > >> >> > >> >> On Sat, Jun 27, 2009 at 11:31 AM, Schaffer, Mark > >> >> E<M.E.Schaffer@hw.ac.uk> wrote: > >> >> > Steve, > >> >> > > >> >> >> -----Original Message----- > >> >> >> From: owner-statalist@hsphsun2.harvard.edu > >> >> >> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf > >> Of Steven > >> >> >> Archambault > >> >> >> Sent: 27 June 2009 00:26 > >> >> >> To: statalist@hsphsun2.harvard.edu; austinnichols@gmail.com; > >> >> >> Alfred.Stiglbauer@oenb.at > >> >> >> Subject: st: Hausman test for clustered random vs. fixed > >> >> >> effects > >> >> >> (again) > >> >> >> > >> >> >> Hi all, > >> >> >> > >> >> >> I know this has been discussed before, but in STATA 10 > >> >> (and versions > >> >> >> before 9 I understand) the canned procedure for Hausman > >> test when > >> >> >> comparing FE and RE models cannot be run when the data > >> >> analysis uses > >> >> >> clustering (and by default corrects for robust errors > >> in STATA 10). > >> >> >> This is the error received > >> >> >> > >> >> >> "hausman cannot be used with vce(robust), > vce(cluster cvar), or > >> >> >> p-weighted data" > >> >> >> > >> >> >> My question is whether or not the approach of using > xtoverid to > >> >> >> compare FE and RE models (analyzed using the clustered and > >> >> by default > >> >> >> robust approach in STATA 10) is accepted in the > >> literature. This > >> >> >> approach produces the Sargan-Hansen stat, which is > >> typically used > >> >> >> with analyses that have instrumentalized variables > and need an > >> >> >> overidentification test. For the sake of publishing I am > >> >> wondering if > >> >> >> it is better just not to worry about heteroskedaticity, > >> and avoid > >> >> >> clustering in the first place (even though > >> >> heteroskedaticity likely > >> >> >> exists)? Or, alternatively one could just calculate the > >> >> Hausman test > >> >> >> by hand following the clustered analyses. > >> >> >> > >> >> >> Thanks for your insight. > >> >> > > >> >> > It's very much accepted in the literature. In the > >> -xtoverid- help > >> >> > file, see especially the paper by Arellano and the book > >> by Hayashi. > >> >> > > >> >> > If you suspect heteroskedasticity or clustered errors, > >> >> there really is > >> >> > no good reason to go with a test (classic Hausman) that is > >> >> invalid in > >> >> > the presence of these problems. The GMM -xtoverid- > >> approach is a > >> >> > generalization of the Hausman test, in the following sense: > >> >> > > >> >> > - The Hausman and GMM tests of fixed vs. random effects > >> >> have the same > >> >> > degrees of freedom. This means the result cited by Hayashi > >> >> (and due > >> >> > to Newey, if I recall) kicks in, namely... > >> >> > > >> >> > - Under the assumption of homoskedasticity and independent > >> >> errors, the > >> >> > Hausman and GMM test statistics are numerically identical. > >> >> Same test. > >> >> > > >> >> > - When you loosen the iid assumption and allow > >> >> heteroskedasticity or > >> >> > dependent data, the robust GMM test is the natural > >> generalization. > >> >> > > >> >> > Hope this helps. > >> >> > > >> >> > Cheers, > >> >> > Mark (author of -xtoverid-) > >> >> > > >> >> >> * > >> >> >> * For searches and help try: > >> >> >> * http://www.stata.com/help.cgi?search > >> >> >> * http://www.stata.com/support/statalist/faq > >> >> >> * http://www.ats.ucla.edu/stat/stata/ > >> >> >> > >> >> > > >> >> > > >> >> > -- > >> >> > Heriot-Watt University is a Scottish charity registered > >> >> under charity > >> >> > number SC000278. > >> >> > > >> >> > > >> >> > * > >> >> > * For searches and help try: > >> >> > * http://www.stata.com/help.cgi?search > >> >> > * http://www.stata.com/support/statalist/faq > >> >> > * http://www.ats.ucla.edu/stat/stata/ > >> >> > > >> >> > >> > > >> > > >> > -- > >> > Heriot-Watt University is a Scottish charity registered > >> under charity > >> > number SC000278. > >> > > >> > > >> > > > > > > -- > > Heriot-Watt University is a Scottish charity registered > under charity > > number SC000278. > > > > > -- Heriot-Watt University is a Scottish charity registered under charity number SC000278. * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**Re: st: RE: Hausman test for clustered random vs. fixed effects (again)***From:*Steven Archambault <archstevej@gmail.com>

**References**:**Re: st: RE: Hausman test for clustered random vs. fixed effects (again)***From:*Steven Archambault <archstevej@gmail.com>

**RE: st: RE: Hausman test for clustered random vs. fixed effects (again)***From:*"Schaffer, Mark E" <M.E.Schaffer@hw.ac.uk>

**RE: st: RE: Hausman test for clustered random vs. fixed effects (again)***From:*"Schaffer, Mark E" <M.E.Schaffer@hw.ac.uk>

**Re: st: RE: Hausman test for clustered random vs. fixed effects (again)***From:*Steven Archambault <archstevej@gmail.com>

- Prev by Date:
**Re: st: RE: Hausman test for clustered random vs. fixed effects (again)** - Next by Date:
**st: Re:** - Previous by thread:
**Re: st: RE: Hausman test for clustered random vs. fixed effects (again)** - Next by thread:
**Re: st: RE: Hausman test for clustered random vs. fixed effects (again)** - Index(es):

© Copyright 1996–2016 StataCorp LP | Terms of use | Privacy | Contact us | What's new | Site index |