Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: st: RE: Hausman test for clustered random vs. fixed effects (again)


From   "Schaffer, Mark E" <M.E.Schaffer@hw.ac.uk>
To   "Steven Archambault" <archstevej@gmail.com>
Subject   RE: st: RE: Hausman test for clustered random vs. fixed effects (again)
Date   Mon, 6 Jul 2009 15:08:04 +0100

Steve,

> -----Original Message-----
> From: Steven Archambault [mailto:archstevej@gmail.com] 
> Sent: 05 July 2009 01:59
> To: Schaffer, Mark E
> Cc: statalist@hsphsun2.harvard.edu
> Subject: Re: st: RE: Hausman test for clustered random vs. 
> fixed effects (again)
> 
> Thanks Mark,
> 
> If I am not mistaken, this old post by Vince Wiggins explains 
> how one would go about setting up a Hausman test for a select 
> number of coefficients. I am trying to see how this test 
> works, and the results compare to just doing a canned 
> procedure (hausman test, xtoverid hausman test, etc.)
> 
> http://www.stata.com/statalist/archive/2003-10/msg00031.html
> 
> -Steve

I think what you're looking for are some later posts by Vince:

http://www.stata.com/statalist/archive/2005-08/msg00807.html
http://www.stata.com/statalist/archive/2005-08/msg00837.html
http://www.stata.com/statalist/archive/2005-08/msg00853.html

Note that the last post contains a correction by Vince to his code.

Also, if you're interested, here is the last word in the accompanying debate on how to guarantee that a Hausman statistic is non-negative in finite samples.

http://www.stata.com/statalist/archive/2005-08/msg00762.html

Cheers,
Mark

NB: Steve, your emails are being posting to Statalist but not showing up here.

Prof. Mark Schaffer FRSE
Director, CERT
Department of Economics
School of Management & Languages
Heriot-Watt University, Edinburgh EH14 4AS
tel +44-131-451-3494 / fax +44-131-451-3296
http://ideas.repec.org/e/psc51.html


> 
> 
> On Thu, Jul 2, 2009 at 6:17 PM, Schaffer, Mark 
> E<M.E.Schaffer@hw.ac.uk> wrote:
> > Steve,
> >
> >> -----Original Message-----
> >> From: Steven Archambault [mailto:archstevej@gmail.com]
> >> Sent: 03 July 2009 00:42
> >> To: Schaffer, Mark E
> >> Cc: statalist@hsphsun2.harvard.edu
> >> Subject: Re: st: RE: Hausman test for clustered random vs.
> >> fixed effects (again)
> >>
> >> Okay that makes sense. For a second there I thought I was not 
> >> understanding the test. The different model specifications 
> I use give 
> >> p values (from the xtoverid test) of .1 to .25. Do you 
> think values 
> >> over say 20% make you less nervous about accepting RE results? My 
> >> plan is to report both FE and RE models, suggesting that 
> RE results 
> >> can be considered valid given the p values.
> >>
> >> -Steve
> >
> > Well, like I said, it's really a matter of taste.  I'm 
> perhaps more nervous and less gung ho than your average 
> applied economist.  20% makes me less nervous than 10%, of 
> course.  But if you want to pursue this seriously, you should 
> consider going down the route of testing specifically the 
> subset of coefficients of interest.
> >
> > --Mark
> >
> >> On Thu, Jul 2, 2009 at 5:13 PM, Schaffer, Mark 
> >> E<M.E.Schaffer@hw.ac.uk> wrote:
> >> > Steve,
> >> >
> >> >> -----Original Message-----
> >> >> From: Steven Archambault [mailto:archstevej@gmail.com]
> >> >> Sent: 03 July 2009 00:01
> >> >> To: Schaffer, Mark E
> >> >> Subject: Re: st: RE: Hausman test for clustered random vs.
> >> >> fixed effects (again)
> >> >>
> >> >> Wait a second, I thought with a Chi sq test we reject the
> >> null that
> >> >> the FE and RE coefficients are different when the critical
> >> value is
> >> >> such that the p-value is greater or equal to .05. This
> >> would give us
> >> >> a 5% (or more) significance that the null is rejected. 
> We get this 
> >> >> with a lower chi-sq value.
> >> >> It was with this logic that I am saying RE is the 
> preferred model.
> >> >
> >> > There's nothing sacred about the 5% level.  Some people,
> >> when constructing tables for their papers, put *s next to 
> >> coefficients that are significant at the 10% level ... 
> which happens 
> >> to be your p-value.
> >> >
> >> > The bigger the contrasts, the smaller the p-value, and 10%
> >> implies contrasts that are large enough to make me nervous.
> >> Of course, de gustibus non est disputandum.
> >> >
> >> > If you want to take this further, you might consider
> >> focusing on the coefficients of interest, whatever they are.
> >> You may well find that the joint contrast between the RE and FE 
> >> coefficients of interest is significant at a still smaller p-value 
> >> (suggesting you dump RE), or is not at all significant 
> (suggesting RE 
> >> is preferred on efficiency grounds).
> >> >
> >> > -xtoverid- doesn't support tests of subsets of coefficients
> >> (I should consider adding this feature, I guess) but you 
> can do the 
> >> test by hand.  It's described in the Arellano paper in the 
> help file, 
> >> and I think Vince Wiggins had a post on Statalist some 
> time ago that 
> >> describes how to do it.
> >> >
> >> > Cheers,
> >> > Mark
> >> >
> >> >>
> >> >> -Steve
> >> >>
> >> >>
> >> >>
> >> >> On Thu, Jul 2, 2009 at 4:47 PM, Schaffer, Mark 
> >> >> E<M.E.Schaffer@hw.ac.uk> wrote:
> >> >> > Steve,
> >> >> >
> >> >> >> -----Original Message-----
> >> >> >> From: Steven Archambault [mailto:archstevej@gmail.com]
> >> >> >> Sent: 02 July 2009 22:41
> >> >> >> To: statalist@hsphsun2.harvard.edu; Schaffer, Mark E
> >> >> >> Cc: austinnichols@gmail.com; Alfred.Stiglbauer@oenb.at
> >> >> >> Subject: Re: st: RE: Hausman test for clustered random vs.
> >> >> >> fixed effects (again)
> >> >> >>
> >> >> >> Mark,
> >> >> >>
> >> >> >> I should have commented on this earlier, but when I eye the 
> >> >> >> coefficients for both the FE and RE results, I see that
> >> >> some of them
> >> >> >> are quite different from one another. However, the
> >> xtoverid result
> >> >> >> suggests RE is the one to use. Does anybody see this as
> >> a problem?
> >> >> >> The numerator of the Hausman wald test is the difference in 
> >> >> >> coefficients of the two models. Is this not missed in
> >> the xtoverid
> >> >> >> approach?
> >> >> >
> >> >> > A few things here:
> >> >> >
> >> >> > - The "xtoverid approach" in this case is **identical** to
> >> >> the traditional Hausman test in concept.  They are both 
> >> >> vector-of-contrast tests, the contrast being between the 9
> >> FE and RE
> >> >> coefficients.  The **only** difference in this case
> >> between the GMM
> >> >> stat reported by -xtoverid- and the traditional Hausman
> >> stat is that
> >> >> the former is cluster-robust.  In addition to the
> >> references on this
> >> >> point that I cited in my previous posting, you should also
> >> check out
> >> >> Ruud's textbook, "An Introduction to Classical 
> Econometric Theory".
> >> >> >
> >> >> > - The test has 9 degrees of freedom because 9 coefficients
> >> >> are being contrasted jointly.  This means that some can 
> indeed be 
> >> >> quite different, but if the others are very similar 
> then a test of 
> >> >> the joint contrasts can be statistically insignificant.
> >> >> >
> >> >> > - The p-value reported by -xtoverid- is 10%, which a little
> >> >> worrisome.  If you were to do a vector-of-contrast tests
> >> focusing on
> >> >> a subset of coefficients instead of all 9 (not supported by
> >> >> -xtoverid- but do-able by hand), you could well find that
> >> you reject
> >> >> the null at 5% or 1% or whatever.  I don't think it's
> >> straightforward
> >> >> to conclude that RE is the estimator of choice.
> >> >> >
> >> >> > Hope this helps.
> >> >> >
> >> >> > Cheers,
> >> >> > Mark
> >> >> >
> >> >> >>
> >> >> >> I am posting my regression results to show what I am
> >> talking about
> >> >> >> more clearly.
> >> >> >>
> >> >> >> Thanks for your input.
> >> >> >> -Steve
> >> >> >>
> >> >> >>
> >> >> >> Fixed-effects (within) regression               Number of obs
> >> >> >>      =       404
> >> >> >> Group variable: id_code_id                      Number of
> >> >> groups   =
> >> >> >> 88
> >> >> >>
> >> >> >> R-sq:  within  = 0.2304                         Obs per
> >> >> >> group: min =         1
> >> >> >>        between = 0.4730
> >> >> >>  avg =       4.6
> >> >> >>        overall = 0.4487
> >> >> >>  max =         7
> >> >> >>
> >> >> >>                                                 F(9,87)
> >> >> >>      =      2.47
> >> >> >> corr(u_i, Xb)  = -0.9558                        Prob > F
> >> >> >>      =    0.0148
> >> >> >>
> >> >> >>                             (Std. Err. adjusted for 88
> >> clusters in
> >> >> >> id_code_id)
> >> >> >> 
> --------------------------------------------------------------
> >> >> >> ----------------
> >> >> >>              |               Robust
> >> >> >>        lnfd |      Coef.   Std. Err.      t    P>|t|
> >> >> [95% Conf.
> >> >> >> Interval]
> >> >> >> 
> -------------+------------------------------------------------
> >> >> >> ----------
> >> >> >> -------------+------
> >> >> >>    lags |  -.0267991   .0185982    -1.44   0.153     -.063765
> >> >> >>    .0101668
> >> >> >>      lagk |   .0964571   .0353269     2.73   0.008
> >> >> >> .0262411     .166673
> >> >> >>     lagp |   .2210296   .1206562     1.83   0.070
> >> >> >> -.0187875    .4608468
> >> >> >> lagdr |  -.0000267   .0000251    -1.06   0.291    -.0000767
> >> >> >>  .0000232
> >> >> >> laglurb |   .3483909   .1234674     2.82   0.006      .102986
> >> >> >>    .5937957
> >> >> >>    lagtra |   .1109513   .1267749     0.88   0.384
> >> >> >> -.1410275    .3629301
> >> >> >>      lagte |   .0067764    .004166     1.63   0.107
> >> >> >> -.0015039    .0150567
> >> >> >>     lagcr |   .0950221   .0683074     1.39   0.168
> >> >> >> -.0407463    .2307905
> >> >> >>     lagp |   .0343752   .1291378     0.27   0.791
> >> >> >> -.2223001    .2910506
> >> >> >>        _cons |   4.316618   1.996618     2.16   0.033
> >> >> >> .348124    8.285112
> >> >> >> 
> -------------+------------------------------------------------
> >> >> >> ----------
> >> >> >> -------------+------
> >> >> >>      sigma_u |  .44721909
> >> >> >>      sigma_e |   .0595116
> >> >> >>          rho |  .98260039   (fraction of variance due to u_i)
> >> >> >> 
> --------------------------------------------------------------
> >> >> >> ----------------
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> Random-effects GLS regression                   Number of obs
> >> >> >>      =       404
> >> >> >> Group variable: id_code_id                      Number of
> >> >> groups   =
> >> >> >> 88
> >> >> >>
> >> >> >> R-sq:  within  = 0.1792                         Obs per
> >> >> >> group: min =         1
> >> >> >>        between = 0.5074
> >> >> >>  avg =       4.6
> >> >> >>        overall = 0.5017
> >> >> >>  max =         7
> >> >> >>
> >> >> >> Random effects u_i ~ Gaussian                   Wald chi2(9)
> >> >> >>      =     48.97
> >> >> >> corr(u_i, X)       = 0 (assumed)                Prob > chi2
> >> >> >>      =    0.0000
> >> >> >>
> >> >> >>                              (Std. Err. adjusted for
> >> clustering on
> >> >> >> id_code_id)
> >> >> >> 
> --------------------------------------------------------------
> >> >> >> ----------------
> >> >> >>              |               Robust
> >> >> >>        lnfd |      Coef.   Std. Err.      z    P>|z|
> >> >> [95% Conf.
> >> >> >> Interval]
> >> >> >> 
> -------------+------------------------------------------------
> >> >> >> ----------
> >> >> >> -------------+------
> >> >> >>    lags |    -.01138   .0135958    -0.84   0.403    -.0380274
> >> >> >>    .0152673
> >> >> >>      lagk |   .0115314   .0180641     0.64   0.523
> >> >> >> -.0238735    .0469363
> >> >> >>     lagp |   .2551701    .119322     2.14   0.032
> >> >> >> .0213033    .4890369
> >> >> >> lagdr |  -6.17e-06   .0000153    -0.40   0.686    -.0000361
> >> >> >>  .0000238
> >> >> >> laglurb |   .0657802   .0153923     4.27   0.000     .0356119
> >> >> >>    .0959486
> >> >> >>    lagtra |   .0022183   .0579203     0.04   0.969
> >> >> >> -.1113034      .11574
> >> >> >>      lagte |   .0048012   .0016128     2.98   0.003
> >> >> >> .00164    .0079623
> >> >> >>     lagcr |   .1051833    .045994     2.29   0.022
> >> >> >> .0150368    .1953298
> >> >> >>     lagp |    .184373   .1191063     1.55   0.122
> >> >> >> -.0490711    .4178171
> >> >> >>        _cons |   9.071133   .2322309    39.06   0.000
> >> >> >> 8.615968    9.526297
> >> >> >> 
> -------------+------------------------------------------------
> >> >> >> ----------
> >> >> >> -------------+------
> >> >> >>      sigma_u |  .10617991
> >> >> >>      sigma_e |   .0595116
> >> >> >>          rho |  .76095591   (fraction of variance due to u_i)
> >> >> >> 
> --------------------------------------------------------------
> >> >> >> ----------------
> >> >> >>
> >> >> >> . xtoverid;
> >> >> >>
> >> >> >> Test of overidentifying restrictions: fixed vs 
> random effects 
> >> >> >> Cross-section time-series model: xtreg re robust 
> Sargan-Hansen 
> >> >> >> statistic  14.684  Chi-sq(9)    P-value = 0.1000
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> On Sat, Jun 27, 2009 at 11:31 AM, Schaffer, Mark 
> >> >> >> E<M.E.Schaffer@hw.ac.uk> wrote:
> >> >> >> > Steve,
> >> >> >> >
> >> >> >> >> -----Original Message-----
> >> >> >> >> From: owner-statalist@hsphsun2.harvard.edu
> >> >> >> >> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf
> >> >> Of Steven
> >> >> >> >> Archambault
> >> >> >> >> Sent: 27 June 2009 00:26
> >> >> >> >> To: statalist@hsphsun2.harvard.edu; 
> austinnichols@gmail.com; 
> >> >> >> >> Alfred.Stiglbauer@oenb.at
> >> >> >> >> Subject: st: Hausman test for clustered random vs. fixed 
> >> >> >> >> effects
> >> >> >> >> (again)
> >> >> >> >>
> >> >> >> >> Hi all,
> >> >> >> >>
> >> >> >> >> I know this has been discussed before, but in STATA 10
> >> >> >> (and versions
> >> >> >> >> before 9 I understand) the canned procedure for Hausman
> >> >> test when
> >> >> >> >> comparing FE and RE models cannot be run when the data
> >> >> >> analysis uses
> >> >> >> >> clustering (and by default corrects for robust errors
> >> >> in STATA 10).
> >> >> >> >> This is the error received
> >> >> >> >>
> >> >> >> >> "hausman cannot be used with vce(robust),
> >> vce(cluster cvar), or
> >> >> >> >> p-weighted data"
> >> >> >> >>
> >> >> >> >> My question is whether or not the approach of using
> >> xtoverid to
> >> >> >> >> compare FE and RE models (analyzed using the clustered and
> >> >> >> by default
> >> >> >> >> robust approach in STATA 10) is accepted in the
> >> >> literature. This
> >> >> >> >> approach produces the Sargan-Hansen stat, which is
> >> >> typically used
> >> >> >> >> with analyses that have instrumentalized variables
> >> and need an
> >> >> >> >> overidentification test. For the sake of publishing I am
> >> >> >> wondering if
> >> >> >> >> it is better just not to worry about heteroskedaticity,
> >> >> and avoid
> >> >> >> >> clustering in the first place (even though
> >> >> >> heteroskedaticity likely
> >> >> >> >> exists)? Or, alternatively one could just calculate the
> >> >> >> Hausman test
> >> >> >> >> by hand following the clustered analyses.
> >> >> >> >>
> >> >> >> >> Thanks for your insight.
> >> >> >> >
> >> >> >> > It's very much accepted in the literature.  In the
> >> >> -xtoverid- help
> >> >> >> > file, see especially the paper by Arellano and the book
> >> >> by Hayashi.
> >> >> >> >
> >> >> >> > If you suspect heteroskedasticity or clustered errors,
> >> >> >> there really is
> >> >> >> > no good reason to go with a test (classic Hausman) that is
> >> >> >> invalid in
> >> >> >> > the presence of these problems.  The GMM -xtoverid-
> >> >> approach is a
> >> >> >> > generalization of the Hausman test, in the following sense:
> >> >> >> >
> >> >> >> > - The Hausman and GMM tests of fixed vs. random effects
> >> >> >> have the same
> >> >> >> > degrees of freedom.  This means the result cited by Hayashi
> >> >> >> (and due
> >> >> >> > to Newey, if I recall) kicks in, namely...
> >> >> >> >
> >> >> >> > - Under the assumption of homoskedasticity and independent
> >> >> >> errors, the
> >> >> >> > Hausman and GMM test statistics are numerically identical.
> >> >> >> Same test.
> >> >> >> >
> >> >> >> > - When you loosen the iid assumption and allow
> >> >> >> heteroskedasticity or
> >> >> >> > dependent data, the robust GMM test is the natural
> >> >> generalization.
> >> >> >> >
> >> >> >> > Hope this helps.
> >> >> >> >
> >> >> >> > Cheers,
> >> >> >> > Mark (author of -xtoverid-)
> >> >> >> >
> >> >> >> >> *
> >> >> >> >> *   For searches and help try:
> >> >> >> >> *   http://www.stata.com/help.cgi?search
> >> >> >> >> *   http://www.stata.com/support/statalist/faq
> >> >> >> >> *   http://www.ats.ucla.edu/stat/stata/
> >> >> >> >>
> >> >> >> >
> >> >> >> >
> >> >> >> > --
> >> >> >> > Heriot-Watt University is a Scottish charity registered
> >> >> >> under charity
> >> >> >> > number SC000278.
> >> >> >> >
> >> >> >> >
> >> >> >> > *
> >> >> >> > *   For searches and help try:
> >> >> >> > *   http://www.stata.com/help.cgi?search
> >> >> >> > *   http://www.stata.com/support/statalist/faq
> >> >> >> > *   http://www.ats.ucla.edu/stat/stata/
> >> >> >> >
> >> >> >>
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Heriot-Watt University is a Scottish charity registered
> >> >> under charity
> >> >> > number SC000278.
> >> >> >
> >> >> >
> >> >>
> >> >
> >> >
> >> > --
> >> > Heriot-Watt University is a Scottish charity registered
> >> under charity
> >> > number SC000278.
> >> >
> >> >
> >>
> >
> >
> > --
> > Heriot-Watt University is a Scottish charity registered 
> under charity 
> > number SC000278.
> >
> >
> 


-- 
Heriot-Watt University is a Scottish charity
registered under charity number SC000278.


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index