Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: RE: Hausman test for clustered random vs. fixed effects (again)


From   Steven Archambault <archstevej@gmail.com>
To   "Schaffer, Mark E" <M.E.Schaffer@hw.ac.uk>
Subject   Re: st: RE: Hausman test for clustered random vs. fixed effects (again)
Date   Sat, 4 Jul 2009 18:58:51 -0600

Thanks Mark,

If I am not mistaken, this old post by Vince Wiggins explains how one
would go about setting up a Hausman test for a select number of
coefficients. I am trying to see how this test works, and the results
compare to just doing a canned procedure (hausman test, xtoverid
hausman test, etc.)

http://www.stata.com/statalist/archive/2003-10/msg00031.html

-Steve




On Thu, Jul 2, 2009 at 6:17 PM, Schaffer, Mark E<M.E.Schaffer@hw.ac.uk> wrote:
> Steve,
>
>> -----Original Message-----
>> From: Steven Archambault [mailto:archstevej@gmail.com]
>> Sent: 03 July 2009 00:42
>> To: Schaffer, Mark E
>> Cc: statalist@hsphsun2.harvard.edu
>> Subject: Re: st: RE: Hausman test for clustered random vs.
>> fixed effects (again)
>>
>> Okay that makes sense. For a second there I thought I was not
>> understanding the test. The different model specifications I
>> use give p values (from the xtoverid test) of .1 to .25. Do
>> you think values over say 20% make you less nervous about
>> accepting RE results? My plan is to report both FE and RE
>> models, suggesting that RE results can be considered valid
>> given the p values.
>>
>> -Steve
>
> Well, like I said, it's really a matter of taste.  I'm perhaps more nervous and less gung ho than your average applied economist.  20% makes me less nervous than 10%, of course.  But if you want to pursue this seriously, you should consider going down the route of testing specifically the subset of coefficients of interest.
>
> --Mark
>
>> On Thu, Jul 2, 2009 at 5:13 PM, Schaffer, Mark
>> E<M.E.Schaffer@hw.ac.uk> wrote:
>> > Steve,
>> >
>> >> -----Original Message-----
>> >> From: Steven Archambault [mailto:archstevej@gmail.com]
>> >> Sent: 03 July 2009 00:01
>> >> To: Schaffer, Mark E
>> >> Subject: Re: st: RE: Hausman test for clustered random vs.
>> >> fixed effects (again)
>> >>
>> >> Wait a second, I thought with a Chi sq test we reject the
>> null that
>> >> the FE and RE coefficients are different when the critical
>> value is
>> >> such that the p-value is greater or equal to .05. This
>> would give us
>> >> a 5% (or more) significance that the null is rejected. We get this
>> >> with a lower chi-sq value.
>> >> It was with this logic that I am saying RE is the preferred model.
>> >
>> > There's nothing sacred about the 5% level.  Some people,
>> when constructing tables for their papers, put *s next to
>> coefficients that are significant at the 10% level ... which
>> happens to be your p-value.
>> >
>> > The bigger the contrasts, the smaller the p-value, and 10%
>> implies contrasts that are large enough to make me nervous.
>> Of course, de gustibus non est disputandum.
>> >
>> > If you want to take this further, you might consider
>> focusing on the coefficients of interest, whatever they are.
>> You may well find that the joint contrast between the RE and
>> FE coefficients of interest is significant at a still smaller
>> p-value (suggesting you dump RE), or is not at all
>> significant (suggesting RE is preferred on efficiency grounds).
>> >
>> > -xtoverid- doesn't support tests of subsets of coefficients
>> (I should consider adding this feature, I guess) but you can
>> do the test by hand.  It's described in the Arellano paper in
>> the help file, and I think Vince Wiggins had a post on
>> Statalist some time ago that describes how to do it.
>> >
>> > Cheers,
>> > Mark
>> >
>> >>
>> >> -Steve
>> >>
>> >>
>> >>
>> >> On Thu, Jul 2, 2009 at 4:47 PM, Schaffer, Mark
>> >> E<M.E.Schaffer@hw.ac.uk> wrote:
>> >> > Steve,
>> >> >
>> >> >> -----Original Message-----
>> >> >> From: Steven Archambault [mailto:archstevej@gmail.com]
>> >> >> Sent: 02 July 2009 22:41
>> >> >> To: statalist@hsphsun2.harvard.edu; Schaffer, Mark E
>> >> >> Cc: austinnichols@gmail.com; Alfred.Stiglbauer@oenb.at
>> >> >> Subject: Re: st: RE: Hausman test for clustered random vs.
>> >> >> fixed effects (again)
>> >> >>
>> >> >> Mark,
>> >> >>
>> >> >> I should have commented on this earlier, but when I eye the
>> >> >> coefficients for both the FE and RE results, I see that
>> >> some of them
>> >> >> are quite different from one another. However, the
>> xtoverid result
>> >> >> suggests RE is the one to use. Does anybody see this as
>> a problem?
>> >> >> The numerator of the Hausman wald test is the difference in
>> >> >> coefficients of the two models. Is this not missed in
>> the xtoverid
>> >> >> approach?
>> >> >
>> >> > A few things here:
>> >> >
>> >> > - The "xtoverid approach" in this case is **identical** to
>> >> the traditional Hausman test in concept.  They are both
>> >> vector-of-contrast tests, the contrast being between the 9
>> FE and RE
>> >> coefficients.  The **only** difference in this case
>> between the GMM
>> >> stat reported by -xtoverid- and the traditional Hausman
>> stat is that
>> >> the former is cluster-robust.  In addition to the
>> references on this
>> >> point that I cited in my previous posting, you should also
>> check out
>> >> Ruud's textbook, "An Introduction to Classical Econometric Theory".
>> >> >
>> >> > - The test has 9 degrees of freedom because 9 coefficients
>> >> are being contrasted jointly.  This means that some can indeed be
>> >> quite different, but if the others are very similar then a test of
>> >> the joint contrasts can be statistically insignificant.
>> >> >
>> >> > - The p-value reported by -xtoverid- is 10%, which a little
>> >> worrisome.  If you were to do a vector-of-contrast tests
>> focusing on
>> >> a subset of coefficients instead of all 9 (not supported by
>> >> -xtoverid- but do-able by hand), you could well find that
>> you reject
>> >> the null at 5% or 1% or whatever.  I don't think it's
>> straightforward
>> >> to conclude that RE is the estimator of choice.
>> >> >
>> >> > Hope this helps.
>> >> >
>> >> > Cheers,
>> >> > Mark
>> >> >
>> >> >>
>> >> >> I am posting my regression results to show what I am
>> talking about
>> >> >> more clearly.
>> >> >>
>> >> >> Thanks for your input.
>> >> >> -Steve
>> >> >>
>> >> >>
>> >> >> Fixed-effects (within) regression               Number of obs
>> >> >>      =       404
>> >> >> Group variable: id_code_id                      Number of
>> >> groups   =
>> >> >> 88
>> >> >>
>> >> >> R-sq:  within  = 0.2304                         Obs per
>> >> >> group: min =         1
>> >> >>        between = 0.4730
>> >> >>  avg =       4.6
>> >> >>        overall = 0.4487
>> >> >>  max =         7
>> >> >>
>> >> >>                                                 F(9,87)
>> >> >>      =      2.47
>> >> >> corr(u_i, Xb)  = -0.9558                        Prob > F
>> >> >>      =    0.0148
>> >> >>
>> >> >>                             (Std. Err. adjusted for 88
>> clusters in
>> >> >> id_code_id)
>> >> >> --------------------------------------------------------------
>> >> >> ----------------
>> >> >>              |               Robust
>> >> >>        lnfd |      Coef.   Std. Err.      t    P>|t|
>> >> [95% Conf.
>> >> >> Interval]
>> >> >> -------------+------------------------------------------------
>> >> >> ----------
>> >> >> -------------+------
>> >> >>    lags |  -.0267991   .0185982    -1.44   0.153     -.063765
>> >> >>    .0101668
>> >> >>      lagk |   .0964571   .0353269     2.73   0.008
>> >> >> .0262411     .166673
>> >> >>     lagp |   .2210296   .1206562     1.83   0.070
>> >> >> -.0187875    .4608468
>> >> >> lagdr |  -.0000267   .0000251    -1.06   0.291    -.0000767
>> >> >>  .0000232
>> >> >> laglurb |   .3483909   .1234674     2.82   0.006      .102986
>> >> >>    .5937957
>> >> >>    lagtra |   .1109513   .1267749     0.88   0.384
>> >> >> -.1410275    .3629301
>> >> >>      lagte |   .0067764    .004166     1.63   0.107
>> >> >> -.0015039    .0150567
>> >> >>     lagcr |   .0950221   .0683074     1.39   0.168
>> >> >> -.0407463    .2307905
>> >> >>     lagp |   .0343752   .1291378     0.27   0.791
>> >> >> -.2223001    .2910506
>> >> >>        _cons |   4.316618   1.996618     2.16   0.033
>> >> >> .348124    8.285112
>> >> >> -------------+------------------------------------------------
>> >> >> ----------
>> >> >> -------------+------
>> >> >>      sigma_u |  .44721909
>> >> >>      sigma_e |   .0595116
>> >> >>          rho |  .98260039   (fraction of variance due to u_i)
>> >> >> --------------------------------------------------------------
>> >> >> ----------------
>> >> >>
>> >> >>
>> >> >>
>> >> >> Random-effects GLS regression                   Number of obs
>> >> >>      =       404
>> >> >> Group variable: id_code_id                      Number of
>> >> groups   =
>> >> >> 88
>> >> >>
>> >> >> R-sq:  within  = 0.1792                         Obs per
>> >> >> group: min =         1
>> >> >>        between = 0.5074
>> >> >>  avg =       4.6
>> >> >>        overall = 0.5017
>> >> >>  max =         7
>> >> >>
>> >> >> Random effects u_i ~ Gaussian                   Wald chi2(9)
>> >> >>      =     48.97
>> >> >> corr(u_i, X)       = 0 (assumed)                Prob > chi2
>> >> >>      =    0.0000
>> >> >>
>> >> >>                              (Std. Err. adjusted for
>> clustering on
>> >> >> id_code_id)
>> >> >> --------------------------------------------------------------
>> >> >> ----------------
>> >> >>              |               Robust
>> >> >>        lnfd |      Coef.   Std. Err.      z    P>|z|
>> >> [95% Conf.
>> >> >> Interval]
>> >> >> -------------+------------------------------------------------
>> >> >> ----------
>> >> >> -------------+------
>> >> >>    lags |    -.01138   .0135958    -0.84   0.403    -.0380274
>> >> >>    .0152673
>> >> >>      lagk |   .0115314   .0180641     0.64   0.523
>> >> >> -.0238735    .0469363
>> >> >>     lagp |   .2551701    .119322     2.14   0.032
>> >> >> .0213033    .4890369
>> >> >> lagdr |  -6.17e-06   .0000153    -0.40   0.686    -.0000361
>> >> >>  .0000238
>> >> >> laglurb |   .0657802   .0153923     4.27   0.000     .0356119
>> >> >>    .0959486
>> >> >>    lagtra |   .0022183   .0579203     0.04   0.969
>> >> >> -.1113034      .11574
>> >> >>      lagte |   .0048012   .0016128     2.98   0.003
>> >> >> .00164    .0079623
>> >> >>     lagcr |   .1051833    .045994     2.29   0.022
>> >> >> .0150368    .1953298
>> >> >>     lagp |    .184373   .1191063     1.55   0.122
>> >> >> -.0490711    .4178171
>> >> >>        _cons |   9.071133   .2322309    39.06   0.000
>> >> >> 8.615968    9.526297
>> >> >> -------------+------------------------------------------------
>> >> >> ----------
>> >> >> -------------+------
>> >> >>      sigma_u |  .10617991
>> >> >>      sigma_e |   .0595116
>> >> >>          rho |  .76095591   (fraction of variance due to u_i)
>> >> >> --------------------------------------------------------------
>> >> >> ----------------
>> >> >>
>> >> >> . xtoverid;
>> >> >>
>> >> >> Test of overidentifying restrictions: fixed vs random effects
>> >> >> Cross-section time-series model: xtreg re robust Sargan-Hansen
>> >> >> statistic  14.684  Chi-sq(9)    P-value = 0.1000
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Sat, Jun 27, 2009 at 11:31 AM, Schaffer, Mark
>> >> >> E<M.E.Schaffer@hw.ac.uk> wrote:
>> >> >> > Steve,
>> >> >> >
>> >> >> >> -----Original Message-----
>> >> >> >> From: owner-statalist@hsphsun2.harvard.edu
>> >> >> >> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf
>> >> Of Steven
>> >> >> >> Archambault
>> >> >> >> Sent: 27 June 2009 00:26
>> >> >> >> To: statalist@hsphsun2.harvard.edu; austinnichols@gmail.com;
>> >> >> >> Alfred.Stiglbauer@oenb.at
>> >> >> >> Subject: st: Hausman test for clustered random vs. fixed
>> >> >> >> effects
>> >> >> >> (again)
>> >> >> >>
>> >> >> >> Hi all,
>> >> >> >>
>> >> >> >> I know this has been discussed before, but in STATA 10
>> >> >> (and versions
>> >> >> >> before 9 I understand) the canned procedure for Hausman
>> >> test when
>> >> >> >> comparing FE and RE models cannot be run when the data
>> >> >> analysis uses
>> >> >> >> clustering (and by default corrects for robust errors
>> >> in STATA 10).
>> >> >> >> This is the error received
>> >> >> >>
>> >> >> >> "hausman cannot be used with vce(robust),
>> vce(cluster cvar), or
>> >> >> >> p-weighted data"
>> >> >> >>
>> >> >> >> My question is whether or not the approach of using
>> xtoverid to
>> >> >> >> compare FE and RE models (analyzed using the clustered and
>> >> >> by default
>> >> >> >> robust approach in STATA 10) is accepted in the
>> >> literature. This
>> >> >> >> approach produces the Sargan-Hansen stat, which is
>> >> typically used
>> >> >> >> with analyses that have instrumentalized variables
>> and need an
>> >> >> >> overidentification test. For the sake of publishing I am
>> >> >> wondering if
>> >> >> >> it is better just not to worry about heteroskedaticity,
>> >> and avoid
>> >> >> >> clustering in the first place (even though
>> >> >> heteroskedaticity likely
>> >> >> >> exists)? Or, alternatively one could just calculate the
>> >> >> Hausman test
>> >> >> >> by hand following the clustered analyses.
>> >> >> >>
>> >> >> >> Thanks for your insight.
>> >> >> >
>> >> >> > It's very much accepted in the literature.  In the
>> >> -xtoverid- help
>> >> >> > file, see especially the paper by Arellano and the book
>> >> by Hayashi.
>> >> >> >
>> >> >> > If you suspect heteroskedasticity or clustered errors,
>> >> >> there really is
>> >> >> > no good reason to go with a test (classic Hausman) that is
>> >> >> invalid in
>> >> >> > the presence of these problems.  The GMM -xtoverid-
>> >> approach is a
>> >> >> > generalization of the Hausman test, in the following sense:
>> >> >> >
>> >> >> > - The Hausman and GMM tests of fixed vs. random effects
>> >> >> have the same
>> >> >> > degrees of freedom.  This means the result cited by Hayashi
>> >> >> (and due
>> >> >> > to Newey, if I recall) kicks in, namely...
>> >> >> >
>> >> >> > - Under the assumption of homoskedasticity and independent
>> >> >> errors, the
>> >> >> > Hausman and GMM test statistics are numerically identical.
>> >> >> Same test.
>> >> >> >
>> >> >> > - When you loosen the iid assumption and allow
>> >> >> heteroskedasticity or
>> >> >> > dependent data, the robust GMM test is the natural
>> >> generalization.
>> >> >> >
>> >> >> > Hope this helps.
>> >> >> >
>> >> >> > Cheers,
>> >> >> > Mark (author of -xtoverid-)
>> >> >> >
>> >> >> >> *
>> >> >> >> *   For searches and help try:
>> >> >> >> *   http://www.stata.com/help.cgi?search
>> >> >> >> *   http://www.stata.com/support/statalist/faq
>> >> >> >> *   http://www.ats.ucla.edu/stat/stata/
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Heriot-Watt University is a Scottish charity registered
>> >> >> under charity
>> >> >> > number SC000278.
>> >> >> >
>> >> >> >
>> >> >> > *
>> >> >> > *   For searches and help try:
>> >> >> > *   http://www.stata.com/help.cgi?search
>> >> >> > *   http://www.stata.com/support/statalist/faq
>> >> >> > *   http://www.ats.ucla.edu/stat/stata/
>> >> >> >
>> >> >>
>> >> >
>> >> >
>> >> > --
>> >> > Heriot-Watt University is a Scottish charity registered
>> >> under charity
>> >> > number SC000278.
>> >> >
>> >> >
>> >>
>> >
>> >
>> > --
>> > Heriot-Watt University is a Scottish charity registered
>> under charity
>> > number SC000278.
>> >
>> >
>>
>
>
> --
> Heriot-Watt University is a Scottish charity
> registered under charity number SC000278.
>
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index