Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: RE: RE: RE: RE: RE: RE: RE: hausman, augmented test from Vince's code and xtoverid after xtivreg


From   "Schaffer, Mark E" <M.E.Schaffer@hw.ac.uk>
To   <statalist@hsphsun2.harvard.edu>
Subject   st: RE: RE: RE: RE: RE: RE: RE: hausman, augmented test from Vince's code and xtoverid after xtivreg
Date   Thu, 2 Aug 2012 23:27:59 +0100

Esther,

> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
> statalist@hsphsun2.harvard.edu] On Behalf Of ESTHER GOYA CARRILLO
> Sent: 02 August 2012 15:48
> To: statalist@hsphsun2.harvard.edu
> Subject: st: RE: RE: RE: RE: RE: RE: hausman, augmented test from Vince's
> code and xtoverid after xtivreg
> 
> Professor Mark,
> 
> Thanks again for your quick reply.
> 
> I was using Vince's code as an alternative to Hausman test (due to the
> problems I have: negative chi-square or "V_b-V_B not positive defined"). So,
> I guess I was testing the orthogonality conditions relating to exogenous
> regressors only... It would be the same that testing after xtreg then?

It's not easy to read someone else's uncommented code (at least for me), but that's what it looks like.  In effect, you're testing whether the RE estimates of the exogenous coeffs are different from the FE estimates, and since the answer is apparently "yes", you draw the usual conclusion that therefore the "between" orthogonality conditions for the exogenous regressors aren't valid.

> The procedure you describe: first applying Vince's code (choosing FE), and
> second, using "xtoverid" and check for the validity of instruments, is it
> correct?

Yes, that's what I had in mind.

HTH,
Mark

> It is worth to mention that I adapted Vince's code for the case "xtivreg"
> because his code was for "xtreg". (I hope this is correct...).
> I attach below the code I used (it is adapted to my model) obtain from:
> http://www.stata.com/statalist/archive/2005-08/msg00760.html (Vince
> Wiggins, Stata Corp).
> 
> Esther
> 
> ******************************* VINCE'S CODE
> ***************************
> * http://www.stata.com/statalist/archive/2005-08/msg00760.html *
> **********************************************************
> ******************
> 
> local id "ident"
> local depvar "lny_l"
> local varlist "medium large grupo intra1 inter1_p"
> xtivreg `depvar' `varlist' (lnRDs_l lnCFs_l= lag1RD lag2RD lag1CF lag2CF), re
> hausman, save
> 
> quietly xtivreg `depvar' `varlist' (lnRDs_l lnCFs_l= lag1RD lag2RD lag1CF
> lag2CF), fe hausman, less
> 
> tokenize `varlist'
> local i 1
> while "``i''" != "" {
>         qui by `id':  gen double mean`i' = sum(``i'') / _n
>         qui by `id':  replace mean`i' = mean`i'[_n]
>         qui by `id':  gen double diff`i' = ``i'' - mean`i'
>         local newlist `newlist' mean`i' diff`i'
>         local i = `i' + 1
> }
> 
> quietly xtivreg `depvar' `newlist' (lnRDs_l lnCFs_l= lag1RD lag2RD lag1CF
> lag2CF), re tempname b matrix `b' = e(b)
> 
> qui test mean1 = mean1 , notest         /* clear test */
> local i 2
> while "``i''" != "" {
>         if `b'[1, colnumb (`b', "mean`i'")] != 0 &      /*
>         */ `b'[1, colnumb (`b', "diff`i'")] != 0 {
>                 qui test mean`i' = diff`i' , accum notest
>         }
>         local i = `i' + 1
> }
> test
> 
> ________________________________________
> De: owner-statalist@hsphsun2.harvard.edu [owner-
> statalist@hsphsun2.harvard.edu] en nom de Schaffer, Mark E
> [M.E.Schaffer@hw.ac.uk] Enviat el: dijous, 2 / agost / 2012 14:13 Per a:
> statalist@hsphsun2.harvard.edu
> Tema: st: RE: RE: RE: RE: RE: hausman, augmented test from Vince's code and
> xtoverid after xtivreg
> 
> Esther,
> 
> > -----Original Message-----
> > From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
> > statalist@hsphsun2.harvard.edu] On Behalf Of ESTHER GOYA CARRILLO
> > Sent: 02 August 2012 09:27
> > To: statalist@hsphsun2.harvard.edu
> > Subject: st: RE: RE: RE: RE: hausman, augmented test from Vince's code
> > and xtoverid after xtivreg
> >
> > Professor Mark,
> >
> > Thanks again for your kind answer.
> >
> > Regarding the two additional orthogonality conditions, I'm familiar
> > with the first one, but I'm not sure about the second one... Are you
> > referring to the exogeneity assumption that the instruments have to
> > satisfy (i.e., the fact that instruments and errors are uncorrelated)?
> 
> Yes
> 
> > Because, as far as I know, this
> > has to be satisfied by FE specification also, am I wrong?
> 
> Sort of.  The RE spec will use the "within" orthogonality conditions relating to
> the excluded instruments also used by the FE spec, but it will also use the
> "between" orthogonality conditions relating to the excluded instruments.  In
> the implementation of these estimators, these correspond to the use of the
> demeaned excluded instruments (for the "within" orthogonality conditions)
> and the group mean excluded instruments (for the "between" orthogonality
> conditions).  These orthog conditions are what -xtoverid- tests.
> 
> The G2SLS RE estimator combines these (the re option of -xtivreg-), whereas
> the EC2SLS estimator uses the demeaned and mean vars separately (the
> ec2sls option of -xtivreg-).  That's why, when you run -xtoverid- after an RE
> estimation using -xtivreg-, the degrees of freedom of the overid test depend
> on which estimator you're using.
> 
> Returning to your original results, it looks like you (1) used Vince's code to
> test the orthogonality conditions relating to the exogenous regressors only,
> and rejected the RE "between" orthogonality conditions; (2) used xtoverid to
> test the orthog conditions relating to the excluded instruments only, and
> failed to reject all the orthog conditions relating to the excluded instruments.
> (You didn't provide the code for (1), so this is a bit of a guess.)
> 
> If so, a reasonable conclusion would be that you should use an FE
> specification, since the rejection of the RE spec via (1) was pretty emphatic.
> 
> HTH,
> Mark
> 
> NB: I really must finish off that version of -xtivreg2- that does RE estimation.
> Among other things, it would allow you to vary which orthog conditions are
> used for the exogenous vs. which are used for the excluded instruments.
> 
> > (I was thinking to use
> > "xtoverid" to check the validity of the instruments, once I choose the
> > correct specification).
> >
> > In my case, I try to estimate a production function with innovation as
> > explanatory variable. It's widely accepted that there is simultaneity
> > problems given the correlation between inputs and productivity shocks
> > (that are capture by the error tem). So, OLS estimation would be
> > inconsistent. For this reason, I want to use instrumental variable
> > (using as instruments the lagged values of the endogenous variables).
> > But, I don't know which specification is more appropriate (FE or
> > RE)... That is why I wanted to use Hausman (or other test, like Vince
> > propose) in order to check if there is correlation between u_i and
> > Xit, because if this was the case the RE spec would be inconsistent.
> > But I faced with the problems I mentioned in the other mail...and I'm not
> sure what to do now.
> >
> > So, I guess I wanted to test the first additional condition that you
> > mentioned above. But now I don't know if I have to test the second one
> also...
> >
> > Many thanks for your help!
> > Esther
> >
> > ________________________________________
> > De: owner-statalist@hsphsun2.harvard.edu [owner-
> > statalist@hsphsun2.harvard.edu] en nom de Schaffer, Mark E
> > [M.E.Schaffer@hw.ac.uk] Enviat el: dimecres, 1 / agost / 2012 18:33 Per a:
> > statalist@hsphsun2.harvard.edu
> > Tema: st: RE: RE: RE: hausman, augmented test from Vince's code and
> > xtoverid after xtivreg
> >
> > Esther,
> >
> > Can you clarify what your xtivreg RE vs. FE will test?  The RE
> > specification probably has two kinds of additional orthogonality
> > conditions that the FE spec doesn't use.
> >
> > The first would be the additional orthogonality conditions that the RE
> > estimator uses in the basic xtreg setting (no endogenous regressors),
> > i.e., the assumption that the exogenous regressors are orthogonal to
> > u_i (as well as to e_it, which the FE estimator also uses).
> >
> > The second would be additional orthogonality conditions that the RE
> > estimator uses that are associated with the excluded instruments.
> >
> > Which do you want to test?
> >
> > --Mark
> >
> >
> > > -----Original Message-----
> > > From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
> > > statalist@hsphsun2.harvard.edu] On Behalf Of ESTHER GOYA CARRILLO
> > > Sent: 31 July 2012 09:31
> > > To: statalist@hsphsun2.harvard.edu
> > > Subject: st: RE: RE: hausman, augmented test from Vince's code and
> > > xtoverid after xtivreg
> > >
> > > Professor Mark,
> > >
> > > Thanks a lot for your answer.
> > > Perhaps behaviour is consistent, but a little bit easy to misunderstand.
> > > However, as you mentioned, help file is very useful.
> > >
> > > Then, how do I compare FE vs RE after "xtivreg" if chi-square in
> > > Hausman test is negative or it has a "V_b- V_B not positive defined"?
> > > Should I follow [R] hausman (in favour of RE) or Vince's code (in
> > > favour of FE) is more appropriate?
> > >
> > > Thanks again,
> > > Esther
> > >
> > > ________________________________________
> > > De: owner-statalist@hsphsun2.harvard.edu [owner-
> > > statalist@hsphsun2.harvard.edu] en nom de Schaffer, Mark E
> > > [M.E.Schaffer@hw.ac.uk] Enviat el: dimarts, 31 / juliol / 2012 00:24 Per a:
> > > statalist@hsphsun2.harvard.edu
> > > Tema: st: RE: hausman, augmented test from Vince's code and xtoverid
> > > after xtivreg
> > >
> > > Esther,
> > >
> > > After xtivreg RE or FE estimation, and after xtivreg2 FE estimation,
> > > xtoverid reports an overid statistic relating to the endogenous
> > > variables being instrumented.  It isn't reporting an FE vs. RE test.
> > >
> > > After xtreg RE estimation, xtoverid reports a Hausman or
> > > Hausman-type test of FE vs. RE.  This can be interpreted as a kind
> > > of overidentification test relating to the (exogenous) regressors,
> > > as the help
> > file explains.
> > >
> > > I agree, this behaviour by xtoverid is not entirely consistent....
> > >
> > > HTH,
> > > Mark (xtoverid author)
> > >
> > >
> > > From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-
> > > statalist@hsphsun2.harvard.edu] On Behalf Of ESTHER GOYA CARRILLO
> > > Sent: 30 July 2012 11:41
> > > To: statalist@hsphsun2.harvard.edu
> > > Subject: st: hausman, augmented test from Vince's code and xtoverid
> > > after xtivreg
> > >
> > > Hi everyone,
> > >
> > > I am a PhD student working on my thesis now. I am struggling with a
> > > "dilemma" and I really appreciate if someone could help me.
> > > I am estimating a FE and RE model with Instrumental variable using
> > > panel data. So, I use xtivreg, fe and xtivreg, re commands. I want
> > > to compare both models and choose the correct one. Hi have two
> questions:
> > >
> > > 1) If I use "hausman" chi-square is negative. Then, and according to
> > > the [R] hausman, "we might interpret this as strong evidence that we
> > > cannot reject the null hypothesis". So, following this
> > > interpretation, I should work with RE (due to that we can assume
> > > that the regressors are uncorrelated with the group specific error (ui)).
> > >
> > > On the other hand, I have read Vince's post about hausman test
> > > (http://www.stata.com/statalist/archive/2005-08/msg00760.html). I
> > > have applied his code (many thanks!) to compare FE vs RE after
> > > xtivreg (I guess I can used this code, not only for xtreg but also for
> xtivreg).
> > > The results are below. P-value=0, so I reject the null hypothesis.
> > > Thus FE are preferred, is this correct?
> > >
> > > Then, my first question is: which is the correct option?
> > >
> > >
> > > 2) I also consider "xtoverid" option. I have read Professor Mark's
> > > post
> > > (http://www.stata.com/statalist/archive/2007-11/msg00721.html) and
> > > the online help for "xtoverid" command. Regarding to the post,
> > > Professor Mark said "the Sargan-Hansen statistic reported by
> > > xtoverid after xtivreg or xtreg is, in fact, an FE vs RE test".
> > > However, in the online help is written "A test for fixed vs. random
> > > effects is also a test of overidentifying restrictions, and xtoverid
> > > will report this after a standard panel data estimation with xtreg,
> > > re". But, here is not
> > consider "xtivreg" case...
> > >
> > > In order to check it, I use "xtoverid" after "xtreg, re" and the
> > > output of stata is in fact a FE vs RE test (results below). But I use
> "xtoverid"
> > after "xtivreg, re"
> > > and output of stata does not suggest that it is a FE vs RE test like
> > > in the previous case...  Moreover, p-value = 0.4112, so I cannot
> > > reject the null hypothesis. If this was a FE vs RE test, the
> > > conclusion would be that RE model is preferred (it is consistent and
> > > more efficient than FE). This is opposite to the result obtained
> > > from Vince's
> > code...
> > > Besides, I can perform "xtoverid" after "xtivreg, FE" (results below).
> > > In this case, p-value=0.3488... And I don't know how to interpret
> > > this result...which is the null hypothesis here?
> > >
> > > So, my second question is: can I use "xtoverid" after xtivreg to do
> > > an FE vs RE test? If the answer is yes, with which option: xtivreg,
> > > RE or xtivreg,
> > FE?
> > >
> > >
> > > Given all of these, I don't know if it's better use Vince's code or
> > > hausman test or "xtoverid"....after my "xtivreg" estimation, because
> > > the conclusions are completely different...
> > >
> > > I would be really grateful if someone could help me in any of these
> > > questions.
> > > Many thanks in advance,
> > > Esther
> > >
> > > **** RESULTS  FROM VINCE'S CODE **** . test
> > >
> > > ( 1) = 0
> > > ( 2)  mean2 - diff2 = 0
> > > ( 3)  mean3 - diff3 = 0
> > > ( 4)  mean4 - diff4 = 0
> > > ( 5)  mean5 - diff5 = 0
> > >        Constraint 1 dropped
> > >
> > >            chi2(  4) =  553.93
> > >          Prob > chi2 =    0.0000
> > >
> > > **** RESULTS  FROM XTOVERID  WITH XTREG **** . quietly xtreg lny_l
> > > medium large grupo intra1 inter1_p lnRDs_l lnCFs_l, re . xtoverid,
> > > robust
> > >
> > > Test of overidentifying restrictions: fixed vs random effects
> > > Cross-section time-series model: xtreg re  robust
> > > Sargan-Hansen statistic 1260.567  Chi-sq(7)   P-value = 0.0000
> > >
> > > **** RESULTS  FROM XTOVERID  WITH XTIVREG ****
> > > * xtoverid after xtivreg, re:
> > > . quietly xtivreg lny_l medium large grupo intra1 inter1_p (lnRDs_l
> > > lnCFs_l= lag1RD lag2RD lag1CF lag2CF), re . xtoverid, robust
> > >
> > > Test of overidentifying restrictions:
> > > Cross-section time-series model: xtivreg g2sls  robust
> > > Sargan-Hansen statistic   1.777  Chi-sq(2)    P-value = 0.4112
> > >
> > > * xtoverid after xtivreg, fe:
> > > . quietly xtivreg lny_l medium large grupo intra1 inter1_p (lnRDs_l
> > > lnCFs_l= lag1RD lag2RD lag1CF lag2CF), fe . xtoverid, robust
> > >
> > > Test of overidentifying restrictions:
> > > Cross-section time-series model: xtivreg fe  robust
> > > Sargan-Hansen statistic   2.107  Chi-sq(2)    P-value = 0.3488
> > >
> > >
> > > Aquest correu electrònic i els annexos poden contenir informació
> > > confidencial o protegida legalment i està adreçat exclusivament a la
> > > persona o entitat destinatària. Si no sou el destinatari final o la
> > > persona encarregada de rebre'l, no esteu autoritzat a llegir-lo,
> > > retenir-lo, modificar-lo, distribuir-lo, copiar-lo ni a revelar-ne
> > > el contingut. Si heu rebut aquest correu electrònic per error, us
> > > preguem que n'informeu al remitent i que elimineu del sistema el
> > > missatge i el material annex que pugui contenir. Gràcies per la vostra
> col·laboració.
> > >
> > > Este correo electrónico y sus anexos pueden contener información
> > > confidencial o legalmente protegida y está exclusivamente dirigido a
> > > la persona o entidad destinataria. Si usted no es el destinatario
> > > final o la persona encargada de recibirlo, no está autorizado a
> > > leerlo, retenerlo, modificarlo, distribuirlo, copiarlo ni a revelar
> > > su contenido. Si ha recibido este mensaje electrónico por error, le
> > > rogamos que informe al remitente y elimine del sistema el mensaje y
> > > el
> > material anexo que pueda contener.
> > > Gracias por su colaboración.
> > >
> > > This email message and any documents attached to it may contain
> > > confidential or legally protected material and are intended solely
> > > for the use of the individual or organization to whom they are addressed.
> > > We remind you that if you are not the intended recipient of this
> > > email message or the person responsible for processing it, then you
> > > are not authorized to read, save, modify, send, copy or disclose any
> > > of its contents. If you have received this email message by mistake,
> > > we kindly ask you to inform the sender of this and to eliminate both
> > > the message and any attachments it carries from your account. Thank
> > > you for
> > your collaboration.
> > >
> > >
> > > --
> > > Heriot-Watt University is the Sunday Times Scottish University of
> > > the Year
> > > 2011-2012
> > >
> > > We invite research leaders and ambitious early career researchers to
> > > join us in leading and driving research in key inter-disciplinary themes.
> > > Please see www.hw.ac.uk/researchleaders for further information and
> > > how to apply.
> > >
> > > Heriot-Watt University is a Scottish charity registered under
> > > charity number SC000278.
> > >
> > >
> > > *
> > > *   For searches and help try:
> > > *   http://www.stata.com/help.cgi?search
> > > *   http://www.stata.com/support/statalist/faq
> > > *   http://www.ats.ucla.edu/stat/stata/
> > >
> > > *
> > > *   For searches and help try:
> > > *   http://www.stata.com/help.cgi?search
> > > *   http://www.stata.com/support/statalist/faq
> > > *   http://www.ats.ucla.edu/stat/stata/
> >
> >
> > --
> > Heriot-Watt University is the Sunday Times Scottish University of the
> > Year
> > 2011-2012
> >
> > We invite research leaders and ambitious early career researchers to
> > join us in leading and driving research in key inter-disciplinary themes.
> > Please see www.hw.ac.uk/researchleaders for further information and
> > how to apply.
> >
> > Heriot-Watt University is a Scottish charity registered under charity
> > number SC000278.
> >
> >
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/help.cgi?search
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
> >
> >
> > Aquest correu electrònic i els annexos poden contenir informació
> > confidencial o protegida legalment i està adreçat exclusivament a la
> > persona o entitat destinatària. Si no sou el destinatari final o la
> > persona encarregada de rebre'l, no esteu autoritzat a llegir-lo,
> > retenir-lo, modificar-lo, distribuir-lo, copiar-lo ni a revelar-ne el
> > contingut. Si heu rebut aquest correu electrònic per error, us preguem
> > que n'informeu al remitent i que elimineu del sistema el missatge i el
> > material annex que pugui contenir. Gràcies per la vostra col·laboració.
> >
> > Este correo electrónico y sus anexos pueden contener información
> > confidencial o legalmente protegida y está exclusivamente dirigido a
> > la persona o entidad destinataria. Si usted no es el destinatario
> > final o la persona encargada de recibirlo, no está autorizado a
> > leerlo, retenerlo, modificarlo, distribuirlo, copiarlo ni a revelar su
> > contenido. Si ha recibido este mensaje electrónico por error, le
> > rogamos que informe al remitente y elimine del sistema el mensaje y el
> material anexo que pueda contener.
> > Gracias por su colaboración.
> >
> > This email message and any documents attached to it may contain
> > confidential or legally protected material and are intended solely for
> > the use of the individual or organization to whom they are addressed.
> > We remind you that if you are not the intended recipient of this email
> > message or the person responsible for processing it, then you are not
> > authorized to read, save, modify, send, copy or disclose any of its
> > contents. If you have received this email message by mistake, we
> > kindly ask you to inform the sender of this and to eliminate both the
> > message and any attachments it carries from your account. Thank you for
> your collaboration.
> >
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/help.cgi?search
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
> 
> 
> --
> Heriot-Watt University is the Sunday Times Scottish University of the Year
> 2011-2012
> 
> We invite research leaders and ambitious early career researchers to join us
> in leading and driving research in key inter-disciplinary themes.
> Please see www.hw.ac.uk/researchleaders for further information and how
> to apply.
> 
> Heriot-Watt University is a Scottish charity registered under charity number
> SC000278.
> 
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
> 
> 
> Aquest correu electrònic i els annexos poden contenir informació
> confidencial o protegida legalment i està adreçat exclusivament a la persona
> o entitat destinatària. Si no sou el destinatari final o la persona encarregada
> de rebre'l, no esteu autoritzat a llegir-lo, retenir-lo, modificar-lo, distribuir-lo,
> copiar-lo ni a revelar-ne el contingut. Si heu rebut aquest correu electrònic
> per error, us preguem que n'informeu al remitent i que elimineu del sistema
> el missatge i el material annex que pugui contenir. Gràcies per la vostra
> col·laboració.
> 
> Este correo electrónico y sus anexos pueden contener información
> confidencial o legalmente protegida y está exclusivamente dirigido a la
> persona o entidad destinataria. Si usted no es el destinatario final o la
> persona encargada de recibirlo, no está autorizado a leerlo, retenerlo,
> modificarlo, distribuirlo, copiarlo ni a revelar su contenido. Si ha recibido este
> mensaje electrónico por error, le rogamos que informe al remitente y
> elimine del sistema el mensaje y el material anexo que pueda contener.
> Gracias por su colaboración.
> 
> This email message and any documents attached to it may contain
> confidential or legally protected material and are intended solely for the use
> of the individual or organization to whom they are addressed. We remind
> you that if you are not the intended recipient of this email message or the
> person responsible for processing it, then you are not authorized to read,
> save, modify, send, copy or disclose any of its contents. If you have received
> this email message by mistake, we kindly ask you to inform the sender of this
> and to eliminate both the message and any attachments it carries from your
> account. Thank you for your collaboration.
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/


-- 
Heriot-Watt University is the Sunday Times
Scottish University of the Year 2011-2012

We invite research leaders and ambitious early career researchers to 
join us in leading and driving research in key inter-disciplinary themes. 
Please see www.hw.ac.uk/researchleaders for further information and how
to apply.

Heriot-Watt University is a Scottish charity
registered under charity number SC000278.


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index