st: RE: RE: RE: xtivreg2: orthog option

["Schaffer, Mark E" <M.E.Schaffer@hw.ac.uk>]

James,

> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu 
> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of 
> Fitzgerald, James
> Sent: 05 July 2012 14:57
> To: statalist@hsphsun2.harvard.edu
> Subject: st: RE: RE: xtivreg2: orthog option
> 
> Mark,
> 
> ________________________________________
> From: owner-statalist@hsphsun2.harvard.edu 
> [owner-statalist@hsphsun2.harvard.edu] on behalf of Schaffer, 
> Mark E [M.E.Schaffer@hw.ac.uk]
> Sent: 05 July 2012 13:48
> To: statalist@hsphsun2.harvard.edu
> Subject: st: RE: RE: RE: RE: RE: RE: RE: RE: RE: RE: 
> xtivreg2: orthog option
> 
> James,
> 
> 
> > -----Original Message-----
> > From: owner-statalist@hsphsun2.harvard.edu
> > [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of
> > Fitzgerald, James
> > Sent: 05 July 2012 11:15
> > To: statalist@hsphsun2.harvard.edu
> > Subject: st: RE: RE: RE: RE: RE: RE: RE: RE: RE: xtivreg2:
> > orthog option
> >
> > Mark,
> >
> > I followed your suggestion as far as I understood it. As
> > such, I undertook the following steps:
> >
> > 1. I estimated the model with suspect instruments treated as
> > endogenous. As I have no reason to suspect any one regressor
> > is endogenous and others are not, I ran the model with all
> > regressors assumed to be endogenous and used 3 lags as
> > exluded instruments.
> >
> > xtivreg2 ltdbv yr* (lnsale tang itang itangdum tax prof mtb
> > capexsa liq ndts=l.lnsale l2.lnsale l3.lnsale l.tang l2.tang
> > l3.tang l.itang l2.itang l3.itang l.itangdum l2.itangdum
> > l3.itangdum l.tax l2.tax l3.tax l.prof l2.prof l3.prof l.mtb
> > l2.mtb l3.mtb l.capexsa l2.capexsa l3.capexsa l.liq l2.liq
> > l3.liq l.ndts l2.ndts l3.ndts), fe cluster(firm) gmm2s
> >
> > The p-value on the Hansen J-Stat turned out to be 0.01.
> >
> > 2. I then tested the orthogonality of the different lags
> >      orthog(l.lnsale l.tang . . . l.ndts) gave a C stat
> > p-value of 0.5196
> >      orthog(l2.lnsale l2.tang . . . l2.ndts) gave a C stat
> > p-value of 0.3318
> >      orthog(l3.lnsale l3.tang . . . l3.ndts) gave a C stat
> > p-value of 0.0022
> >
> > 3. I dropped the l3 lags and the Hansen J Stat p-value was 0.5588.
> >     I then used the endog option on each of the endogenous
> > variables i.e.
> >
> > xtivreg2 ltdbv yr* (lnsale tang itang itangdum tax prof mtb
> > capexsa liq ndts=l.lnsale l2.lnsale l3.lnsale l.tang l2.tang
> > l.itang l2.itang l.itangdum l2.itangdum l.tax l2.tax l.prof
> > l2.prof l.mtb l2.mtb l.capexsa l2.capexsa l.liq l2.liq l.ndts
> > l2.ndts), fe cluster(firm) gmm2s endog(lnsale)
> >
> > And replaced lnsale with tang, itang etc.
> >
> > 4. All the endog tests indicated the regressors are not
> > endogenous, so I conclude there is no need to use xtivreg2,
> > fe and instead I can use xtreg, fe
> >
> > How does this sound??
> >
> > James
> > ________________________________________
> 
> <snip>
> 
> This looks reasonable.  Just a few thoughts:
> 
> In steps 1-2, it looks like you are getting a large C stat for L3
> because L1 and L2 are identifying one beta_hat, and L3 is 
> identifying a
> different beta_hat.  At least one of these two beta_hats must be
> inconsistent.  You're concluding that the 2nd one is inconsistent, and
> so you're dropping the L3s as IVs.
> 
> This could be defensible, but it looks a bit odd.  The more usual case
> is that older lags are more likely to be valid IVs than recent lags.
> 
> An alternative interpretation of your results is that the 1st beta_hat
> is inconsistent, and so you should drop the L1s and L2s and 
> use just the
> L3s as IVs.  You might want to try that and see what happens. 
>  (There's
> no point doing a C test for the L1s and L2s, by the way, because using
> just the L3s gives you an exactly identified equation, and the C stat
> will the same large J stat you got when you used all the IVs.) 
> 
> I just tried this and I found that all my estimates become 
> completely insignificant when I use L3s as IVs, but are 
> aprroximately what would be expected when i use L1s and L2s. 
> Also, the underidentification statistic is completely 
> insignificant with the L3s, but marginally significant when I 
> use the L1s and L2s

This is a problem.  The weak ID stat with L1s and L2s is probably very
low, suggesting that even your L1-L2-based estimates aren't reliable, or
more precisely, at least one of the coeffs in the beta_hat vector isn't
well identified.

See also below.

> (I think it is only marginally 
> significant as for some of the regressors the lags may not be 
> good instruments).
> Does this suggest that beta_hat based on L1 and L2 is consistent?

Not quite.  It suggests that the beta_hat based on L3 is inconsistent,
or to be more precise, at least one of the coeffs in the beta_hat vector
is inconsistent.

> Also, in step 3, you can test for the endogeneity of all your
regressors
> lnsale-ndts all at once - the endog option takes varlists.
> 
> When I test them one at a time (employing L1 and L2 as lags) 
> I get the following endogeneity test p-values:
> lnsale = 0.6859
> tang = 0.2336
> itang = 0.7719
> itangdum = 0.001
> tax = 0.0068
> prof = 0.7691
> mtb = 0.7357
> capexsa = 0.2933
> liq = 0.2511
> ndts = 0.5358
> 
> I conclude that itangdum and tax need to be instrumented.
> Please ignore my earlier comment that all regressors are exogenous!

But be a bit careful here.  There are 11 coeffs.  You shouldn't be too
surprised if p-values for the endogeneity tests are spread around -
that's what you would expect to see under the null of exogeneity.  Some
big p-values, some small, some in-between.

> When i test for the endogeneity of all my regressors at once 
> I get a p-value of 0.0002.
> This tells me that one or more of my regressors are indeed endogenous

Which is an effective rejoinder to my point just above.  But see also
below.

> Given the p-values from the individual endog tests I now 
> specify itangdum and tax as endogenous, and the other 
> variables as exogenous.
> To confirm the other variables are exogenous, I specify 
> orthog(varlist) and I get a C Stat p-value of 0.4742.
> 
> Does this seem right?
> 
> Now I am left with the issue of assessing the "strength" of 
> the instruments.

Ah - you should have done this first.  The tests for orthogonality,
endogeneity, etc., all assume that the underlying IV/GMM estimations are
well-specified, and that includes being strongly identified.  See my
note above.

> 
> I get the following statistics (I have kept all of the 
> excluded instruments i.e. L1s and L2s of all 10 explanatory variables)
> 
> Summary results for first-stage regressions
> 
>                              (Underid)                        
>                      (Weak id)
> Variable      F( 20,  1049)  P-val  AP Chi-sq( 19) P-val  AP 
> F( 19,  1049)
> itangdum          111.58    0.0000      2194.99   0.0000       114.94
> tax                      3.66    0.0000          72.32   
> 0.0000         3.79
> NB: first-stage test statistics cluster-robust
> Stock-Yogo weak ID test critical values for single endogenous 
> regressor:
> 5% maximal IV relative bias    21.38
> 10% maximal IV relative bias    11.46
> 20% maximal IV relative bias     6.31
> 30% maximal IV relative bias     4.51
> 10% maximal IV size             59.92
> 15% maximal IV size             31.58
> 20% maximal IV size             21.90
> 25% maximal IV size             16.99
> Source: Stock-Yogo (2005).  Reproduced by permission.
> NB: Critical values are for Cragg-Donald F statistic and 
> i.i.d. errors.
> Underidentification test
> Ho: matrix of reduced form coefficients has rank=K1-1 
> (underidentified)
> Ha: matrix has rank=K1 (identified)
> Kleibergen-Paap rk LM statistic          Chi-sq(19)=59.63   
> P-val=0.0000
> Weak identification test
> Ho: equation is weakly identified
> Cragg-Donald Wald F statistic                                 
>       5.56
> Kleibergen-Paap Wald rk F statistic                           
>       3.60
> Stock-Yogo weak ID test critical values for K1=2 and L1=20:
> 5% maximal IV relative bias    20.48
> 10% maximal IV relative bias    11.03
> 20% maximal IV relative bias     6.11
> 30% maximal IV relative bias     4.39
> 10% maximal IV size             46.62
> 15% maximal IV size             24.96
> 20% maximal IV size             17.61
> 25% maximal IV size             13.84
> Source: Stock-Yogo (2005).  Reproduced by permission.
> NB: Critical values are for Cragg-Donald F statistic and 
> i.i.d. errors.
> Weak-instrument-robust inference
> Tests of joint significance of endogenous regressors B1 in 
> main equation
> Ho: B1=0 and orthogonality conditions are valid
> Anderson-Rubin Wald test           F(20,1049)=     2.82     
> P-val=0.0000
> Anderson-Rubin Wald test           Chi-sq(20)=    56.61     
> P-val=0.0000
> Stock-Wright LM S statistic        Chi-sq(20)=    47.94     
> P-val=0.0004
> NB: Underidentification, weak identification and 
> weak-identification-robust
> test statistics cluster-robust
> 
> 
> My intuition is that the stats relating to itangdum are 
> strong, but the stats relating to tax are weak.

That looks right, though strictly speaking you shouldn't use the A-P
stats like that.  They're actually meant for the case where you are
interested a priori in one coeff and not in another.

> I specify the first option and STATA

It's "Stata", by the way.

> generates the first 
> stage regressions of tax and itangdum. The results suggest 
> that many of the instruments do not explain variation in 
> either variable.
> Can I remove these instruments and, as long as my Hansen J 
> stat indicates the remaining excluded instruments are still 
> valid, still conclude that the variables specified as 
> exogenous can still be considered exogenous? The reason I 
> want to do this is that I find that the weak i.d stats often 
> improve dramatically when these instruments are removed.

Too much specification tweaking makes me uneasy, but that's my personal
view.  Maybe someone else wants to comment.

> Also, if I find an instrument to be weak, as I believe tax 
> is,

tax is a regressor, not an instrument.  I think I know what you mean,
though.

> should I; drop tax from the model, leave the instrument 
> in and just conclude that it is uninterpretable, or specify 
> tax as exogenous but that it is uninterpretable?

Dropping tax is defensible.  So is specifying it as exogenous -
including it doesn't necessary mean the results are uninterpretable.

I'm running out of steam on this thread and have to turn to other
things.  But I can see you're on top of the issues now.  And perhaps
someone else will want to comment.

--Mark

> 
> Thanks again
> 
> James
> 
> 
> 
> 
> 
> Cheers,
> Mark
> 
> 
> --
> Heriot-Watt University is the Sunday Times
> Scottish University of the Year 2011-2012
> 
> Heriot-Watt University is a Scottish charity
> registered under charity number SC000278.
> 
> 
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
> 


-- 
Heriot-Watt University is the Sunday Times
Scottish University of the Year 2011-2012

Heriot-Watt University is a Scottish charity
registered under charity number SC000278.


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/