|
|  |
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]
RE: st: ivreset
Emails crossing in the post... I think you're probably right, though.
--Mark
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of
> Arne Risa Hole
> Sent: 20 March 2006 18:08
> To: [email protected]
> Subject: Re: st: ivreset
>
> Mark,
>
> Note, however, that it is only because of the collinearity
> problem - the fact that one variable is dropped from the
> model when the forecast is not rescaled - that the test
> statistics differ in your OLS example.
> The rescaling does not matter in a test with only the squared
> and cubed forecasts for example.
>
> This is different in the case of IV: even when collinearity
> is not a problem the test statistics differ, as the example
> of running Austin's code with and without the rescaling shows.
>
> The solution is probably to rescale after creating the
> polynomials as you suggest - this way of rescaling the
> forecasts does not affect the test statistic in either case,
> IV or OLS, and may solve the numerical problems that using
> the actual forcast might introduce - on the other hand it
> doesn't seem to sort out the collinearity so this problem remains..
>
> Cheers,
> Arne
>
>
>
>
> On 20/03/06, Schaffer, Mark E <[email protected]> wrote:
> > Arne,
> >
> > I think you are onto something here. The rescaling should perhaps
> > take place *after* creating the polynomials, not before.
> >
> > That said, Stata's official -ovtest- also rescales first,
> then creates
> > the polynomials. When I wrote -ivreset-, I had used replication of
> > the output of -ovtest- as a check, and I used the same approach to
> > rescaling yhat, namely rescale and then create polynomials (rather
> > than create polynomials and then rescale).
> >
> > Here is an example that demonstrates that (a) rescaling
> after creating
> > the polynomials leaves the reset statistic unchanged, and (b)
> > rescaling before creating the polynomials replicates the output of
> > official -ovtest-. Note that there are collinearity
> problems with (a)
> > even after rescaling, which makes me hesitate...
> >
> > Does anyone else want to comment?
> >
> > --Mark
> >
> > *********** do code ***************
> >
> > use http://fmwww.bc.edu/ec-p/data/hayashi/griliches76.dta, clear
> > capture drop yhat* qui regress lw s qui predict double yhat
> > * yhatr=rescaled yhat
> > sum yhat, meanonly
> > qui gen double yhatr = (yhat-r(min))/(r(max)-r(min)) qui gen double
> > yhat2=yhat^2 qui gen double yhat3=yhat^3 qui gen double
> yhat4=yhat^4
> > qui gen double yhatr2=yhatr^2 qui gen double yhatr3=yhatr^3 qui gen
> > double yhatr4=yhatr^4
> > * yhatrr2=rescaled yhat2
> > * yhatrr3=rescaled yhat3
> > * yhatrr4=rescaled yhat4
> > sum yhat2, meanonly
> > qui gen double yhatrr2 = (yhat2-r(min))/(r(max)-r(min)) sum yhat3,
> > meanonly qui gen double yhatrr3 =
> (yhat3-r(min))/(r(max)-r(min)) sum
> > yhat4, meanonly qui gen double yhatrr4 =
> > (yhat4-r(min))/(r(max)-r(min)) sum yhat*
> >
> > * Unrescaled RESET
> > qui regress lw s yhat2 yhat3 yhat4
> > testparm yhat*
> > * yhat that is first ^2, ^3, ^4, then rescaled
> > * Output identical to unrescaled RESET qui regress lw s yhatrr2
> > yhatrr3 yhatrr4 testparm yhat*
> > * yhat that is first rescaled, then ^2, ^3, ^4
> > * Output different from unrescaled RESET qui regress lw s yhatr2
> > yhatr3 yhatr4 testparm yhat*
> > * Stata's built-in ovtest
> > * Output again different from unrescaled RESET qui regress
> lw s Ovtest
> >
> > ************* output ****************
> >
> >
> > . use http://fmwww.bc.edu/ec-p/data/hayashi/griliches76.dta, clear
> > (Wages of Very Young Men, Zvi Griliches, J.Pol.Ec. 1976)
> >
> > . capture drop yhat*
> >
> > . qui regress lw s
> >
> > . qui predict double yhat
> >
> > . * yhatr=rescaled yhat
> > . sum yhat, meanonly
> >
> > . qui gen double yhatr = (yhat-r(min))/(r(max)-r(min))
> >
> > . qui gen double yhat2=yhat^2
> >
> > . qui gen double yhat3=yhat^3
> >
> > . qui gen double yhat4=yhat^4
> >
> > . qui gen double yhatr2=yhatr^2
> >
> > . qui gen double yhatr3=yhatr^3
> >
> > . qui gen double yhatr4=yhatr^4
> >
> > . * yhatrr2=rescaled yhat2
> > . * yhatrr3=rescaled yhat3
> > . * yhatrr4=rescaled yhat4
> > . sum yhat2, meanonly
> >
> > . qui gen double yhatrr2 = (yhat2-r(min))/(r(max)-r(min))
> >
> > . sum yhat3, meanonly
> >
> > . qui gen double yhatrr3 = (yhat3-r(min))/(r(max)-r(min))
> >
> > . sum yhat4, meanonly
> >
> > . qui gen double yhatrr4 = (yhat4-r(min))/(r(max)-r(min))
> >
> > . sum yhat*
> >
> > Variable | Obs Mean Std. Dev. Min
> Max
> >
> -------------+--------------------------------------------------------
> > yhat | 758 5.686739 .2156493 5.261107
> 6.130727
> > yhatr | 758 .4894459 .2479809 0
> 1
> > yhat2 | 758 32.38544 2.473995 27.67924
> 37.58581
> > yhat3 | 758 184.7003 21.3139 145.6234
> 230.4284
> > yhat4 | 758 1054.929 163.4247 766.1405
> 1412.693
> >
> -------------+--------------------------------------------------------
> > yhatr2 | 758 .3009707 .2769761 0
> 1
> > yhatr3 | 758 .2142687 .2681644 0
> 1
> > yhatr4 | 758 .1671979 .2530434 0
> 1
> > yhatrr2 | 758 .4750582 .2497327 0
> 1
> > yhatrr3 | 758 .4607848 .2513286 0
> 1
> >
> -------------+--------------------------------------------------------
> > yhatrr4 | 758 .4466593 .252763 0
> 1
> >
> > .
> > . * Unrescaled RESET
> > . qui regress lw s yhat2 yhat3 yhat4
> >
> > . testparm yhat*
> >
> > ( 1) yhat2 = 0
> > ( 2) yhat3 = 0
> > ( 3) yhat4 = 0
> > Constraint 1 dropped
> >
> > F( 2, 754) = 0.87
> > Prob > F = 0.4191
> >
> > . * yhat that is first ^2, ^3, ^4, then rescaled . * Output
> identical
> > to unrescaled RESET . qui regress lw s yhatrr2 yhatrr3 yhatrr4
> >
> > . testparm yhat*
> >
> > ( 1) yhatrr2 = 0
> > ( 2) yhatrr3 = 0
> > ( 3) yhatrr4 = 0
> > Constraint 1 dropped
> >
> > F( 2, 754) = 0.87
> > Prob > F = 0.4191
> >
> > . * yhat that is first rescaled, then ^2, ^3, ^4 . * Output
> different
> > from unrescaled RESET . qui regress lw s yhatr2 yhatr3 yhatr4
> >
> > . testparm yhat*
> >
> > ( 1) yhatr2 = 0
> > ( 2) yhatr3 = 0
> > ( 3) yhatr4 = 0
> >
> > F( 3, 753) = 0.59
> > Prob > F = 0.6216
> >
> > . * Stata's built-in ovtest
> > . * Output again different from unrescaled RESET . qui regress lw s
> >
> > . ovtest
> >
> > Ramsey RESET test using powers of the fitted values of lw
> > Ho: model has no omitted variables
> > F(3, 753) = 0.59
> > Prob > F = 0.6216
> >
> > .
> > end of do-file
> >
> > *************************************
> >
> > > -----Original Message-----
> > > From: [email protected]
> > > [mailto:[email protected]] On Behalf
> Of Arne Risa
> > > Hole
> > > Sent: 20 March 2006 16:03
> > > To: [email protected]
> > > Subject: Re: st: ivreset
> > >
> > > Hi Mark,
> > >
> > > Thank you very much for the clarifications. The rescaling
> does have
> > > an impact on the test statistic, however; this can be seen from
> > > using Austin's code and comparing the results with and
> without the
> > > line:
> > >
> > > replace yhat = (yhat-r(min))/(r(max)-r(min))
> > >
> > > So even when the correct "optimal" forecast of yhat is used
> > > (yhat=X-hat*beta-hat), rescaling the forecast affects the result.
> > > This is not a problem in the case of -ovtest-, however, since the
> > > Reset test statistic is invariant to the rescaling in the
> OLS case.
> > >
> > > Sorry for going on about this, but it seems to me that
> since the two
> > > statistics differ the correct statistic is the one without the
> > > rescaling of the (yhat=X-hat*beta-hat) forecast (even though this
> > > may introduce numerical precision problems in some cases).
> > >
> > > Cheers,
> > > Arne
> > >
> > > On 20/03/06, Schaffer, Mark E <[email protected]> wrote:
> > > > Arne,
> > > >
> > > > > -----Original Message-----
> > > > > From: [email protected]
> > > > > [mailto:[email protected]] On Behalf
> > > Of Arne Risa
> > > > > Hole
> > > > > Sent: 18 March 2006 11:43
> > > > > To: [email protected]
> > > > > Subject: RE: st: ivreset
> > > > >
> > > > > Austin, Mark,
> > > > >
> > > > > Thank you both for your replies, rescaling the
> forecast did the
> > > > > trick (sorry for the bad formatting of my code
> before, it looked
> > > > > fine in my email programme).
> > > > >
> > > > > I understand the motivation behind the rescaling, but I'm
> > > slightly
> > > > > concerned about the fact that it produces a different
> > > test statistic
> > > > > compared to using the actual forecast. Note that this
> is not the
> > > > > case when using the Reset test following OLS - the test
> > > statistic is
> > > > > invariant to the rescaling in this case.
> > > > >
> > > > > I would think that since the two approaches (rescaling/ no
> > > > > rescaling) produce different results, the correct test
> > > statistic is
> > > > > that using the actual forecast?
> > > >
> > > > There are two different issues here. Rescaling is one, and
> > > it is, in
> > > > some sense, a side issue. The problem is that sometimes
> > > the yhat has
> > > > large-ish values, and the higher order polynomials of yhat that
> > > > are included in the artificial regression can get so
> big that the
> > > > regression fails for numerical precision reasons. Stata's
> > > own version
> > > > of the reset test, -ovtest-, also does this rescaling.
> The test
> > > > statistic is, of course, invariant in theory to the units
> > > used and hence to rescaling.
> > > >
> > > > The other issue is the one you might have missed. As
> > > Austin pointed
> > > > out, the IV version of the RESET test cannot use
> standard fitted
> > > > values that would be generated by -predict- after
> estimation using
> > > > -ivreg- or -ivreg2-. These are yhat=X*beta-hat, and
> the problem
> > > > is that X includes some endogenous regressors.
> > > >
> > > > As the help file for -ivreset- explains, there are two
> alternatives.
> > > > One is to use reduced form predictions for yhat, i.e.,
> regress y
> > > > on all the exogenous variables (including the excluded
> > > instruments) and
> > > > then use -predict-. The other is to get what Pesaran and
> > > Taylor call
> > > > the "optimal forecast" of yhat. This is not
> yhat=X*beta-hat, but
> > > > yhat=X-hat*beta-hat, where X-hat includes the reduced form
> > > predicted
> > > > values of the endogenous regressors (rather than the
> actual values).
> > > > The code that Austin kindly posted to Statalast implemented
> > > the latter.
> > > >
> > > > Cheers,
> > > > Mark
> > > >
> > > > > Cheers
> > > > > Arne
> > > > >
> > > > > On Mar 17 2006, Schaffer, Mark E wrote:
> > > > >
> > > > > > Austin, Arne,
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Austin Nichols [mailto:[email protected]]
> > > > > > > Sent: 16 March 2006 23:30
> > > > > > > To: [email protected]
> > > > > > > Subject: Re: st: ivreset
> > > > > > >
> > > > > > > -findit ivreset- then -help ivreset- when installed has a
> > > > > excellent
> > > > > > > exposition that begins:
> > > > > > >
> > > > > > > As Pagan and Hall (1983) and Pesaran and Taylor (1999)
> > > > > point out, a
> > > > > > > RESET test for an IV regression cannot use the standard
> > > > > IV predicted
> > > > > > > values X*beta-hat, because X includes endogenous
> > > > > regressors that are
> > > > > > > correlated with u.
> > > > > > >
> > > > > > > Try this code instead:
> > > > > > >
> > > > > > > use http://fmwww.bc.edu/ec-p/data/hayashi/griliches76.dta
> > > > > > > qui ivreg2 lw s expr tenure rns smsa (iq=med kww) predict
> > > > > ytilde mat
> > > > > > > b=e(b) mat li b qui regress iq s expr tenure rns smsa med
> > > > > > > kww qui predict double xh gen
> yhat=ytil-b[1,1]*iq+b[1,1]*xh
> > > > > > > sum yhat, meanonly qui replace yhat =
> > > (yhat-r(min))/(r(max)-r(min))
> > > > > > > qui gen double yhat2=yhat^2 qui ivreg2 lw s expr
> > > tenure rns smsa
> > > > > > > yhat2 (iq=med kww) test yhat2 qui ivreg2 lw s expr tenure
> > > > > > > rns
> > > > > smsa (iq=med
> > > > > > > kww) ivreset
> > > > > > >
> > > > > > > Now, as to why
> > > > > > > replace yhat = (yhat-r(min))/(r(max)-r(min)) I cannot
> > > > > tell you, but
> > > > > > > it's in ivreset.ado
> > > > > >
> > > > > > It's to rescale yhat so that when it's squared,
> cubed, etc.,
> > > > > > it doesn't get wildly out of scale relative to the other
> > > regressors.
> > > > > > This can cause problems for the regression that includes
> > > > > these terms.
> > > > > >
> > > > > > Cheers,
> > > > > > Mark
> > > > > >
> > > > > >
> > > > > > > On 16 Mar 2006 19:10:13 +0000, Arne Risa Hole wrote:
> > > > > > > > I am using ivreset to do a Pesaran-Taylor Reset test
> > > > > after ivreg2.
> > > > > > > > However, I am not able to replicate the result from
> > > > > > > > ivreset
> > > > > > > manually. For example:
> > > > > > > >
> > > > > > > > use
> http://fmwww.bc.edu/ec-p/data/hayashi/griliches76.dta
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > *
> > > > > > * For searches and help try:
> > > > > > * http://www.stata.com/support/faqs/res/findit.html
> > > > > > * http://www.stata.com/support/statalist/faq
> > > > > > * http://www.ats.ucla.edu/stat/stata/
> > > > > >
> > > > >
> > > > > *
> > > > > * For searches and help try:
> > > > > * http://www.stata.com/support/faqs/res/findit.html
> > > > > * http://www.stata.com/support/statalist/faq
> > > > > * http://www.ats.ucla.edu/stat/stata/
> > > > >
> > > > >
> > > >
> > > > *
> > > > * For searches and help try:
> > > > * http://www.stata.com/support/faqs/res/findit.html
> > > > * http://www.stata.com/support/statalist/faq
> > > > * http://www.ats.ucla.edu/stat/stata/
> > > >
> > >
> > > *
> > > * For searches and help try:
> > > * http://www.stata.com/support/faqs/res/findit.html
> > > * http://www.stata.com/support/statalist/faq
> > > * http://www.ats.ucla.edu/stat/stata/
> > >
> > >
> >
> > *
> > * For searches and help try:
> > * http://www.stata.com/support/faqs/res/findit.html
> > * http://www.stata.com/support/statalist/faq
> > * http://www.ats.ucla.edu/stat/stata/
> >
>
> *
> * For searches and help try:
> * http://www.stata.com/support/faqs/res/findit.html
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
>
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
