[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
"Thomas Jacobs" <thomasjacobs@gmail.com> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
Re: st: Troubleshooting 'not sorted' and 'not regularly spaced' errors in newey and newey2 |

Date |
Mon, 13 Oct 2008 11:16:43 -0500 |

Mike, I can't thank you enough for the newey "clinic" you provided. I confirmed all of your points with both your sample data as well as my own. That was a big help. Tom On Sat, Oct 11, 2008 at 11:14 AM, Michael Hanson <mshanson@mac.com> wrote: > I have seen this problem before. The bottom line seems to be: > > 1. -newey2- does not recognize time-series operators in Stata. (That may be > because -newey2- was last updated in early 2004.) The "not sorted" error is > what you seen instead. To get around this problem, first create each lagged > variable as its own series before estimating your regressions. For example: > > gen L1_LnRtFiveYrMid = L1.LnRtFiveYrMid > gen L2_LnRtFiveYrMid = L2.LnRtFiveYrMid > etc. > > and then use the appropriate L?_* variables as regressors. > > 2. -newey- and -newey2- do not work if there are any "gaps" in your data in > time. (This should not be surprising, as it is not possible to produce > standard errors that are robust to serial correlation without sequential > observations in time.) You can get around this problem by either using the > -force- option in -newey2-, or by constructing a linear time series for your > observations in sequence and -tsset- or -xtset- to that series. You should > carefully explore whether the statistics you get from such a procedure are > meaningful, however. (I suspect this also means you will need to work with > a balanced panel, but I leave that as an exercise.) > > Below is a sample program, using a dataset made generally available through > StataCorp, that illustrates these points. Note that if you try to run this > program in its entirety, it will halt at the first line that produces an > error -- and there are several errors by the nature of this demonstration > program. I've noted with a comment after each estimation command whether it > "Works!" or, if it fails, what error message it produces. > > HTH, > Mike > > > // Test -newey-, -newey2- with gaps and lags > > webuse ibm, clear > // Note that "date" is a Stata date variable with gaps (business daily) > // Note that "t" is a linear time trend (sequential) series imposed on the > sample > > tsset date > newey ibm L(1/2).ibm L(0/2).spx, lag(2) > // date is not regularly spaced > newey2 ibm L(1/2).ibm L(0/2).spx, lag(2) > // date is not regularly spaced -- use the force option to override > newey2 ibm L(1/2).ibm L(0/2).spx, lag(2) force > // not sorted > > tsset t > newey ibm L(1/2).ibm L(0/2).spx, lag(2) > // Works! > newey2 ibm L(1/2).ibm L(0/2).spx, lag(2) > // not sorted > newey2 ibm L(1/2).ibm L(0/2).spx, lag(2) force > // not sorted > > > foreach var of varlist ibm spx { > forval l = 0/2 { > g L`l'_`var' = L`l'.`var' > } > } > drop L0_ibm > > tsset date > newey ibm L?_ibm L?_spx, lag(2) > // date is not regularly spaced > newey2 ibm L?_ibm L?_spx, lag(2) > // date is not regularly spaced -- use the force option to override > newey2 ibm L?_ibm L?_spx, lag(2) force > // Works! > > tsset t > newey ibm L?_ibm L?_spx, lag(2) > // Works! > newey2 ibm L?_ibm L?_spx, lag(2) > // Works! > newey2 ibm L?_ibm L?_spx, lag(2) force > // Works! > > // end teset of -newey-, -newey2- > > > On Oct 9, 2008, at 7:09 PM, Thomas Jacobs wrote: > >> Fellow listers, I am trying to understand the cause of intermittent errors >> >> "[time series variable] is not regularly spaced" and >> >> "not sorted" >> >> when trying to run newey and newey2 on panel data with approximately >> 50 companies and 900 trade days of market variables. >> >> I have ensured that my time variable is consecutive for all firms: >> >> . xtset >> panel variable: CompanyNum (unbalanced) >> time variable: TradeDateNe~m, 1 to 902 >> delta: 1 unit >> >> and while not perfectly balanced (one firm begins at >> TradeDateNewNumber 23 as opposed to 1) it is much closer to balanced >> than not. I can force newey to work >> >> . newey LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT >> LnRtTED L(0/2).LnRtStock , >>> >>> lag(4) force >> >> [output excluded placed at bottom of note for readability] >> >> but the same command on newey2 produces not sorted: >> . newey2 LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT >> LnRtTED L(0/2).LnRtStock >>> >>> , lag(4) force >> >> not sorted >> >> I have tried sorting by time variable first as opposed to panel and >> time, the default, with no effect. >> >> Without force I nearly always get: >> >> . newey LnRtStock LnRtSP500, lag(2) >> TradeDateNewNum is not regularly spaced >> >> If I drop the one firm I mentioned to perfectly balance the dataset: >> >> . xtset >> panel variable: CompanyNum (strongly balanced) >> time variable: TradeDateNe~m, 1 to 902 >> delta: 1 unit >> >> and run newey without force: >> >> . newey LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT >> LnRtTED L(0/2).LnRtStock , >>> >>> lag(4) >> >> TradeDateNewNum is not regularly spaced >> >> force works: >> . newey LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT >> LnRtTED L(0/2).LnRtStock , >>> >>> lag(4) force >> >> but I have problems with newey2 either way >> >> . newey2 LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT >> LnRtTED L(0/2).LnRtStock >>> >>> , lag(4) >> >> TradeDateNewNum is not regularly spaced -- use the force option to >> override >> r(198); >> >> . newey2 LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT >> LnRtTED L(0/2).LnRtStock >>> >>> , lag(4) force >> >> not sorted >> r(5); >> >> The only thing I can think of is that one of my independent variables >> in the regressions, LnRTCDX, is missing about 15 values over the range >> of the time series. >> >> This question was posted four years ago without a satisfactory reply >> save for read the FAQ. >> Can anyone help me to continue troubleshooting or to understand the >> cause of these errors? >> >> Output from original successful implementation of newey: >> >> Regression with Newey-West standard errors Number of obs = >> 46545 >> maximum lag: 4 F( 12, 46532) = >> 323.37 >> Prob > F = >> 0.0000 >> >> >> ------------------------------------------------------------------------------ >> | Newey-West >> LnRtFiveYr~d | Coef. Std. Err. t P>|t| [95% Conf. >> Interval] >> >> -------------+---------------------------------------------------------------- >> LnRtFiveYr~d | >> L1. | -.1238135 .0154237 -8.03 0.000 -.1540441 >> -.0935829 >> L2. | -.043356 .013119 -3.30 0.001 -.0690694 >> -.0176426 >> LnRtCDX | >> --. | .4423975 .01975 22.40 0.000 .4036871 >> .4811079 >> L1. | .3530493 .0143778 24.56 0.000 .3248685 >> .38123 >> L2. | .0776956 .0143038 5.43 0.000 .0496599 >> .1057312 >> L3. | .0978518 .0105124 9.31 0.000 .0772474 >> .1184562 >> L4. | .0906659 .0097509 9.30 0.000 .0715539 >> .1097778 >> LnRtCMT | -.2242314 .0194782 -11.51 0.000 -.262409 >> -.1860538 >> LnRtTED | -.0140888 .0032 -4.40 0.000 -.0203609 >> -.0078166 >> LnRtStock | >> --. | -.1728564 .0566356 -3.05 0.002 -.2838631 >> -.0618497 >> L1. | -.1885385 .0431331 -4.37 0.000 -.27308 >> -.1039969 >> L2. | -.0669546 .0234642 -2.85 0.004 -.1129447 >> -.0209644 >> _cons | .0008108 .0002241 3.62 0.000 .0003716 >> .0012499 >> >> ------------------------------------------------------------------------------ >> >> -- >> Thomas Jacobs >> * >> * For searches and help try: >> * http://www.stata.com/help.cgi?search >> * http://www.stata.com/support/statalist/faq >> * http://www.ats.ucla.edu/stat/stata/ > > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > -- Thomas Jacobs * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**References**:**st: Troubleshooting 'not sorted' and 'not regularly spaced' errors in newey and newey2***From:*"Thomas Jacobs" <thomasjacobs@gmail.com>

**Re: st: Troubleshooting 'not sorted' and 'not regularly spaced' errors in newey and newey2***From:*Michael Hanson <mshanson@mac.com>

- Prev by Date:
**Re: st: RE: Dropping observation after it has 'failed'** - Next by Date:
**Re: st: re: Troubleshooting 'not sorted' and 'not regularly spaced' errors** - Previous by thread:
**Re: st: Troubleshooting 'not sorted' and 'not regularly spaced' errors in newey and newey2** - Next by thread:
**st: loop to rename vars** - Index(es):

© Copyright 1996–2015 StataCorp LP | Terms of use | Privacy | Contact us | What's new | Site index |