Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Troubleshooting 'not sorted' and 'not regularly spaced' errors in newey and newey2


From   Michael Hanson <mshanson@mac.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: Troubleshooting 'not sorted' and 'not regularly spaced' errors in newey and newey2
Date   Sat, 11 Oct 2008 12:14:47 -0400

I have seen this problem before.  The bottom line seems to be:

1. -newey2- does not recognize time-series operators in Stata. (That may be because -newey2- was last updated in early 2004.) The "not sorted" error is what you seen instead. To get around this problem, first create each lagged variable as its own series before estimating your regressions. For example:

gen L1_LnRtFiveYrMid = L1.LnRtFiveYrMid
gen L2_LnRtFiveYrMid = L2.LnRtFiveYrMid
etc.

and then use the appropriate L?_* variables as regressors.

2. -newey- and -newey2- do not work if there are any "gaps" in your data in time. (This should not be surprising, as it is not possible to produce standard errors that are robust to serial correlation without sequential observations in time.) You can get around this problem by either using the -force- option in -newey2-, or by constructing a linear time series for your observations in sequence and -tsset- or -xtset- to that series. You should carefully explore whether the statistics you get from such a procedure are meaningful, however. (I suspect this also means you will need to work with a balanced panel, but I leave that as an exercise.)

Below is a sample program, using a dataset made generally available through StataCorp, that illustrates these points. Note that if you try to run this program in its entirety, it will halt at the first line that produces an error -- and there are several errors by the nature of this demonstration program. I've noted with a comment after each estimation command whether it "Works!" or, if it fails, what error message it produces.

HTH,
Mike


// Test -newey-, -newey2- with gaps and lags

webuse ibm, clear
// Note that "date" is a Stata date variable with gaps (business daily)
// Note that "t" is a linear time trend (sequential) series imposed on the sample

tsset date
newey ibm L(1/2).ibm L(0/2).spx, lag(2)			
// date is not regularly spaced
newey2 ibm L(1/2).ibm L(0/2).spx, lag(2)		
// date is not regularly spaced -- use the force option to override
newey2 ibm L(1/2).ibm L(0/2).spx, lag(2) force
// not sorted

tsset t
newey ibm L(1/2).ibm L(0/2).spx, lag(2)
// Works!
newey2 ibm L(1/2).ibm L(0/2).spx, lag(2)
// not sorted
newey2 ibm L(1/2).ibm L(0/2).spx, lag(2) force
// not sorted


foreach var of varlist ibm spx {
	forval l = 0/2 {
		g L`l'_`var' = L`l'.`var'
	}
}
drop L0_ibm

tsset date
newey ibm L?_ibm L?_spx, lag(2)
// date is not regularly spaced
newey2 ibm L?_ibm L?_spx, lag(2)
// date is not regularly spaced -- use the force option to override
newey2 ibm L?_ibm L?_spx, lag(2) force
// Works!

tsset t
newey ibm L?_ibm L?_spx, lag(2)
// Works!
newey2 ibm L?_ibm L?_spx, lag(2)
// Works!
newey2 ibm L?_ibm L?_spx, lag(2) force
// Works!

// end teset of -newey-, -newey2-


On Oct 9, 2008, at 7:09 PM, Thomas Jacobs wrote:

Fellow listers, I am trying to understand the cause of intermittent errors

"[time series variable] is not regularly spaced" and

"not sorted"

when trying to run newey and newey2 on panel data with approximately
50 companies and 900 trade days of market variables.

I have ensured that my time variable is consecutive for all firms:

. xtset
       panel variable:  CompanyNum (unbalanced)
        time variable:  TradeDateNe~m, 1 to 902
                delta:  1 unit

and while not perfectly balanced (one firm begins at
TradeDateNewNumber 23 as opposed to 1) it is much closer to balanced
than not.  I can force newey to work

. newey LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT
LnRtTED L(0/2).LnRtStock ,
 lag(4) force

[output excluded placed at bottom of note for readability]

but the same command on newey2 produces not sorted:
. newey2 LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT
LnRtTED L(0/2).LnRtStock
, lag(4) force

not sorted

I have tried sorting by time variable first as opposed to panel and
time, the default, with no effect.

Without force I nearly always get:

. newey LnRtStock LnRtSP500, lag(2)
TradeDateNewNum is not regularly spaced

If I drop the one firm I mentioned to perfectly balance the dataset:

. xtset
       panel variable:  CompanyNum (strongly balanced)
        time variable:  TradeDateNe~m, 1 to 902
                delta:  1 unit

and run newey without force:

. newey LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT
LnRtTED L(0/2).LnRtStock ,
 lag(4)
TradeDateNewNum is not regularly spaced

force works:
. newey LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT
LnRtTED L(0/2).LnRtStock ,
 lag(4) force

but I have problems with newey2 either way

. newey2 LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT
LnRtTED L(0/2).LnRtStock
, lag(4)
TradeDateNewNum is not regularly spaced -- use the force option to override
r(198);

. newey2 LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT
LnRtTED L(0/2).LnRtStock
, lag(4) force

not sorted
r(5);

The only thing I can think of is that one of my independent variables
in the regressions, LnRTCDX, is missing about 15 values over the range
of the time series.

This question was posted four years ago without a satisfactory reply
save for read the FAQ.
Can anyone help me to continue troubleshooting or to understand the
cause of these errors?

Output from original successful implementation of newey:

Regression with Newey-West standard errors Number of obs = 46545 maximum lag: 4 F( 12, 46532) = 323.37 Prob > F = 0.0000

---------------------------------------------------------------------- --------
             |             Newey-West
LnRtFiveYr~d | Coef. Std. Err. t P>|t| [95% Conf. Interval] ------------- +----------------------------------------------------------------
LnRtFiveYr~d |
L1. | -.1238135 .0154237 -8.03 0.000 -. 1540441 -.0935829 L2. | -.043356 .013119 -3.30 0.001 -. 0690694 -.0176426
     LnRtCDX |
--. | .4423975 .01975 22.40 0.000 . 4036871 .4811079 L1. | .3530493 .0143778 24.56 0.000 . 3248685 .38123 L2. | .0776956 .0143038 5.43 0.000 . 0496599 .1057312 L3. | .0978518 .0105124 9.31 0.000 . 0772474 .1184562 L4. | .0906659 .0097509 9.30 0.000 . 0715539 .1097778 LnRtCMT | -.2242314 .0194782 -11.51 0.000 -. 262409 -.1860538 LnRtTED | -.0140888 .0032 -4.40 0.000 -. 0203609 -.0078166
   LnRtStock |
--. | -.1728564 .0566356 -3.05 0.002 -. 2838631 -.0618497 L1. | -.1885385 .0431331 -4.37 0.000 -. 27308 -.1039969 L2. | -.0669546 .0234642 -2.85 0.004 -. 1129447 -.0209644 _cons | .0008108 .0002241 3.62 0.000 . 0003716 .0012499 ---------------------------------------------------------------------- --------

--
Thomas Jacobs
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index