Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Troubleshooting 'not sorted' and 'not regularly spaced' errors in newey and newey2


From   "Thomas Jacobs" <thomasjacobs@gmail.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: Troubleshooting 'not sorted' and 'not regularly spaced' errors in newey and newey2
Date   Mon, 13 Oct 2008 11:16:43 -0500

Mike,

I can't thank you enough for the newey "clinic" you provided.  I
confirmed all of your points with both your sample data as well as my
own.  That was a big help.

Tom

On Sat, Oct 11, 2008 at 11:14 AM, Michael Hanson <mshanson@mac.com> wrote:
> I have seen this problem before.  The bottom line seems to be:
>
> 1. -newey2- does not recognize time-series operators in Stata.  (That may be
> because -newey2- was last updated in early 2004.)  The "not sorted" error is
> what you seen instead.  To get around this problem, first create each lagged
> variable as its own series before estimating your regressions.  For example:
>
> gen L1_LnRtFiveYrMid = L1.LnRtFiveYrMid
> gen L2_LnRtFiveYrMid = L2.LnRtFiveYrMid
> etc.
>
> and then use the appropriate L?_* variables as regressors.
>
> 2. -newey- and -newey2- do not work if there are any "gaps" in your data in
> time.  (This should not be surprising, as it is not possible to produce
> standard errors that are robust to serial correlation without sequential
> observations in time.)  You can get around this problem by either using the
> -force- option in -newey2-, or by constructing a linear time series for your
> observations in sequence and -tsset- or -xtset- to that series.  You should
> carefully explore whether the statistics you get from such a procedure are
> meaningful, however.  (I suspect this also means you will need to work with
> a balanced panel, but I leave that as an exercise.)
>
> Below is a sample program, using a dataset made generally available through
> StataCorp, that illustrates these points.  Note that if you try to run this
> program in its entirety, it will halt at the first line that produces an
> error -- and there are several errors by the nature of this demonstration
> program.  I've noted with a comment after each estimation command whether it
> "Works!" or, if it fails, what error message it produces.
>
> HTH,
> Mike
>
>
> // Test -newey-, -newey2- with gaps and lags
>
> webuse ibm, clear
> // Note that "date" is a Stata date variable with gaps (business daily)
> // Note that "t" is a linear time trend (sequential) series imposed on the
> sample
>
> tsset date
> newey ibm L(1/2).ibm L(0/2).spx, lag(2)
> // date is not regularly spaced
> newey2 ibm L(1/2).ibm L(0/2).spx, lag(2)
> // date is not regularly spaced -- use the force option to override
> newey2 ibm L(1/2).ibm L(0/2).spx, lag(2) force
> // not sorted
>
> tsset t
> newey ibm L(1/2).ibm L(0/2).spx, lag(2)
> // Works!
> newey2 ibm L(1/2).ibm L(0/2).spx, lag(2)
> // not sorted
> newey2 ibm L(1/2).ibm L(0/2).spx, lag(2) force
> // not sorted
>
>
> foreach var of varlist ibm spx {
>        forval l = 0/2 {
>                g L`l'_`var' = L`l'.`var'
>        }
> }
> drop L0_ibm
>
> tsset date
> newey ibm L?_ibm L?_spx, lag(2)
> // date is not regularly spaced
> newey2 ibm L?_ibm L?_spx, lag(2)
> // date is not regularly spaced -- use the force option to override
> newey2 ibm L?_ibm L?_spx, lag(2) force
> // Works!
>
> tsset t
> newey ibm L?_ibm L?_spx, lag(2)
> // Works!
> newey2 ibm L?_ibm L?_spx, lag(2)
> // Works!
> newey2 ibm L?_ibm L?_spx, lag(2) force
> // Works!
>
> // end teset of -newey-, -newey2-
>
>
> On Oct 9, 2008, at 7:09 PM, Thomas Jacobs wrote:
>
>> Fellow listers, I am trying to understand the cause of intermittent errors
>>
>> "[time series variable] is not regularly spaced" and
>>
>> "not sorted"
>>
>> when trying to run newey and newey2 on panel data with approximately
>> 50 companies and 900 trade days of market variables.
>>
>> I have ensured that my time variable is consecutive for all firms:
>>
>> . xtset
>>       panel variable:  CompanyNum (unbalanced)
>>        time variable:  TradeDateNe~m, 1 to 902
>>                delta:  1 unit
>>
>> and while not perfectly balanced (one firm begins at
>> TradeDateNewNumber 23 as opposed to 1) it is much closer to balanced
>> than not.  I can force newey to work
>>
>> . newey LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT
>> LnRtTED L(0/2).LnRtStock ,
>>>
>>>  lag(4) force
>>
>> [output excluded placed at bottom of note for readability]
>>
>> but the same command on newey2 produces not sorted:
>> . newey2 LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT
>> LnRtTED L(0/2).LnRtStock
>>>
>>> , lag(4) force
>>
>> not sorted
>>
>> I have tried sorting by time variable first as opposed to panel and
>> time, the default, with no effect.
>>
>> Without force I nearly always get:
>>
>> . newey LnRtStock LnRtSP500, lag(2)
>> TradeDateNewNum is not regularly spaced
>>
>> If I drop the one firm I mentioned to perfectly balance the dataset:
>>
>> . xtset
>>       panel variable:  CompanyNum (strongly balanced)
>>        time variable:  TradeDateNe~m, 1 to 902
>>                delta:  1 unit
>>
>> and run newey without force:
>>
>> . newey LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT
>> LnRtTED L(0/2).LnRtStock ,
>>>
>>>  lag(4)
>>
>> TradeDateNewNum is not regularly spaced
>>
>> force works:
>> . newey LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT
>> LnRtTED L(0/2).LnRtStock ,
>>>
>>>  lag(4) force
>>
>> but I have problems with newey2 either way
>>
>> . newey2 LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT
>> LnRtTED L(0/2).LnRtStock
>>>
>>> , lag(4)
>>
>> TradeDateNewNum is not regularly spaced -- use the force option to
>> override
>> r(198);
>>
>> . newey2 LnRtFiveYrMid L(1/2).LnRtFiveYrMid L(0/4).LnRtCDX LnRtCMT
>> LnRtTED L(0/2).LnRtStock
>>>
>>> , lag(4) force
>>
>> not sorted
>> r(5);
>>
>> The only thing I can think of is that one of my independent variables
>> in the regressions, LnRTCDX, is missing about 15 values over the range
>> of the time series.
>>
>> This question was posted four years ago without a satisfactory reply
>> save for read the FAQ.
>> Can anyone help me to continue troubleshooting or to understand the
>> cause of these errors?
>>
>> Output from original successful implementation of newey:
>>
>> Regression with Newey-West standard errors          Number of obs  =
>> 46545
>> maximum lag: 4                                      F( 12, 46532)  =
>>  323.37
>>                                                    Prob > F       =
>>  0.0000
>>
>>
>> ------------------------------------------------------------------------------
>>             |             Newey-West
>> LnRtFiveYr~d |      Coef.   Std. Err.      t    P>|t|     [95% Conf.
>> Interval]
>>
>> -------------+----------------------------------------------------------------
>> LnRtFiveYr~d |
>>         L1. |  -.1238135   .0154237    -8.03   0.000    -.1540441
>> -.0935829
>>         L2. |   -.043356    .013119    -3.30   0.001    -.0690694
>> -.0176426
>>     LnRtCDX |
>>         --. |   .4423975     .01975    22.40   0.000     .4036871
>>  .4811079
>>         L1. |   .3530493   .0143778    24.56   0.000     .3248685
>>  .38123
>>         L2. |   .0776956   .0143038     5.43   0.000     .0496599
>>  .1057312
>>         L3. |   .0978518   .0105124     9.31   0.000     .0772474
>>  .1184562
>>         L4. |   .0906659   .0097509     9.30   0.000     .0715539
>>  .1097778
>>     LnRtCMT |  -.2242314   .0194782   -11.51   0.000     -.262409
>> -.1860538
>>     LnRtTED |  -.0140888      .0032    -4.40   0.000    -.0203609
>> -.0078166
>>   LnRtStock |
>>         --. |  -.1728564   .0566356    -3.05   0.002    -.2838631
>> -.0618497
>>         L1. |  -.1885385   .0431331    -4.37   0.000      -.27308
>> -.1039969
>>         L2. |  -.0669546   .0234642    -2.85   0.004    -.1129447
>> -.0209644
>>       _cons |   .0008108   .0002241     3.62   0.000     .0003716
>>  .0012499
>>
>> ------------------------------------------------------------------------------
>>
>> --
>> Thomas Jacobs
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>



-- 
Thomas Jacobs
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index