Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: RE: e(sample) after xtivreg2


From   Chris Parker <cparker.phd2007@london.edu>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: RE: e(sample) after xtivreg2
Date   Tue, 12 Apr 2011 14:26:39 +0100

Mark,

Thanks for the clarification. Yes, I agree that singletons cannot be
used because the FE wipes them out. I guess I'm just a little confused
by xtivreg2 including singletons in e(sample) when they are not used
in the regression itself as evidenced by the decrease in number of obs
in the output.

Chris

On Tue, Apr 12, 2011 at 2:06 PM, Schaffer, Mark E <M.E.Schaffer@hw.ac.uk> wrote:
> Chris,
>
> To answer your specific question below:
>
>> Can someone explain what is going on here? Is xtivreg2 actually using
>> the 172 observations or not?
>
> -xtivreg2- includes the 172 singletons in e(sample).  But of course the
> FE (or FD) transformation effectively wipes these out, because there's
> no within-group variation when there's only one observation in the
> group.  -xtivreg2- warns about this.
>
> -xtreg- also does not - cannot! - use singletons to estimate the slope
> coefficients with the FE estimator.  However, -xtreg- reports a constant
> term, and if I'm not mistaken, it *does* use the singletons for
> estimating that.
>
> I hope this clarifies things.
>
> Cheers,
> Mark
>
>> -----Original Message-----
>> From: owner-statalist@hsphsun2.harvard.edu
>> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of
>> Chris Parker
>> Sent: Tuesday, April 12, 2011 12:52 PM
>> To: statalist@hsphsun2.harvard.edu
>> Subject: st: e(sample) after xtivreg2
>>
>> Hello statalist,
>>
>> I am using xtivreg2 from ssc to perform two-way error clustering.
>> However I would also like to look at the iid errors as well. I've
>> noticed that xtreg is faster that xtivreg2 so I wanted to calculate
>> the iid errors using xtreg and then use xtivreg2 only to calculate the
>> two-way clustered errors.
>>
>> . xtreg highprice volume, fe
>>
>> Fixed-effects (within) regression               Number of obs
>>      =   1845327
>> Group variable: cropstat~tid                    Number of
>> groups   =      8825
>>
>> R-sq:  within  = 0.0005                         Obs per
>> group: min =         1
>>        between = 0.0006
>>  avg =     209.1
>>        overall = 0.0000
>>  max =       873
>>
>>                                                 F(1,1836501)
>>      =    852.88
>> corr(u_i, Xb)  = -0.0140                        Prob > F
>>      =    0.0000
>>
>> --------------------------------------------------------------
>> ----------------
>>    highprice |      Coef.   Std. Err.      t    P>|t|
>> [95% Conf. Interval]
>> -------------+------------------------------------------------
>> ----------------
>>       volume |  -.0037862   .0001296   -29.20   0.000
>> -.0040403   -.0035321
>>        _cons |   1595.633   .4527128  3524.60   0.000
>> 1594.745     1596.52
>> -------------+------------------------------------------------
>> ----------------
>>      sigma_u |  3092.7207
>>      sigma_e |   602.9378
>>          rho |  .96338464   (fraction of variance due to u_i)
>> --------------------------------------------------------------
>> ----------------
>> F test that all u_i=0:     F(8824, 1836501) =  1758.26
>> Prob > F = 0.0000
>> r; t=28.67 12:32:57
>>
>> Which, on my four core license takes 28.67 seconds and uses 1,845,327
>> observations. Now when I run the same regression using xtivreg2 I am
>> issued a warning that singleton groups are detected and 172
>> observations will not be used:
>>
>>
>> . xtivreg2 highprice volume, fe
>> Warning - singleton groups detected.  172 observation(s) not used.
>>
>> FIXED EFFECTS ESTIMATION
>> ------------------------
>> Number of groups =      8653                    Obs per
>> group: min =         2
>>
>>  avg =     213.2
>>
>>  max =       873
>>
>> OLS estimation
>> --------------
>>
>> Estimates efficient for homoskedasticity only
>> Statistics consistent for homoskedasticity only
>>
>>                                                       Number
>> of obs =  1845155
>>                                                       F(
>> 1,1836501) =   852.88
>>                                                       Prob >
>> F      =   0.0000
>> Total (centered) SS     =  6.67941e+11
>> Centered R2   =   0.0005
>> Total (uncentered) SS   =  6.67941e+11
>> Uncentered R2 =   0.0005
>> Residual SS             =  6.67631e+11                Root
>> MSE      =    602.9
>>
>> --------------------------------------------------------------
>> ----------------
>>    highprice |      Coef.   Std. Err.      z    P>|z|
>> [95% Conf. Interval]
>> -------------+------------------------------------------------
>> ----------------
>>       volume |  -.0037862   .0001296   -29.20   0.000
>> -.0040403   -.0035321
>> --------------------------------------------------------------
>> ----------------
>> Included instruments: volume
>> --------------------------------------------------------------
>> ----------------
>> r; t=50.46 12:34:19
>>
>> Just a note that this does in fact take longer, 50.46 seconds. Also,
>> the 172 observations are not included in the regression as noted by
>> number of obs. I thought it would be best to look at the xtreg
>> regression without the 172 observations. So I attempted to run the
>> xtreg regression with the sample from xtivreg2 only to find that all
>> of the observations are included in the sample from xtivreg2.
>>
>> . gen mysample = e(sample)
>> r; t=0.20 12:35:04
>>
>> . tab mysample
>>
>>    mysample |      Freq.     Percent        Cum.
>> ------------+-----------------------------------
>>           1 |  1,845,327      100.00      100.00
>> ------------+-----------------------------------
>>       Total |  1,845,327      100.00
>> r; t=0.56 12:35:07
>>
>> Can someone explain what is going on here? Is xtivreg2 actually using
>> the 172 observations or not?
>>
>> My plan is to drop the singleton groups. However, I am running
>> regressions with different groups of independent variables with
>> missing observations. This could potentially create singleton groups
>> again. Is there a way to make xtreg ignore the singleton groups
>> automatically as well?
>>
>> Additionally, does anyone know of a command that will let me calculate
>> the two-way clustered errors after xtreg? I don't really see the point
>> in running the same regression again just to cluster the errors
>> differently but short of coding the two-way clustered errors myself I
>> don't know how else to do this.
>>
>> Chris
>>
>>
>> Chris Parker
>>
>> ________________________________
>>
>> PhD Candidate | Management Science & Operations
>> London Business School | Regent's Park | London NW1 4SA |
>> United Kingdom
>> Direct line +44 (0)20 7000 8816 | Email cparker.phd2007@london.edu
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>
>
> --
> Heriot-Watt University is a Scottish charity
> registered under charity number SC000278.
>
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
> ______________________________________________________________________
>
> This email has been scanned by the MessageLabs Email Security System
> on behalf of the London Business School community.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index