Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: st: RE: e(sample) after xtivreg2


From   "Schaffer, Mark E" <M.E.Schaffer@hw.ac.uk>
To   <statalist@hsphsun2.harvard.edu>
Subject   RE: st: RE: e(sample) after xtivreg2
Date   Tue, 12 Apr 2011 14:45:19 +0100

Chris,

> -----Original Message-----
> From: christopher.d.parker@gmail.com 
> [mailto:christopher.d.parker@gmail.com] On Behalf Of Chris Parker
> Sent: Tuesday, April 12, 2011 2:27 PM
> To: statalist@hsphsun2.harvard.edu
> Cc: Schaffer, Mark E
> Subject: Re: st: RE: e(sample) after xtivreg2
> 
> Mark,
> 
> Thanks for the clarification. Yes, I agree that singletons cannot be
> used because the FE wipes them out. I guess I'm just a little confused
> by xtivreg2 including singletons in e(sample) when they are not used
> in the regression itself as evidenced by the decrease in number of obs
> in the output.

I think my reasoning at the time was that I should mimic what happens with, say, a probit estimation when you get the message "xx failures and yy successes completely determined".  These observations are kept in e(sample) even though, in effect, they aren't "used".

--Mark

> 
> Chris
> 
> On Tue, Apr 12, 2011 at 2:06 PM, Schaffer, Mark E 
> <M.E.Schaffer@hw.ac.uk> wrote:
> > Chris,
> >
> > To answer your specific question below:
> >
> >> Can someone explain what is going on here? Is xtivreg2 
> actually using
> >> the 172 observations or not?
> >
> > -xtivreg2- includes the 172 singletons in e(sample).  But 
> of course the
> > FE (or FD) transformation effectively wipes these out, 
> because there's
> > no within-group variation when there's only one observation in the
> > group.  -xtivreg2- warns about this.
> >
> > -xtreg- also does not - cannot! - use singletons to 
> estimate the slope
> > coefficients with the FE estimator.  However, -xtreg- 
> reports a constant
> > term, and if I'm not mistaken, it *does* use the singletons for
> > estimating that.
> >
> > I hope this clarifies things.
> >
> > Cheers,
> > Mark
> >
> >> -----Original Message-----
> >> From: owner-statalist@hsphsun2.harvard.edu
> >> [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of
> >> Chris Parker
> >> Sent: Tuesday, April 12, 2011 12:52 PM
> >> To: statalist@hsphsun2.harvard.edu
> >> Subject: st: e(sample) after xtivreg2
> >>
> >> Hello statalist,
> >>
> >> I am using xtivreg2 from ssc to perform two-way error clustering.
> >> However I would also like to look at the iid errors as well. I've
> >> noticed that xtreg is faster that xtivreg2 so I wanted to calculate
> >> the iid errors using xtreg and then use xtivreg2 only to 
> calculate the
> >> two-way clustered errors.
> >>
> >> . xtreg highprice volume, fe
> >>
> >> Fixed-effects (within) regression               Number of obs
> >>      =   1845327
> >> Group variable: cropstat~tid                    Number of
> >> groups   =      8825
> >>
> >> R-sq:  within  = 0.0005                         Obs per
> >> group: min =         1
> >>        between = 0.0006
> >>  avg =     209.1
> >>        overall = 0.0000
> >>  max =       873
> >>
> >>                                                 F(1,1836501)
> >>      =    852.88
> >> corr(u_i, Xb)  = -0.0140                        Prob > F
> >>      =    0.0000
> >>
> >> --------------------------------------------------------------
> >> ----------------
> >>    highprice |      Coef.   Std. Err.      t    P>|t|
> >> [95% Conf. Interval]
> >> -------------+------------------------------------------------
> >> ----------------
> >>       volume |  -.0037862   .0001296   -29.20   0.000
> >> -.0040403   -.0035321
> >>        _cons |   1595.633   .4527128  3524.60   0.000
> >> 1594.745     1596.52
> >> -------------+------------------------------------------------
> >> ----------------
> >>      sigma_u |  3092.7207
> >>      sigma_e |   602.9378
> >>          rho |  .96338464   (fraction of variance due to u_i)
> >> --------------------------------------------------------------
> >> ----------------
> >> F test that all u_i=0:     F(8824, 1836501) =  1758.26
> >> Prob > F = 0.0000
> >> r; t=28.67 12:32:57
> >>
> >> Which, on my four core license takes 28.67 seconds and 
> uses 1,845,327
> >> observations. Now when I run the same regression using 
> xtivreg2 I am
> >> issued a warning that singleton groups are detected and 172
> >> observations will not be used:
> >>
> >>
> >> . xtivreg2 highprice volume, fe
> >> Warning - singleton groups detected.  172 observation(s) not used.
> >>
> >> FIXED EFFECTS ESTIMATION
> >> ------------------------
> >> Number of groups =      8653                    Obs per
> >> group: min =         2
> >>
> >>  avg =     213.2
> >>
> >>  max =       873
> >>
> >> OLS estimation
> >> --------------
> >>
> >> Estimates efficient for homoskedasticity only
> >> Statistics consistent for homoskedasticity only
> >>
> >>                                                       Number
> >> of obs =  1845155
> >>                                                       F(
> >> 1,1836501) =   852.88
> >>                                                       Prob >
> >> F      =   0.0000
> >> Total (centered) SS     =  6.67941e+11
> >> Centered R2   =   0.0005
> >> Total (uncentered) SS   =  6.67941e+11
> >> Uncentered R2 =   0.0005
> >> Residual SS             =  6.67631e+11                Root
> >> MSE      =    602.9
> >>
> >> --------------------------------------------------------------
> >> ----------------
> >>    highprice |      Coef.   Std. Err.      z    P>|z|
> >> [95% Conf. Interval]
> >> -------------+------------------------------------------------
> >> ----------------
> >>       volume |  -.0037862   .0001296   -29.20   0.000
> >> -.0040403   -.0035321
> >> --------------------------------------------------------------
> >> ----------------
> >> Included instruments: volume
> >> --------------------------------------------------------------
> >> ----------------
> >> r; t=50.46 12:34:19
> >>
> >> Just a note that this does in fact take longer, 50.46 
> seconds. Also,
> >> the 172 observations are not included in the regression as noted by
> >> number of obs. I thought it would be best to look at the xtreg
> >> regression without the 172 observations. So I attempted to run the
> >> xtreg regression with the sample from xtivreg2 only to 
> find that all
> >> of the observations are included in the sample from xtivreg2.
> >>
> >> . gen mysample = e(sample)
> >> r; t=0.20 12:35:04
> >>
> >> . tab mysample
> >>
> >>    mysample |      Freq.     Percent        Cum.
> >> ------------+-----------------------------------
> >>           1 |  1,845,327      100.00      100.00
> >> ------------+-----------------------------------
> >>       Total |  1,845,327      100.00
> >> r; t=0.56 12:35:07
> >>
> >> Can someone explain what is going on here? Is xtivreg2 
> actually using
> >> the 172 observations or not?
> >>
> >> My plan is to drop the singleton groups. However, I am running
> >> regressions with different groups of independent variables with
> >> missing observations. This could potentially create 
> singleton groups
> >> again. Is there a way to make xtreg ignore the singleton groups
> >> automatically as well?
> >>
> >> Additionally, does anyone know of a command that will let 
> me calculate
> >> the two-way clustered errors after xtreg? I don't really 
> see the point
> >> in running the same regression again just to cluster the errors
> >> differently but short of coding the two-way clustered 
> errors myself I
> >> don't know how else to do this.
> >>
> >> Chris
> >>
> >>
> >> Chris Parker
> >>
> >> ________________________________
> >>
> >> PhD Candidate | Management Science & Operations
> >> London Business School | Regent's Park | London NW1 4SA |
> >> United Kingdom
> >> Direct line +44 (0)20 7000 8816 | Email cparker.phd2007@london.edu
> >> *
> >> *   For searches and help try:
> >> *   http://www.stata.com/help.cgi?search
> >> *   http://www.stata.com/support/statalist/faq
> >> *   http://www.ats.ucla.edu/stat/stata/
> >>
> >
> >
> > --
> > Heriot-Watt University is a Scottish charity
> > registered under charity number SC000278.
> >
> >
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/help.cgi?search
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
> >
> > 
> ______________________________________________________________________
> >
> > This email has been scanned by the MessageLabs Email Security System
> > on behalf of the London Business School community.
> > For more information please visit http://www.messagelabs.com/email
> > 
> ______________________________________________________________________
> >
> 


-- 
Heriot-Watt University is a Scottish charity
registered under charity number SC000278.


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index