Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: st: RE: e(sample) after xtivreg2
From
"Schaffer, Mark E" <[email protected]>
To
<[email protected]>
Subject
RE: st: RE: e(sample) after xtivreg2
Date
Tue, 12 Apr 2011 14:45:19 +0100
Chris,
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Chris Parker
> Sent: Tuesday, April 12, 2011 2:27 PM
> To: [email protected]
> Cc: Schaffer, Mark E
> Subject: Re: st: RE: e(sample) after xtivreg2
>
> Mark,
>
> Thanks for the clarification. Yes, I agree that singletons cannot be
> used because the FE wipes them out. I guess I'm just a little confused
> by xtivreg2 including singletons in e(sample) when they are not used
> in the regression itself as evidenced by the decrease in number of obs
> in the output.
I think my reasoning at the time was that I should mimic what happens with, say, a probit estimation when you get the message "xx failures and yy successes completely determined". These observations are kept in e(sample) even though, in effect, they aren't "used".
--Mark
>
> Chris
>
> On Tue, Apr 12, 2011 at 2:06 PM, Schaffer, Mark E
> <[email protected]> wrote:
> > Chris,
> >
> > To answer your specific question below:
> >
> >> Can someone explain what is going on here? Is xtivreg2
> actually using
> >> the 172 observations or not?
> >
> > -xtivreg2- includes the 172 singletons in e(sample). But
> of course the
> > FE (or FD) transformation effectively wipes these out,
> because there's
> > no within-group variation when there's only one observation in the
> > group. -xtivreg2- warns about this.
> >
> > -xtreg- also does not - cannot! - use singletons to
> estimate the slope
> > coefficients with the FE estimator. However, -xtreg-
> reports a constant
> > term, and if I'm not mistaken, it *does* use the singletons for
> > estimating that.
> >
> > I hope this clarifies things.
> >
> > Cheers,
> > Mark
> >
> >> -----Original Message-----
> >> From: [email protected]
> >> [mailto:[email protected]] On Behalf Of
> >> Chris Parker
> >> Sent: Tuesday, April 12, 2011 12:52 PM
> >> To: [email protected]
> >> Subject: st: e(sample) after xtivreg2
> >>
> >> Hello statalist,
> >>
> >> I am using xtivreg2 from ssc to perform two-way error clustering.
> >> However I would also like to look at the iid errors as well. I've
> >> noticed that xtreg is faster that xtivreg2 so I wanted to calculate
> >> the iid errors using xtreg and then use xtivreg2 only to
> calculate the
> >> two-way clustered errors.
> >>
> >> . xtreg highprice volume, fe
> >>
> >> Fixed-effects (within) regression Number of obs
> >> = 1845327
> >> Group variable: cropstat~tid Number of
> >> groups = 8825
> >>
> >> R-sq: within = 0.0005 Obs per
> >> group: min = 1
> >> between = 0.0006
> >> avg = 209.1
> >> overall = 0.0000
> >> max = 873
> >>
> >> F(1,1836501)
> >> = 852.88
> >> corr(u_i, Xb) = -0.0140 Prob > F
> >> = 0.0000
> >>
> >> --------------------------------------------------------------
> >> ----------------
> >> highprice | Coef. Std. Err. t P>|t|
> >> [95% Conf. Interval]
> >> -------------+------------------------------------------------
> >> ----------------
> >> volume | -.0037862 .0001296 -29.20 0.000
> >> -.0040403 -.0035321
> >> _cons | 1595.633 .4527128 3524.60 0.000
> >> 1594.745 1596.52
> >> -------------+------------------------------------------------
> >> ----------------
> >> sigma_u | 3092.7207
> >> sigma_e | 602.9378
> >> rho | .96338464 (fraction of variance due to u_i)
> >> --------------------------------------------------------------
> >> ----------------
> >> F test that all u_i=0: F(8824, 1836501) = 1758.26
> >> Prob > F = 0.0000
> >> r; t=28.67 12:32:57
> >>
> >> Which, on my four core license takes 28.67 seconds and
> uses 1,845,327
> >> observations. Now when I run the same regression using
> xtivreg2 I am
> >> issued a warning that singleton groups are detected and 172
> >> observations will not be used:
> >>
> >>
> >> . xtivreg2 highprice volume, fe
> >> Warning - singleton groups detected. 172 observation(s) not used.
> >>
> >> FIXED EFFECTS ESTIMATION
> >> ------------------------
> >> Number of groups = 8653 Obs per
> >> group: min = 2
> >>
> >> avg = 213.2
> >>
> >> max = 873
> >>
> >> OLS estimation
> >> --------------
> >>
> >> Estimates efficient for homoskedasticity only
> >> Statistics consistent for homoskedasticity only
> >>
> >> Number
> >> of obs = 1845155
> >> F(
> >> 1,1836501) = 852.88
> >> Prob >
> >> F = 0.0000
> >> Total (centered) SS = 6.67941e+11
> >> Centered R2 = 0.0005
> >> Total (uncentered) SS = 6.67941e+11
> >> Uncentered R2 = 0.0005
> >> Residual SS = 6.67631e+11 Root
> >> MSE = 602.9
> >>
> >> --------------------------------------------------------------
> >> ----------------
> >> highprice | Coef. Std. Err. z P>|z|
> >> [95% Conf. Interval]
> >> -------------+------------------------------------------------
> >> ----------------
> >> volume | -.0037862 .0001296 -29.20 0.000
> >> -.0040403 -.0035321
> >> --------------------------------------------------------------
> >> ----------------
> >> Included instruments: volume
> >> --------------------------------------------------------------
> >> ----------------
> >> r; t=50.46 12:34:19
> >>
> >> Just a note that this does in fact take longer, 50.46
> seconds. Also,
> >> the 172 observations are not included in the regression as noted by
> >> number of obs. I thought it would be best to look at the xtreg
> >> regression without the 172 observations. So I attempted to run the
> >> xtreg regression with the sample from xtivreg2 only to
> find that all
> >> of the observations are included in the sample from xtivreg2.
> >>
> >> . gen mysample = e(sample)
> >> r; t=0.20 12:35:04
> >>
> >> . tab mysample
> >>
> >> mysample | Freq. Percent Cum.
> >> ------------+-----------------------------------
> >> 1 | 1,845,327 100.00 100.00
> >> ------------+-----------------------------------
> >> Total | 1,845,327 100.00
> >> r; t=0.56 12:35:07
> >>
> >> Can someone explain what is going on here? Is xtivreg2
> actually using
> >> the 172 observations or not?
> >>
> >> My plan is to drop the singleton groups. However, I am running
> >> regressions with different groups of independent variables with
> >> missing observations. This could potentially create
> singleton groups
> >> again. Is there a way to make xtreg ignore the singleton groups
> >> automatically as well?
> >>
> >> Additionally, does anyone know of a command that will let
> me calculate
> >> the two-way clustered errors after xtreg? I don't really
> see the point
> >> in running the same regression again just to cluster the errors
> >> differently but short of coding the two-way clustered
> errors myself I
> >> don't know how else to do this.
> >>
> >> Chris
> >>
> >>
> >> Chris Parker
> >>
> >> ________________________________
> >>
> >> PhD Candidate | Management Science & Operations
> >> London Business School | Regent's Park | London NW1 4SA |
> >> United Kingdom
> >> Direct line +44 (0)20 7000 8816 | Email [email protected]
> >> *
> >> * For searches and help try:
> >> * http://www.stata.com/help.cgi?search
> >> * http://www.stata.com/support/statalist/faq
> >> * http://www.ats.ucla.edu/stat/stata/
> >>
> >
> >
> > --
> > Heriot-Watt University is a Scottish charity
> > registered under charity number SC000278.
> >
> >
> > *
> > * For searches and help try:
> > * http://www.stata.com/help.cgi?search
> > * http://www.stata.com/support/statalist/faq
> > * http://www.ats.ucla.edu/stat/stata/
> >
> >
> ______________________________________________________________________
> >
> > This email has been scanned by the MessageLabs Email Security System
> > on behalf of the London Business School community.
> > For more information please visit http://www.messagelabs.com/email
> >
> ______________________________________________________________________
> >
>
--
Heriot-Watt University is a Scottish charity
registered under charity number SC000278.
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/