Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: RE: e(sample) after xtivreg2
From
Chris Parker <[email protected]>
To
[email protected]
Subject
Re: st: RE: e(sample) after xtivreg2
Date
Tue, 12 Apr 2011 14:26:39 +0100
Mark,
Thanks for the clarification. Yes, I agree that singletons cannot be
used because the FE wipes them out. I guess I'm just a little confused
by xtivreg2 including singletons in e(sample) when they are not used
in the regression itself as evidenced by the decrease in number of obs
in the output.
Chris
On Tue, Apr 12, 2011 at 2:06 PM, Schaffer, Mark E <[email protected]> wrote:
> Chris,
>
> To answer your specific question below:
>
>> Can someone explain what is going on here? Is xtivreg2 actually using
>> the 172 observations or not?
>
> -xtivreg2- includes the 172 singletons in e(sample). But of course the
> FE (or FD) transformation effectively wipes these out, because there's
> no within-group variation when there's only one observation in the
> group. -xtivreg2- warns about this.
>
> -xtreg- also does not - cannot! - use singletons to estimate the slope
> coefficients with the FE estimator. However, -xtreg- reports a constant
> term, and if I'm not mistaken, it *does* use the singletons for
> estimating that.
>
> I hope this clarifies things.
>
> Cheers,
> Mark
>
>> -----Original Message-----
>> From: [email protected]
>> [mailto:[email protected]] On Behalf Of
>> Chris Parker
>> Sent: Tuesday, April 12, 2011 12:52 PM
>> To: [email protected]
>> Subject: st: e(sample) after xtivreg2
>>
>> Hello statalist,
>>
>> I am using xtivreg2 from ssc to perform two-way error clustering.
>> However I would also like to look at the iid errors as well. I've
>> noticed that xtreg is faster that xtivreg2 so I wanted to calculate
>> the iid errors using xtreg and then use xtivreg2 only to calculate the
>> two-way clustered errors.
>>
>> . xtreg highprice volume, fe
>>
>> Fixed-effects (within) regression Number of obs
>> = 1845327
>> Group variable: cropstat~tid Number of
>> groups = 8825
>>
>> R-sq: within = 0.0005 Obs per
>> group: min = 1
>> between = 0.0006
>> avg = 209.1
>> overall = 0.0000
>> max = 873
>>
>> F(1,1836501)
>> = 852.88
>> corr(u_i, Xb) = -0.0140 Prob > F
>> = 0.0000
>>
>> --------------------------------------------------------------
>> ----------------
>> highprice | Coef. Std. Err. t P>|t|
>> [95% Conf. Interval]
>> -------------+------------------------------------------------
>> ----------------
>> volume | -.0037862 .0001296 -29.20 0.000
>> -.0040403 -.0035321
>> _cons | 1595.633 .4527128 3524.60 0.000
>> 1594.745 1596.52
>> -------------+------------------------------------------------
>> ----------------
>> sigma_u | 3092.7207
>> sigma_e | 602.9378
>> rho | .96338464 (fraction of variance due to u_i)
>> --------------------------------------------------------------
>> ----------------
>> F test that all u_i=0: F(8824, 1836501) = 1758.26
>> Prob > F = 0.0000
>> r; t=28.67 12:32:57
>>
>> Which, on my four core license takes 28.67 seconds and uses 1,845,327
>> observations. Now when I run the same regression using xtivreg2 I am
>> issued a warning that singleton groups are detected and 172
>> observations will not be used:
>>
>>
>> . xtivreg2 highprice volume, fe
>> Warning - singleton groups detected. 172 observation(s) not used.
>>
>> FIXED EFFECTS ESTIMATION
>> ------------------------
>> Number of groups = 8653 Obs per
>> group: min = 2
>>
>> avg = 213.2
>>
>> max = 873
>>
>> OLS estimation
>> --------------
>>
>> Estimates efficient for homoskedasticity only
>> Statistics consistent for homoskedasticity only
>>
>> Number
>> of obs = 1845155
>> F(
>> 1,1836501) = 852.88
>> Prob >
>> F = 0.0000
>> Total (centered) SS = 6.67941e+11
>> Centered R2 = 0.0005
>> Total (uncentered) SS = 6.67941e+11
>> Uncentered R2 = 0.0005
>> Residual SS = 6.67631e+11 Root
>> MSE = 602.9
>>
>> --------------------------------------------------------------
>> ----------------
>> highprice | Coef. Std. Err. z P>|z|
>> [95% Conf. Interval]
>> -------------+------------------------------------------------
>> ----------------
>> volume | -.0037862 .0001296 -29.20 0.000
>> -.0040403 -.0035321
>> --------------------------------------------------------------
>> ----------------
>> Included instruments: volume
>> --------------------------------------------------------------
>> ----------------
>> r; t=50.46 12:34:19
>>
>> Just a note that this does in fact take longer, 50.46 seconds. Also,
>> the 172 observations are not included in the regression as noted by
>> number of obs. I thought it would be best to look at the xtreg
>> regression without the 172 observations. So I attempted to run the
>> xtreg regression with the sample from xtivreg2 only to find that all
>> of the observations are included in the sample from xtivreg2.
>>
>> . gen mysample = e(sample)
>> r; t=0.20 12:35:04
>>
>> . tab mysample
>>
>> mysample | Freq. Percent Cum.
>> ------------+-----------------------------------
>> 1 | 1,845,327 100.00 100.00
>> ------------+-----------------------------------
>> Total | 1,845,327 100.00
>> r; t=0.56 12:35:07
>>
>> Can someone explain what is going on here? Is xtivreg2 actually using
>> the 172 observations or not?
>>
>> My plan is to drop the singleton groups. However, I am running
>> regressions with different groups of independent variables with
>> missing observations. This could potentially create singleton groups
>> again. Is there a way to make xtreg ignore the singleton groups
>> automatically as well?
>>
>> Additionally, does anyone know of a command that will let me calculate
>> the two-way clustered errors after xtreg? I don't really see the point
>> in running the same regression again just to cluster the errors
>> differently but short of coding the two-way clustered errors myself I
>> don't know how else to do this.
>>
>> Chris
>>
>>
>> Chris Parker
>>
>> ________________________________
>>
>> PhD Candidate | Management Science & Operations
>> London Business School | Regent's Park | London NW1 4SA |
>> United Kingdom
>> Direct line +44 (0)20 7000 8816 | Email [email protected]
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/statalist/faq
>> * http://www.ats.ucla.edu/stat/stata/
>>
>
>
> --
> Heriot-Watt University is a Scottish charity
> registered under charity number SC000278.
>
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
> ______________________________________________________________________
>
> This email has been scanned by the MessageLabs Email Security System
> on behalf of the London Business School community.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________
>
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/