Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: RE: Treatreg with Bootstrap SEs - first stage
From
Guy Grossman <[email protected]>
To
[email protected]
Subject
Re: st: RE: Treatreg with Bootstrap SEs - first stage
Date
Wed, 9 Mar 2011 13:06:13 -0500
Thanks Jeff - this has been extremely useful.
First are the regression results without clustering nor bootstrap when
the reduced form is restricted. Then are regression results with
unrestricted reduced form and without clustering nor bootstrap
Guy
treatreg stranger npos_before agedc, treat(vrule =z)
Treatment-effects model -- MLE Number of obs = 44
Wald chi2(3) = 3.10
Log likelihood = -293.30954 Prob > chi2 = 0.3766
------------------------------------------------------------------------------
| Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
stranger |
npos_before | -4.003312 3.670015 -1.09 0.275 -11.19641 3.189785
agedc | 19.2581 19.78198 0.97 0.330 -19.51387 58.03006
vrule | -36.13351 51.39635 -0.70 0.482 -136.8685 64.60148
_cons | 233.3789 77.89184 3.00 0.003 80.71371 386.0441
-------------+----------------------------------------------------------------
vrule |
z | 2.478021 .5804117 4.27 0.000 1.340435 3.615607
_cons | -.9345324 .2744347 -3.41 0.001 -1.472415 -.3966503
-------------+----------------------------------------------------------------
/athrho | -.1668083 .2934679 -0.57 0.570 -.7419949 .4083783
/lnsigma | 4.866947 .1075092 45.27 0.000 4.656233 5.077661
-------------+----------------------------------------------------------------
rho | -.1652782 .2854513 -.6303489 .3870948
sigma | 129.9237 13.96799 105.2389 160.3985
lambda | -21.47355 37.48562 -94.94401 51.99692
------------------------------------------------------------------------------
LR test of indep. eqns. (rho = 0): chi2(1) = 0.32 Prob > chi2 = 0.5690
------------------------------------------------------------------------------
treatreg stranger npos_before agedc, treat(vrule =z npos_before agedc) first
Probit regression Number of obs = 44
LR chi2(3) = 26.60
Prob > chi2 = 0.0000
Log likelihood = -16.787309 Pseudo R2 = 0.4421
------------------------------------------------------------------------------
vrule | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
z | 2.504705 .5923703 4.23 0.000 1.343681 3.66573
npos_before | .0312567 .0485168 0.64 0.519 -.0638344 .1263479
agedc | .0488113 .2522365 0.19 0.847 -.4455631 .5431857
_cons | -1.402968 1.018711 -1.38 0.168 -3.399605 .5936681
------------------------------------------------------------------------------
Treatment-effects model -- MLE Number of obs = 44
Wald chi2(3) = 3.42
Log likelihood = -293.13633 Prob > chi2 = 0.3319
------------------------------------------------------------------------------
| Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
stranger |
npos_before | -4.209007 3.680126 -1.14 0.253 -11.42192 3.003907
agedc | 19.04293 19.88008 0.96 0.338 -19.9213 58.00717
vrule | -39.05213 51.3033 -0.76 0.447 -139.6047 61.50049
_cons | 237.2232 77.54155 3.06 0.002 85.2446 389.2019
-------------+----------------------------------------------------------------
vrule |
z | 2.519301 .5973311 4.22 0.000 1.348553 3.690048
npos_before | .0285118 .0492956 0.58 0.563 -.0681059 .1251294
agedc | .0302288 .2572896 0.12 0.906 -.4740496 .5345072
_cons | -1.311585 1.045363 -1.25 0.210 -3.360458 .737288
-------------+----------------------------------------------------------------
/athrho | -.1417195 .2938456 -0.48 0.630 -.7176462 .4342073
/lnsigma | 4.866185 .107235 45.38 0.000 4.656008 5.076361
-------------+----------------------------------------------------------------
rho | -.1407782 .288022 -.6154492 .4088314
sigma | 129.8247 13.92175 105.2152 160.1901
lambda | -18.27649 37.67748 -92.12299 55.57002
------------------------------------------------------------------------------
LR test of indep. eqns. (rho = 0): chi2(1) = 0.24 Prob > chi2 = 0.6277
------------------------------------------------------------------------------
On Wed, Mar 9, 2011 at 12:52 PM, Wooldridge, Jeffrey <[email protected]> wrote:
> I just noticed another problem which is probably not related to the standard error issue. You're only including z in the reduced form for vrule. While some find it acceptable, most do not. You are making restrictions on a reduced form by assuming npos_before and agedc have no partial effect on vrule
>
> What do the treatreg standard errors when you neither cluster nor use the bootstrap?
>
> By the way, there is nothing wrong with the usual 2SLS estimator in this context even though vrule is binary. I would use that, too, as it forces you to estimate an unrestricted reduced form.
>
> -----Original Message-----
> From: [email protected] [mailto:[email protected]] On Behalf Of Guy Grossman
> Sent: Wednesday, March 09, 2011 12:47 PM
> To: [email protected]
> Subject: Re: st: RE: Treatreg with Bootstrap SEs - first stage
>
> Thanks Jeff - see below results from the first stage.
>
>
> Probit regression Number of obs = 44
> LR chi2(1) = 26.17
> Prob > chi2 = 0.0000
> Log likelihood = -17.005048 Pseudo R2 = 0.4348
>
> ------------------------------------------------------------------------------
> vrule | Coef. Std. Err. z P>|z| [95% Conf. Interval]
> -------------+----------------------------------------------------------------
> z | 2.445756 .5688183 4.30 0.000 1.330892 3.560619
> _cons | -.9446696 .2747046 -3.44 0.001 -1.483081 -.4062585
> ------------------------------------------------------------------------------
>
>
> On Wed, Mar 9, 2011 at 12:17 PM, Wooldridge, Jeffrey <[email protected]> wrote:
>> A few observations.
>>
>> 1. I don't see how the bootstrapped standard errors are robust to clustering. Where have you specified that the bootstrap should be done by resampling the clusters?
>> 2. More importantly, I think you should not be trying to cluster with 44 observations and five clusters. Cluster-robust inference is not justified with such a small number of clusters. Heck, you have more observations per cluster than number of clusters! You really need lots of clusters that aren't very large. I believe you can get spurious rejections when you cluster with such a small number of clusters. From Stata's perspective, you have five observations when you cluster.
>> 3. N = 44 is small to be using any kind of IV procedure, especially a nonlinear one. But if you must, you should not be clustering.
>> 4. If you estimate the first-stage probit for vrule without clustering or bootstrapping, what is the t statistic on z?
>>
>> Jeff
>>
>>
>> -----Original Message-----
>> From: [email protected] [mailto:[email protected]] On Behalf Of Guy Grossman
>> Sent: Wednesday, March 09, 2011 11:53 AM
>> To: [email protected]
>> Subject: st: Treatreg with Bootstrap SEs - first stage
>>
>> Dear friends,
>>
>> I run the fit the following IV model, where stranger is a continuous
>> dependent variable, and vrule is an endogenous binary predictor,
>> instrumented by z (also binary). The associastion between the
>> instrument z and the endogenous predictor (vrule) is strong.
>>
>> (1) stranger = npos_before + agedc + vrule + u
>> (2) vrule = z + e
>>
>> I first fit a model with clustered SEs. I then fit a second model
>> with bootstrapped SEs. What I find strange is the differences in the
>> SEs of the instrument in the bootstrap model. When standard errors
>> were clustered, the standard error of z is equal to .478 and is highly
>> significant, but in the bootstrap model the standard error of z is
>> equal to 6.58 (13 times larger).
>>
>> My question is what can explain such difference in results, given that
>> I know the association between the binary endogenous predictor and the
>> instrument is strong.
>>
>> Thanks!
>> Guy
>>
>>
>> eststo: treatreg stranger npos_before agedc, treat(vrule =z)
>> cluster(strata) nolog
>> Treatment-effects model -- MLE Number of obs = 44
>> Wald chi2(0) = .
>> Log pseudolikelihood = -293.30954 Prob > chi2 = .
>> (Std. Err. adjusted for 5 clusters in strata)
>> ------------------------------------------------------------------------------
>> | Robust
>> | Coef. Std. Err. z P>|z| [95% Conf. Interval]
>> -------------+----------------------------------------------------------------
>> stranger |
>> npos_before | -4.003312 2.181331 -1.84 0.066 -8.278642 .2720188
>> agedc | 19.2581 11.40533 1.69 0.091 -3.095944 41.61213
>> vrule | -36.13351 19.98177 -1.81 0.071 -75.29706 3.030042
>> _cons | 233.3789 33.044 7.06 0.000 168.6139 298.144
>> -------------+----------------------------------------------------------------
>> vrule |
>> z | 2.478021 .4780614 5.18 0.000 1.541038 3.415005
>> _cons | -.9345324 .1497078 -6.24 0.000 -1.227954 -.6411105
>> -------------+----------------------------------------------------------------
>> /athrho | -.1668083 .3755606 -0.44 0.657 -.9028935 .569277
>> /lnsigma | 4.866947 .1144261 42.53 0.000 4.642676 5.091218
>> -------------+----------------------------------------------------------------
>> rho | -.1652782 .3653015 -.7177039 .5148281
>> sigma | 129.9237 14.86666 103.8218 162.5878
>> lambda | -21.47355 47.79572 -115.1514 72.20435
>> ------------------------------------------------------------------------------
>> Wald test of indep. eqns. (rho = 0): chi2(1) = 0.20 Prob > chi2 = 0.6569
>> ------------------------------------------------------------------------------
>>
>> eststo: treatreg stranger npos_before agedc, treat(vrule =z)
>> vce(bootstrap, reps(1000)) first
>> Treatment-effects model -- MLE Number of obs = 44
>> Replications = 954
>> Wald chi2(3) = 3.76
>> Log likelihood = -293.30954 Prob > chi2 = 0.2892
>> ------------------------------------------------------------------------------
>> | Observed Bootstrap Normal-based
>> | Coef. Std. Err. z P>|z| [95% Conf. Interval]
>> -------------+----------------------------------------------------------------
>> stranger |
>> npos_before | -4.003312 3.781918 -1.06 0.290 -11.41573 3.409111
>> agedc | 19.2581 20.01746 0.96 0.336 -19.9754 58.49159
>> vrule | -36.13351 50.30376 -0.72 0.473 -134.7271 62.46005
>> _cons | 233.3789 86.26582 2.71 0.007 64.30102 402.4568
>> -------------+----------------------------------------------------------------
>> vrule |
>> z | 2.478021 6.583954 0.38 0.707 -10.42629 15.38233
>> _cons | -.9345324 .2802209 -3.33 0.001 -1.483755 -.3853095
>> -------------+----------------------------------------------------------------
>> /athrho | -.1668083 .4793972 -0.35 0.728 -1.10641 .772793
>> /lnsigma | 4.866947 .1126358 43.21 0.000 4.646185 5.087709
>> -------------+----------------------------------------------------------------
>> rho | -.1652782 .4663016 -.8027896 .6485506
>> sigma | 129.9237 14.63405 104.1868 162.0183
>> lambda | -21.47355 60.81632 -140.6713 97.72425
>> ------------------------------------------------------------------------------
>>
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/statalist/faq
>> * http://www.ats.ucla.edu/stat/stata/
>>
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/statalist/faq
>> * http://www.ats.ucla.edu/stat/stata/
>>
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/