Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
"Jason Zarmulski" <Jason.Zarmulski@mtox.de> |

To |
statalist@hsphsun2.harvard.edu |

Subject |
st: Output problem attnd , attr, atts |

Date |
Tue, 28 Jun 2011 11:10:17 +0200 |

Hi Statalists, > > > Now I have the problem with interpreting the different outcomes. My Questions: > > -How can I Interpret the different Outcomes? > -Why isnt the .pstest method working with these outcomes? > -Are these Outcomes my the final ones, or do I have to do some robustness > tests as well? (If yes, which one?) > > > My new Propensity Score and the different outcomes are: > > > The balancing property is satisfied > > > This table shows the inferior bound, the number of treated > and the number of controls for each block > > Inferior | > of block | REL(Dummy) > of pscore | 0 1 | Total > -----------+----------------------+---------- > .2 | 5 6 | 11 > .4 | 16 21 | 37 > .6 | 20 17 | 37 > .7 | 14 47 | 61 > .8 | 19 110 | 129 > .9 | 9 223 | 232 > -----------+----------------------+---------- > Total | 83 424 | 507 > > > > . attnd stockprice2010 reldummy, pscore(mypscore)comsup > > > The program is searching the nearest neighbor of each treated unit. > This operation may take a while. > > > > ATT estimation with Nearest Neighbor Matching method > (random draw version) > Analytical standard errors > > --------------------------------------------------------- > n. treat. n. contr. ATT Std. Err. t > --------------------------------------------------------- > > 424 65 -0.572 6.828 -0.084 > > --------------------------------------------------------- > Note: the numbers of treated and controls refer to actual > nearest neighbour matches > > > > . attr stockprice2010 reldummy, pscore(mypscore) radius(0.1) comsup > > > The program is searching for matches of treated units within radius. > This operation may take a while. > > > > ATT estimation with the Radius Matching method > Analytical standard errors > > --------------------------------------------------------- > n. treat. n. contr. ATT Std. Err. t > --------------------------------------------------------- > > 424 83 1.354 3.588 0.377 > > --------------------------------------------------------- > Note: the numbers of treated and controls refer to actual > matches within radius > > > . atts stockprice2010 reldummy, pscore(mypscore) comsup blockid(myblock) > > > > > ATT estimation with the Stratification method > Analytical standard errors > > --------------------------------------------------------- > n. treat. n. contr. ATT Std. Err. t > --------------------------------------------------------- > > 424 83 0.273 2.520 0.109 > > --------------------------------------------------------- > >Thanks Jason > > > > --- Ursprüngliche Nachrsht --- > Von: Jan Bryla <JBR@finansraadet.dk> > Datum: 21.06.2011 15:04:02 > An: "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu> > > Betreff: RE: st: RE: Ousout problem psmatch2 > > > Ah, yes. You are right - my mistake. > > > > Regarding -attnd- I see a few reasons why you can get different results: > > > > > - using -psmatch2- you "restrict" to 10 neighbours. What is > the > > number og nearest neighbours selected by -attnd-? > > - you set caliper to 0.1 using psmatch2. But I don't think a similar > "restriction" > > is imposed in attnd? > > > > Maybe this is a start... > > > > /Jan > > > > > > -----Original Message----- > > From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] > > > On Behalf Of Jason Zarmulski > > Sent: 21. juni 2011 12:23 > > To: statalist@hsphsun2.harvard.edu > > Subject: Re: st: RE: Ousout problem psmatch2 > > > > Hi Jan, > > > > thanks a lot for your support, I changed it and my new output is: > > :-) > > > > . psmatch2 reldummy , outcome( stockprice2010) pscore(mypscore) neighbor(10) > > > caliper(0.1) ate > > ---------------------------------------------------------------------------------------- > > > > > Variable Sample | Treated Controls Difference > > S.E. T-stat > > ----------------------------+----------------------------------------------------------- > > > > > stockprice2010 Unmatched | 20.6236969 21.7278571 -1.10416018 > 2.08916252 > > -0.53 > > ATT | 20.6236969 20.4730466 .150650303 > 4.16715274 > > 0.04 > > ATU | 21.7278571 17.8625509 -3.86530621 > > . . > > ATE | -.513405498 > > > . . > > ----------------------------+----------------------------------------------------------- > > > > > Note: S.E. does not take into account that the propensity score is estimated. > > > > > > > | psmatch2: > > psmatch2: | Common > > Treatment | support > > assignment | On suppor | Total > > -----------+-----------+---------- > > Untreated | 84 | 84 > > Treated | 424 | 424 > > -----------+-----------+---------- > > Total | 508 | 508 > > > > > > To the attnd problem : I didnt install a package for attnd. Maybe it > was > > with the"nnmatch ado" package > > > > The differences between the psmatch2 and the attnd are still there.... > > > > > This is the help attnd file: > > > > > > Calculate the average treatment effect on the treated using nearest > neighbor > > matching > > > > attnd outcome treatment [varlist] [weight] [if exp] [in > range] > > [ , > > pscore(scorevar) logit index comsup detail bootstrap > reps(#) > > > > noisily dots ] > > > > fweights, iweights, and pweights are allowed; see help weights. > > > > > > > Description > > > > attnd estimates the average treatment effect on the treated (ATT) > using > > nearest > > neighbor matching. attnd should be run after the correct propensity > score > > > > specification; i.e., the one satisfying the balancing property has > been > > found > > using, for example, pscore. If users do not provide a variable name > for > > the > > propensity score, the propensity score is estimated based on the > specification > > > > in varlist. Note that in this case the balancing property is not > tested. > > > > > > It is left under the responsibility of the user to select the comsup > > > option if > > the user provided propensity score has been estimated on a common > support > > for > > treated and controls. Otherwise, the ATT is estimated using also > the > > > > observations outside the common support for which the propensity > score > > may not > > be balanced. > > > > To save on computing time, nearest neighbors are not determined > by comparing > > > > treated observations to every single control, but by first sorting > all > > records > > by the estimated propensity score and then searching forward and > backward > > for > > the closest control unit(s). If a treated unit forward and backward > matches > > > > happen to be equally good, this program randomly draws (hence the > letters > > "nd" > > for Nearest neighbor and random Draw) either the forward or backward > > > matches. > > This approach is one of two computationally feasible options to > obtain > > > > analytical standard errors while at the same time exploiting the > very > > fast > > forward and backward search strategy. The second possibility is > based > > on giving > > equal weight to the groups of forward and backward matches in case > of > > equally > > good forward and backward matches and is performed by attnw. In > practice, > > the > > case of multiple nearest neighbors should be very rare. In particular, > > > if the > > set of X's contains continuous variables, in which case, both attnd > and > > attnw > > should give equal results (except for bootstrapped standard errors). > > > The > > likelihood of multiple nearest neighbors is further reduced if the > propensity > > > > score is estimated and saved in double precision, which is what > pscore > > does by > > default. > > > > The ATT is computed by averaging over the unit-level treatment effects > > > of the > > treated where the control(s) matched to a treated observation is/are > > > those > > observations in the control group that have the closest propensity > score. > > If > > there are multiple nearest neighbors, the average outcome of those > controls > > is > > used. > > > > > > Options > > > > pscore(scorevar) specifies the name of the user-provided variable > name > > for the > > estimated propensity score. If no name is provided the propensity > > > score is > > estimated based on the specification in varlist. > > > > logit uses a logit model to estimate the propensity score instead > of > > the default > > probit model when the option pscore(scorevar) is not specified > by > > the user. > > Otherwise, no effect is produced. > > > > index requires the use of the linear index as the propensity score > when > > the > > option pscore(scorevar) is not specified by the user. Otherwise, > > > no effect > > is produced. > > > > comsup restricts the computation of the ATT to the region of common > support. > > > > > > detail displays more detailed output documenting the steps performed > > > to obtain > > the final results. > > > > bootstrap bootstraps the standard error of the treatment effect. > > > > > reps(#) specifies the number of bootstrap replications to be performed. > > > The > > default is 50. This option produces an effect only if the bootstrap > > > option > > is specified. > > > > noisily requests that any output from the replications be displayed. > > > This > > option produces an effect only if the bootstrap option is specified. > > > > > > > dots requests that a dot be placed on the screen at the beginning > of > > each > > replication. This option produces an effect only if the bootstrap > > > option is > > specified. > > > > > > Remarks > > > > Please remember to use the update query command before running this > program > > to > > make sure you have an up-to-date version of Stata installed. Otherwise, > > > this > > program may not run properly. > > > > The treatment has to be binary. > > > > When users do not specify their own previously estimated propensity > score, > > the > > bootstrap encompasses the estimation of the propensity score based > on > > the > > specification given by varlist. This procedure is actually recommended > > > to > > account for the uncertainty associated with the estimation of the > propensity > > > > score. Even more so when the comsup option is specified because > in this > > case > > the region of common support changes with every bootstrap sample, > and > > > > bootstrapped standard errors pick up this uncertainty as well. So, > typically > > > > users would first identify a specification satisfying the balancing > property > > -- > > using pscore -- and then provide exactly this specification in varlist > > > and use > > bootstrapped standard errors. > > > > > > Saved results > > > > The program stores the estimated treatment effect, its standard > error, > > and the t > > statistic respectively in the scalars r(attnd), r(seattnd), and > r(tsattnd). > > > > > > The number of treated and the number of controls are stored respectively > > > in the > > scalars r(ntnd) and r(ncnd). > > > > The bootstrapped standard error and t statistic are stored respectively > > > in the > > scalars r(bseattnd) and r(btsattnd). > > > > > > Examples > > > > . attnd wage training age age2 exp exp2 > > > > . attnd wage training age age2 exp exp2, boot reps(100) dots > > > > . attnd wage training age age2 exp exp2, logit boot reps(100) > > > > . attnd wage training age age2 exp exp2, comsup boot reps(100) > > > > > > Authors > > > > Sascha O. Becker > > Center for Economic Studies, University of Munich > > > > Andrea Ichino > > Department of Economics, European University Institute, Florence > > > > > > > Email so.b@gmx.net or andrea.ichino@iue.it if you observe any problems. > > > > > > > > > Acknowledgments > > > > The way to implement the propensity score estimation in the bootstrap > > > procedure > > has been adapted from the psmatch program written by Barbara Sianesi > > > (University > > College London and Institute for Fiscal Studies) Email: barbara_s@ifs.org.uk. > > > > > > > > > Also see > > > > Online: help for pscore, atts, attr, attk, attnw (if installed), > and > > bs. > > > > Further details on the analytical formulas and on the > algorithms > > used > > in these programs can be found under http://www.sobecker.de > > > or > > http://www.iue.it/Personal/Ichino. > > > > Thanks > > Jason > > > > > > --- Ursprüngliche Nachricht --- > > Von: Jan Bryla <JBR@finansraadet.dk> > > Datum: 20.06.2011 20:14:03 > > An: "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu> > > > > > Betreff: st: RE: Ousout problem psmatch2 > > > > > Jason, your first question seems easy to solve: I think your treatment > > > variable > > > and the outcome variable are identical. Recall the syntax for psmatch2, > > > see > > > -help psmatch2-. > > > > > > The second point left me a bit confused. Searching for -attnd- > using > > -findit > > > attnd- didn't really turn out any hints. Is -attnd- available from > SSC? > > Maybe > > > you can clarify your steps there? Maybe differences are due to > the issue > > > > > with the first question. > > > > > > Hope it helps > > > Jan Bryla > > > > > > > > > -----Original Message----- > > > From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] > > > > > > On Behalf Of Jason Zarmulski > > > Sent: 20. juni 2011 16:45 > > > To: statalist@hsphsun2.harvard.edu > > > Subject: st: Ousout problem psmatch2 > > > > > > Dear Statalists, > > > I got two problems, first one: > > > > > > I have a problem understanding the Output of my psmatch2 results. > I'm > > new > > > to > > > this so it could be a trivial error, but I'm not sure. I wanted > to do > > a > > > nearest neighbor matching with replacement. > > > My results are: > > > > > > . psmatch2 stockprice2010, outcome( stockprice2010) pscore(mypscore) > > > > > > neighbor(10) caliper(0.1) ate > > > ---------------------------------------------------------------------------------------- > > > > > > > > > Variable Sample | Treated Controls Difference > > > > S.E. T-stat > > > ----------------------------+----------------------------------------------------------- > > > > > > > > > stockprice2010 Unmatched | 1.628 .479999997 .99964329 > > > > .000849677 1176.50 > > > ATT | 1.62 .689999992 .930000013 > > > > > > .399478823 2.33 > > > ATU | .479999997 1.79500002 1.31500002 > > > > . . > > > ATE | 1.05833335 > > > > > > . . > > > ----------------------------+----------------------------------------------------------- > > > > > > > > > Note: S.E. does not take into account that the propensity score > is > > > estimated. > > > > > > psmatch2: | psmatch2: Common > > > Treatment | support > > > assignment | Off suppo On suppor | Total > > > -----------+----------------------+---------- > > > Untreated | 0 2 | 2 > > > Treated | 1 4 | 5 > > > 2 | 0 11 | 11 > > > 3 | 0 12 | 12 > > > 4 | 0 17 | 17 > > > 5 | 0 34 | 34 > > > 6 | 0 15 | 15 > > > 7 | 0 19 | 19 > > > 8 | 0 18 | 18 > > > 9 | 0 17 | 17 > > > 10 | 0 18 | 18 > > > 11 | 0 13 | 13 > > > 12 | 0 13 | 13 > > > 13 | 0 20 | 20 > > > 14 | 0 23 | 23 > > > 15 | 0 23 | 23 > > > 16 | 0 10 | 10 > > > 17 | 0 13 | 13 > > > 18 | 0 10 | 10 > > > 19 | 0 11 | 11 > > > 20 | 0 11 | 11 > > > 21 | 0 10 | 10 > > > 22 | 0 10 | 10 > > > 23 | 0 11 | 11 > > > 24 | 0 10 | 10 > > > 25 | 0 7 | 7 > > > 26 | 0 9 | 9 > > > 27 | 0 6 | 6 > > > 28 | 0 7 | 7 > > > 29 | 0 4 | 4 > > > 30 | 0 7 | 7 > > > 31 | 0 6 | 6 > > > 32 | 0 7 | 7 > > > 33 | 0 7 | 7 > > > 34 | 0 5 | 5 > > > 35 | 0 6 | 6 > > > 36 | 0 6 | 6 > > > 37 | 0 6 | 6 > > > 38 | 0 5 | 5 > > > 39 | 0 4 | 4 > > > 40 | 0 4 | 4 > > > 41 | 0 9 | 9 > > > 42 | 0 7 | 7 > > > 43 | 0 5 | 5 > > > 44 | 0 1 | 1 > > > 45 | 0 1 | 1 > > > 46 | 0 1 | 1 > > > 47 | 0 1 | 1 > > > 48 | 0 3 | 3 > > > 49 | 0 4 | 4 > > > 50 | 0 1 | 1 > > > 51 | 0 1 | 1 > > > 52 | 0 1 | 1 > > > 53 | 0 1 | 1 > > > 55 | 0 1 | 1 > > > 56 | 0 3 | 3 > > > 57 | 0 1 | 1 > > > 59 | 0 1 | 1 > > > 60 | 0 2 | 2 > > > 62 | 0 1 | 1 > > > 63 | 0 2 | 2 > > > 69 | 0 1 | 1 > > > 70 | 0 1 | 1 > > > 78 | 0 1 | 1 > > > 83 | 0 1 | 1 > > > 84 | 0 2 | 2 > > > 92 | 0 1 | 1 > > > -----------+----------------------+---------- > > > Total | 1 505 | 506 > > > > > > Question: Why arent there only the untreated and treated in the > table > > and > > > > > > what ist the meaning of the numbers 2-92? > > > > > > > > > Second problem: > > > > > > I've done the nearest neighbor matching of the same data sample > with > > the > > > > > > attnd function. > > > My results are: > > > > > > . attnd stockprice2010 reldummy, pscore(mypscore) > > > > > > > > > The program is searching the nearest neighbor of each treated > unit. > > > > > This operation may take a while. > > > > > > > > > > > > ATT estimation with Nearest Neighbor Matching method > > > (random draw version) > > > Analytical standard errors > > > > > > --------------------------------------------------------- > > > n. treat. n. contr. ATT Std. Err. t > > > --------------------------------------------------------- > > > > > > 424 63 -2.255 6.070 -0.371 > > > > > > --------------------------------------------------------- > > > Note: the numbers of treated and controls refer to actual > > > nearest neighbour matches > > > > > > Why is this result for ATT so different to the one of psmatch2? > > > > Is the attnd matching with replacement? > > > > > > Any suggestions would be much appreciated > > > > > > Thanks > > > > > > Jason > > > > > > -- > > > View this message in context: http://statalist.1588530.n2.nabble.com/Ousout-problem-psmatch2-tp6496035p6496035.html > > > > > > > > > Sent from the Statalist mailing list archive at Nabble.com. > > > * > > > * For searches and help try: > > > * http://www.stata.com/help.cgi?search > > > * http://www.stata.com/support/statalist/faq > > > * http://www.ats.ucla.edu/stat/stata/ > > > > > > * > > > * For searches and help try: > > > * http://www.stata.com/help.cgi?search > > > * http://www.stata.com/support/statalist/faq > > > * http://www.ats.ucla.edu/stat/stata/ > > > > * > > * For searches and help try: > > * http://www.stata.com/help.cgi?search > > * http://www.stata.com/support/statalist/faq > > * http://www.ats.ucla.edu/stat/stata/ > > > > * > > * For searches and help try: > > * http://www.stata.com/help.cgi?search > > * http://www.stata.com/support/statalist/faq > > * http://www.ats.ucla.edu/stat/stata/ > > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**st: RE: Output problem attnd , attr, atts***From:*Jan Bryla <JBR@finansraadet.dk>

**References**:**st: Ousout problem psmatch2***From:*Jason Zarmulski <jason.zarmulski@mtox.de>

**st: RE: Ousout problem psmatch2***From:*Jan Bryla <JBR@finansraadet.dk>

**Re: st: RE: Ousout problem psmatch2***From:*"Jason Zarmulski" <Jason.Zarmulski@mtox.de>

**RE: st: RE: Ousout problem psmatch2***From:*Jan Bryla <JBR@finansraadet.dk>

**Re: RE: st: RE: Ousout problem psmatch2***From:*"Jason Zarmulski" <Jason.Zarmulski@mtox.de>

- Prev by Date:
**Re: st: A problem while dealing with massive amount of data** - Next by Date:
**st: Analysis of experiment involving baseline measurements** - Previous by thread:
**Re: RE: st: RE: Ousout problem psmatch2** - Next by thread:
**st: RE: Output problem attnd , attr, atts** - Index(es):