Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

# st: Output problem attnd , attr, atts

 From "Jason Zarmulski" To statalist@hsphsun2.harvard.edu Subject st: Output problem attnd , attr, atts Date Tue, 28 Jun 2011 11:10:17 +0200

``` Hi Statalists,
>
>
> Now I have the problem with interpreting the different outcomes.

My Questions:
>
> -How can I Interpret the different Outcomes?
> -Why isnt the .pstest method working with these outcomes?
> -Are these Outcomes my the final ones, or do I have to do some robustness
> tests as well? (If yes, which one?)
>
>

> My new Propensity Score and the different outcomes are:
>
>
> The balancing property is satisfied
>
>
> This table shows the inferior bound, the number of treated
> and the number of controls for each block
>
>   Inferior |
>   of block |      REL(Dummy)
> of pscore  |         0          1 |     Total
> -----------+----------------------+----------
>         .2 |         5          6 |        11
>         .4 |        16         21 |        37
>         .6 |        20         17 |        37
>         .7 |        14         47 |        61
>         .8 |        19        110 |       129
>         .9 |         9        223 |       232
> -----------+----------------------+----------
>      Total |        83        424 |       507
>
>
>
> . attnd stockprice2010 reldummy, pscore(mypscore)comsup
>
>
>  The program is searching the nearest neighbor of each treated unit.
>  This operation may take a while.
>
>
>
> ATT estimation with Nearest Neighbor Matching method
> (random draw version)
> Analytical standard errors
>
> ---------------------------------------------------------
> n. treat.   n. contr.         ATT   Std. Err.           t
> ---------------------------------------------------------
>
>       424          65      -0.572       6.828      -0.084
>
> ---------------------------------------------------------
> Note: the numbers of treated and controls refer to actual
> nearest neighbour matches
>
>
>
> . attr stockprice2010 reldummy, pscore(mypscore) radius(0.1) comsup
>
>
>  The program is searching for matches of treated units within radius.
>  This operation may take a while.
>
>
>
> ATT estimation with the Radius Matching method
> Analytical standard errors
>
> ---------------------------------------------------------
> n. treat.   n. contr.         ATT   Std. Err.           t
> ---------------------------------------------------------
>
>       424          83       1.354       3.588       0.377
>
> ---------------------------------------------------------
> Note: the numbers of treated and controls refer to actual
>
>
> . atts stockprice2010 reldummy, pscore(mypscore) comsup blockid(myblock)
>
>
>
>
> ATT estimation with the Stratification method
> Analytical standard errors
>
> ---------------------------------------------------------
> n. treat.   n. contr.         ATT   Std. Err.           t
> ---------------------------------------------------------
>
>       424          83       0.273       2.520       0.109
>
> ---------------------------------------------------------
>
>Thanks
Jason
>
>
>
> --- Ursprüngliche Nachrsht ---
> Datum: 21.06.2011 15:04:02
> An: "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
>
> Betreff: RE: st: RE: Ousout problem psmatch2
>
> > Ah, yes. You are right - my mistake.
> >
> > Regarding -attnd- I see a few reasons why you can get different results:
>
> >
> > - using -psmatch2- you "restrict" to 10 neighbours. What is
> the
> > number og nearest neighbours selected by -attnd-?
> > - you set caliper to 0.1 using psmatch2. But I don't think a similar
> "restriction"
> > is imposed in attnd?
> >
> > Maybe this is a start...
> >
> > /Jan
> >
> >
> > -----Original Message-----
> > From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu]
>
> > On Behalf Of Jason Zarmulski
> > Sent: 21. juni 2011 12:23
> > To: statalist@hsphsun2.harvard.edu
> > Subject: Re: st: RE: Ousout problem psmatch2
> >
> > Hi Jan,
> >
> > thanks a lot for your support, I changed it and my new output is:
> > :-)
> >
> > . psmatch2 reldummy , outcome( stockprice2010) pscore(mypscore) neighbor(10)
>
> > caliper(0.1) ate
> > ----------------------------------------------------------------------------------------
>
> >
> >         Variable     Sample |    Treated     Controls   Difference
> >   S.E.   T-stat
> > ----------------------------+-----------------------------------------------------------
>
> >
> >   stockprice2010  Unmatched | 20.6236969   21.7278571  -1.10416018
>  2.08916252
> >    -0.53
> >                         ATT | 20.6236969   20.4730466   .150650303
>  4.16715274
> >     0.04
> >                         ATU | 21.7278571   17.8625509  -3.86530621
> >      .        .
> >                         ATE |                          -.513405498
>
> >      .        .
> > ----------------------------+-----------------------------------------------------------
>
> >
> > Note: S.E. does not take into account that the propensity score is estimated.
>
> >
> >
> >            | psmatch2:
> >  psmatch2: |   Common
> >  Treatment |  support
> > assignment | On suppor |     Total
> > -----------+-----------+----------
> >  Untreated |        84 |        84
> >    Treated |       424 |       424
> > -----------+-----------+----------
> >      Total |       508 |       508
> >
> >
> > To the attnd problem : I didnt install a package for attnd. Maybe it
> was
> > with the"nnmatch ado" package
> >
> > The differences between the psmatch2 and the attnd are still there....
>
> >
> > This is the help attnd file:
> >
> >
> > Calculate the average treatment effect on the treated using nearest
> neighbor
> > matching
> >
> >             attnd outcome treatment [varlist] [weight] [if exp] [in
> range]
> > [ ,
> >                   pscore(scorevar) logit index comsup detail bootstrap
> reps(#)
> >
> >                   noisily dots ]
> >
> >         fweights, iweights, and pweights are allowed; see help weights.
>
> >
> >
> > Description
> >
> >     attnd estimates the average treatment effect on the treated (ATT)
> using
> > nearest
> >     neighbor matching. attnd should be run after the correct propensity
> score
> >
> >     specification; i.e., the one satisfying the balancing property has
> been
> > found
> >     using, for example, pscore. If users do not provide a variable name
> for
> > the
> >     propensity score, the propensity score is estimated based on the
> specification
> >
> >     in varlist. Note that in this case the balancing property is not
> tested.
> >
> >
> >     It is left under the responsibility of the user to select the comsup
>
> > option if
> >     the user provided propensity score has been estimated on a common
> support
> > for
> >     treated and controls.  Otherwise, the ATT is estimated using also
> the
> >
> >     observations outside the common support for which the propensity
> score
> > may not
> >     be balanced.
> >
> >     To save on computing time, nearest neighbors are not determined
> by comparing
> >
> >     treated observations to every single control, but by first sorting
> all
> > records
> >     by the estimated propensity score and then searching forward and
> backward
> > for
> >     the closest control unit(s). If a treated unit forward and backward
> matches
> >
> >     happen to be equally good, this program randomly draws (hence the
> letters
> > "nd"
> >     for Nearest neighbor and random Draw) either the forward or backward
>
> > matches.
> >     This approach is one of two computationally feasible options to
> obtain
> >
> >     analytical standard errors while at the same time exploiting the
> very
> > fast
> >     forward and backward search strategy. The second possibility is
> based
> > on giving
> >     equal weight to the groups of forward and backward matches in case
> of
> > equally
> >     good forward and backward matches and is performed by attnw. In
> practice,
> > the
> >     case of multiple nearest neighbors should be very rare. In particular,
>
> > if the
> >     set of X's contains continuous variables, in which case, both attnd
> and
> > attnw
> >     should give equal results (except for bootstrapped standard errors).
>
> > The
> >     likelihood of multiple nearest neighbors is further reduced if the
> propensity
> >
> >     score is estimated and saved in double precision, which is what
> pscore
> > does by
> >     default.
> >
> >     The ATT is computed by averaging over the unit-level treatment effects
>
> > of the
> >     treated where the control(s) matched to a treated observation is/are
>
> > those
> >     observations in the control group that have the closest propensity
> score.
> > If
> >     there are multiple nearest neighbors, the average outcome of those
> controls
> > is
> >     used.
> >
> >
> > Options
> >
> >     pscore(scorevar) specifies the name of the user-provided variable
> name
> > for the
> >         estimated propensity score. If no name is provided the propensity
>
> > score is
> >         estimated based on the specification in varlist.
> >
> >     logit uses a logit model to estimate the propensity score instead
> of
> > the default
> >         probit model when the option pscore(scorevar) is not specified
> by
> > the user.
> >         Otherwise, no effect is produced.
> >
> >     index requires the use of the linear index as the propensity score
> when
> > the
> >         option pscore(scorevar) is not specified by the user.  Otherwise,
>
> > no effect
> >         is produced.
> >
> >     comsup restricts the computation of the ATT to the region of common
> support.
> >
> >
> >     detail displays more detailed output documenting the steps performed
>
> > to obtain
> >         the final results.
> >
> >     bootstrap bootstraps the standard error of the treatment effect.
>
> >
> >     reps(#) specifies the number of bootstrap replications to be performed.
>
> >  The
> >         default is 50.  This option produces an effect only if the bootstrap
>
> > option
> >         is specified.
> >
> >     noisily requests that any output from the replications be displayed.
>
> >  This
> >         option produces an effect only if the bootstrap option is specified.
>
> >
> >
> >     dots requests that a dot be placed on the screen at the beginning
> of
> > each
> >         replication.  This option produces an effect only if the bootstrap
>
> > option is
> >         specified.
> >
> >
> > Remarks
> >
> >     Please remember to use the update query command before running this
> program
> > to
> >     make sure you have an up-to-date version of Stata installed.  Otherwise,
>
> > this
> >     program may not run properly.
> >
> >     The treatment has to be binary.
> >
> >     When users do not specify their own previously estimated propensity
> score,
> > the
> >     bootstrap encompasses the estimation of the propensity score based
> on
> > the
> >     specification given by varlist. This procedure is actually recommended
>
> > to
> >     account for the uncertainty associated with the estimation of the
> propensity
> >
> >     score.  Even more so when the comsup option is specified because
> in this
> > case
> >     the region of common support changes with every bootstrap sample,
> and
> >
> >     bootstrapped standard errors pick up this uncertainty as well. So,
> typically
> >
> >     users would first identify a specification satisfying the balancing
> property
> > --
> >     using pscore -- and then provide exactly this specification in varlist
>
> > and use
> >     bootstrapped standard errors.
> >
> >
> > Saved results
> >
> >     The program stores the estimated treatment effect, its standard
> error,
> > and the t
> >     statistic respectively in the scalars r(attnd), r(seattnd), and
> r(tsattnd).
> >
> >
> >     The number of treated and the number of controls are stored respectively
>
> > in the
> >     scalars r(ntnd) and r(ncnd).
> >
> >     The bootstrapped standard error and t statistic are stored respectively
>
> > in the
> >     scalars r(bseattnd) and r(btsattnd).
> >
> >
> > Examples
> >
> >     . attnd wage training age age2 exp exp2
> >
> >     . attnd wage training age age2 exp exp2, boot reps(100) dots
> >
> >     . attnd wage training age age2 exp exp2, logit boot reps(100)
> >
> >     . attnd wage training age age2 exp exp2, comsup boot reps(100)
> >
> >
> > Authors
> >
> >         Sascha O. Becker
> >         Center for Economic Studies, University of Munich
> >
> >         Andrea Ichino
> >         Department of Economics, European University Institute, Florence
>
> >
> >
> >     Email so.b@gmx.net or andrea.ichino@iue.it if you observe any problems.
>
> >
> >
> >
> > Acknowledgments
> >
> >     The way to implement the propensity score estimation in the bootstrap
>
> > procedure
> >     has been adapted from the psmatch program written by Barbara Sianesi
>
> > (University
> >     College London and Institute for Fiscal Studies) Email: barbara_s@ifs.org.uk.
>
> >
> >
> >
> > Also see
> >
> >     Online:  help for pscore, atts, attr, attk, attnw (if installed),
> and
> > bs.
> >
> >               Further details on the analytical formulas and on the
> algorithms
> > used
> >               in these programs can be found under http://www.sobecker.de
>
> > or
> >               http://www.iue.it/Personal/Ichino.
> >
> > Thanks
> > Jason
> >
> >
> > --- Ursprüngliche Nachricht ---
> > Von: Jan Bryla <JBR@finansraadet.dk>
> > Datum: 20.06.2011 20:14:03
> > An: "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
>
> >
> > Betreff: st: RE: Ousout problem psmatch2
> >
> > > Jason, your first question seems easy to solve: I think your treatment
>
> > variable
> > > and the outcome variable are identical. Recall the syntax for psmatch2,
>
> > see
> > > -help psmatch2-.
> > >
> > > The second point left me a bit confused. Searching for -attnd-
> using
> > -findit
> > > attnd- didn't really turn out any hints. Is -attnd- available from
> SSC?
> > Maybe
> > > you can clarify your steps there? Maybe differences are due to
> the issue
> >
> > > with the first question.
> > >
> > > Hope it helps
> > > Jan Bryla
> > >
> > >
> > > -----Original Message-----
> > > From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu]
>
> >
> > > On Behalf Of Jason Zarmulski
> > > Sent: 20. juni 2011 16:45
> > > To: statalist@hsphsun2.harvard.edu
> > > Subject: st: Ousout problem psmatch2
> > >
> > > Dear Statalists,
> > > I got two problems, first one:
> > >
> > > I have a problem understanding the Output of my psmatch2 results.
> I'm
> > new
> > > to
> > > this so it could be a trivial error, but I'm not sure. I wanted
> to do
> > a
> > > nearest neighbor matching with replacement.
> > > My results are:
> > >
> > > . psmatch2 stockprice2010, outcome( stockprice2010) pscore(mypscore)
>
> >
> > > neighbor(10) caliper(0.1) ate
> > > ----------------------------------------------------------------------------------------
>
> >
> > >
> > >         Variable     Sample |    Treated     Controls   Difference
>
> > > S.E.   T-stat
> > > ----------------------------+-----------------------------------------------------------
>
> >
> > >
> > >   stockprice2010  Unmatched |      1.628   .479999997    .99964329
>
> > > .000849677  1176.50
> > >                         ATT |       1.62   .689999992   .930000013
>
> >
> > > .399478823     2.33
> > >                         ATU | .479999997   1.79500002   1.31500002
>
> > > .        .
> > >                         ATE |                           1.05833335
>
> >
> > > .        .
> > > ----------------------------+-----------------------------------------------------------
>
> >
> > >
> > > Note: S.E. does not take into account that the propensity score
> is
> > > estimated.
> > >
> > >  psmatch2: |   psmatch2: Common
> > >  Treatment |        support
> > > assignment | Off suppo  On suppor |     Total
> > > -----------+----------------------+----------
> > >  Untreated |         0          2 |         2
> > >    Treated |         1          4 |         5
> > >          2 |         0         11 |        11
> > >          3 |         0         12 |        12
> > >          4 |         0         17 |        17
> > >          5 |         0         34 |        34
> > >          6 |         0         15 |        15
> > >          7 |         0         19 |        19
> > >          8 |         0         18 |        18
> > >          9 |         0         17 |        17
> > >         10 |         0         18 |        18
> > >         11 |         0         13 |        13
> > >         12 |         0         13 |        13
> > >         13 |         0         20 |        20
> > >         14 |         0         23 |        23
> > >         15 |         0         23 |        23
> > >         16 |         0         10 |        10
> > >         17 |         0         13 |        13
> > >         18 |         0         10 |        10
> > >         19 |         0         11 |        11
> > >         20 |         0         11 |        11
> > >         21 |         0         10 |        10
> > >         22 |         0         10 |        10
> > >         23 |         0         11 |        11
> > >         24 |         0         10 |        10
> > >         25 |         0          7 |         7
> > >         26 |         0          9 |         9
> > >         27 |         0          6 |         6
> > >         28 |         0          7 |         7
> > >         29 |         0          4 |         4
> > >         30 |         0          7 |         7
> > >         31 |         0          6 |         6
> > >         32 |         0          7 |         7
> > >         33 |         0          7 |         7
> > >         34 |         0          5 |         5
> > >         35 |         0          6 |         6
> > >         36 |         0          6 |         6
> > >         37 |         0          6 |         6
> > >         38 |         0          5 |         5
> > >         39 |         0          4 |         4
> > >         40 |         0          4 |         4
> > >         41 |         0          9 |         9
> > >         42 |         0          7 |         7
> > >         43 |         0          5 |         5
> > >         44 |         0          1 |         1
> > >         45 |         0          1 |         1
> > >         46 |         0          1 |         1
> > >         47 |         0          1 |         1
> > >         48 |         0          3 |         3
> > >         49 |         0          4 |         4
> > >         50 |         0          1 |         1
> > >         51 |         0          1 |         1
> > >         52 |         0          1 |         1
> > >         53 |         0          1 |         1
> > >         55 |         0          1 |         1
> > >         56 |         0          3 |         3
> > >         57 |         0          1 |         1
> > >         59 |         0          1 |         1
> > >         60 |         0          2 |         2
> > >         62 |         0          1 |         1
> > >         63 |         0          2 |         2
> > >         69 |         0          1 |         1
> > >         70 |         0          1 |         1
> > >         78 |         0          1 |         1
> > >         83 |         0          1 |         1
> > >         84 |         0          2 |         2
> > >         92 |         0          1 |         1
> > > -----------+----------------------+----------
> > >      Total |         1        505 |       506
> > >
> > > Question: Why arent there only the untreated and treated in the
> table
> > and
> > >
> > > what ist the meaning of the numbers 2-92?
> > >
> > >
> > > Second problem:
> > >
> > > I've done the nearest neighbor matching of the same data sample
> with
> > the
> > >
> > > attnd function.
> > > My results are:
> > >
> > > . attnd  stockprice2010 reldummy, pscore(mypscore)
> > >
> > >
> > >  The program is searching the nearest neighbor of each treated
> unit.
> >
> > >  This operation may take a while.
> > >
> > >
> > >
> > > ATT estimation with Nearest Neighbor Matching method
> > > (random draw version)
> > > Analytical standard errors
> > >
> > > ---------------------------------------------------------
> > > n. treat.   n. contr.         ATT   Std. Err.           t
> > > ---------------------------------------------------------
> > >
> > >       424          63      -2.255       6.070      -0.371
> > >
> > > ---------------------------------------------------------
> > > Note: the numbers of treated and controls refer to actual
> > > nearest neighbour matches
> > >
> > > Why is this result for ATT so different to the one of psmatch2?
>
> > > Is the attnd matching with replacement?
> > >
> > > Any suggestions would be much appreciated
> > >
> > > Thanks
> > >
> > > Jason
> > >
> > > --
> > > View this message in context: http://statalist.1588530.n2.nabble.com/Ousout-problem-psmatch2-tp6496035p6496035.html
>
> >
> > >
> > > Sent from the Statalist mailing list archive at Nabble.com.
> > > *
> > > *   For searches and help try:
> > > *   http://www.stata.com/help.cgi?search
> > > *   http://www.stata.com/support/statalist/faq
> > > *   http://www.ats.ucla.edu/stat/stata/
> > >
> > > *
> > > *   For searches and help try:
> > > *   http://www.stata.com/help.cgi?search
> > > *   http://www.stata.com/support/statalist/faq
> > > *   http://www.ats.ucla.edu/stat/stata/
> >
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/help.cgi?search
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
> >
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/help.cgi?search
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```