Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

Re: RE: st: RE: Ousout problem psmatch2

 From "Jason Zarmulski" To statalist@hsphsun2.harvard.edu Subject Re: RE: st: RE: Ousout problem psmatch2 Date Fri, 24 Jun 2011 15:07:28 +0200

```Hi again.

I dont know the number selected by attnd.
To test the ATT with caliper, I use attr. This is Radius/Caliper Matching.

another method would be the estimation with the starctification method. (.atts)

Now I have the problem with interpreting the different outcomes.

My new Propensity Score and the different outcomes are:

The balancing property is satisfied

This table shows the inferior bound, the number of treated
and the number of controls for each block

Inferior |
of block |      REL(Dummy)
of pscore  |         0          1 |     Total
-----------+----------------------+----------
.2 |         5          6 |        11
.4 |        16         21 |        37
.6 |        20         17 |        37
.7 |        14         47 |        61
.8 |        19        110 |       129
.9 |         9        223 |       232
-----------+----------------------+----------
Total |        83        424 |       507

. attnd stockprice2010 reldummy, pscore(mypscore)comsup

The program is searching the nearest neighbor of each treated unit.
This operation may take a while.

ATT estimation with Nearest Neighbor Matching method
(random draw version)
Analytical standard errors

---------------------------------------------------------
n. treat.   n. contr.         ATT   Std. Err.           t
---------------------------------------------------------

424          65      -0.572       6.828      -0.084

---------------------------------------------------------
Note: the numbers of treated and controls refer to actual
nearest neighbour matches

. attr stockprice2010 reldummy, pscore(mypscore) radius(0.1) comsup

The program is searching for matches of treated units within radius.
This operation may take a while.

ATT estimation with the Radius Matching method
Analytical standard errors

---------------------------------------------------------
n. treat.   n. contr.         ATT   Std. Err.           t
---------------------------------------------------------

424          83       1.354       3.588       0.377

---------------------------------------------------------
Note: the numbers of treated and controls refer to actual

. atts stockprice2010 reldummy, pscore(mypscore) comsup blockid(myblock)

ATT estimation with the Stratification method
Analytical standard errors

---------------------------------------------------------
n. treat.   n. contr.         ATT   Std. Err.           t
---------------------------------------------------------

424          83       0.273       2.520       0.109

---------------------------------------------------------

My Questions:

-How can I Interpret the different Outcomes?
-Why isnt the .pstest method working with these outcomes?
-Are these Outcomes my the final ones, or do I have to do some robustness tests as well? (If yes, which one?)

Thanks
Jason

--- Ursprüngliche Nachrsht ---
Datum: 21.06.2011 15:04:02
An: "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
Betreff: RE: st: RE: Ousout problem psmatch2

> Ah, yes. You are right - my mistake.
>
> Regarding -attnd- I see a few reasons why you can get different results:
>
> - using -psmatch2- you "restrict" to 10 neighbours. What is the
> number og nearest neighbours selected by -attnd-?
> - you set caliper to 0.1 using psmatch2. But I don't think a similar "restriction"
> is imposed in attnd?
>
> Maybe this is a start...
>
> /Jan
>
>
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu]
> On Behalf Of Jason Zarmulski
> Sent: 21. juni 2011 12:23
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: RE: Ousout problem psmatch2
>
> Hi Jan,
>
> thanks a lot for your support, I changed it and my new output is:
> :-)
>
> . psmatch2 reldummy , outcome( stockprice2010) pscore(mypscore) neighbor(10)
> caliper(0.1) ate
> ----------------------------------------------------------------------------------------
>
>         Variable     Sample |    Treated     Controls   Difference
>   S.E.   T-stat
> ----------------------------+-----------------------------------------------------------
>
>   stockprice2010  Unmatched | 20.6236969   21.7278571  -1.10416018   2.08916252
>    -0.53
>                         ATT | 20.6236969   20.4730466   .150650303   4.16715274
>     0.04
>                         ATU | 21.7278571   17.8625509  -3.86530621
>      .        .
>                         ATE |                          -.513405498
>      .        .
> ----------------------------+-----------------------------------------------------------
>
> Note: S.E. does not take into account that the propensity score is estimated.
>
>
>            | psmatch2:
>  psmatch2: |   Common
>  Treatment |  support
> assignment | On suppor |     Total
> -----------+-----------+----------
>  Untreated |        84 |        84
>    Treated |       424 |       424
> -----------+-----------+----------
>      Total |       508 |       508
>
>
> To the attnd problem : I didnt install a package for attnd. Maybe it was
>
> The differences between the psmatch2 and the attnd are still there....
>
> This is the help attnd file:
>
>
> Calculate the average treatment effect on the treated using nearest neighbor
> matching
>
>             attnd outcome treatment [varlist] [weight] [if exp] [in range]
> [ ,
>                   pscore(scorevar) logit index comsup detail bootstrap reps(#)
>
>                   noisily dots ]
>
>         fweights, iweights, and pweights are allowed; see help weights.
>
>
> Description
>
>     attnd estimates the average treatment effect on the treated (ATT) using
> nearest
>     neighbor matching. attnd should be run after the correct propensity score
>
>     specification; i.e., the one satisfying the balancing property has been
> found
>     using, for example, pscore. If users do not provide a variable name for
> the
>     propensity score, the propensity score is estimated based on the specification
>
>     in varlist. Note that in this case the balancing property is not tested.
>
>
>     It is left under the responsibility of the user to select the comsup
> option if
>     the user provided propensity score has been estimated on a common support
> for
>     treated and controls.  Otherwise, the ATT is estimated using also the
>
>     observations outside the common support for which the propensity score
> may not
>     be balanced.
>
>     To save on computing time, nearest neighbors are not determined by comparing
>
>     treated observations to every single control, but by first sorting all
> records
>     by the estimated propensity score and then searching forward and backward
> for
>     the closest control unit(s). If a treated unit forward and backward matches
>
>     happen to be equally good, this program randomly draws (hence the letters
> "nd"
>     for Nearest neighbor and random Draw) either the forward or backward
> matches.
>     This approach is one of two computationally feasible options to obtain
>
>     analytical standard errors while at the same time exploiting the very
> fast
>     forward and backward search strategy. The second possibility is based
> on giving
>     equal weight to the groups of forward and backward matches in case of
> equally
>     good forward and backward matches and is performed by attnw. In practice,
> the
>     case of multiple nearest neighbors should be very rare. In particular,
> if the
>     set of X's contains continuous variables, in which case, both attnd and
> attnw
>     should give equal results (except for bootstrapped standard errors).
> The
>     likelihood of multiple nearest neighbors is further reduced if the propensity
>
>     score is estimated and saved in double precision, which is what pscore
> does by
>     default.
>
>     The ATT is computed by averaging over the unit-level treatment effects
> of the
>     treated where the control(s) matched to a treated observation is/are
> those
>     observations in the control group that have the closest propensity score.
> If
>     there are multiple nearest neighbors, the average outcome of those controls
> is
>     used.
>
>
> Options
>
>     pscore(scorevar) specifies the name of the user-provided variable name
> for the
>         estimated propensity score. If no name is provided the propensity
> score is
>         estimated based on the specification in varlist.
>
>     logit uses a logit model to estimate the propensity score instead of
> the default
>         probit model when the option pscore(scorevar) is not specified by
> the user.
>         Otherwise, no effect is produced.
>
>     index requires the use of the linear index as the propensity score when
> the
>         option pscore(scorevar) is not specified by the user.  Otherwise,
> no effect
>         is produced.
>
>     comsup restricts the computation of the ATT to the region of common support.
>
>
>     detail displays more detailed output documenting the steps performed
> to obtain
>         the final results.
>
>     bootstrap bootstraps the standard error of the treatment effect.
>
>     reps(#) specifies the number of bootstrap replications to be performed.
>  The
>         default is 50.  This option produces an effect only if the bootstrap
> option
>         is specified.
>
>     noisily requests that any output from the replications be displayed.
>  This
>         option produces an effect only if the bootstrap option is specified.
>
>
>     dots requests that a dot be placed on the screen at the beginning of
> each
>         replication.  This option produces an effect only if the bootstrap
> option is
>         specified.
>
>
> Remarks
>
>     Please remember to use the update query command before running this program
> to
>     make sure you have an up-to-date version of Stata installed.  Otherwise,
> this
>     program may not run properly.
>
>     The treatment has to be binary.
>
>     When users do not specify their own previously estimated propensity score,
> the
>     bootstrap encompasses the estimation of the propensity score based on
> the
>     specification given by varlist. This procedure is actually recommended
> to
>     account for the uncertainty associated with the estimation of the propensity
>
>     score.  Even more so when the comsup option is specified because in this
> case
>     the region of common support changes with every bootstrap sample, and
>
>     bootstrapped standard errors pick up this uncertainty as well. So, typically
>
>     users would first identify a specification satisfying the balancing property
> --
>     using pscore -- and then provide exactly this specification in varlist
> and use
>     bootstrapped standard errors.
>
>
> Saved results
>
>     The program stores the estimated treatment effect, its standard error,
> and the t
>     statistic respectively in the scalars r(attnd), r(seattnd), and r(tsattnd).
>
>
>     The number of treated and the number of controls are stored respectively
> in the
>     scalars r(ntnd) and r(ncnd).
>
>     The bootstrapped standard error and t statistic are stored respectively
> in the
>     scalars r(bseattnd) and r(btsattnd).
>
>
> Examples
>
>     . attnd wage training age age2 exp exp2
>
>     . attnd wage training age age2 exp exp2, boot reps(100) dots
>
>     . attnd wage training age age2 exp exp2, logit boot reps(100)
>
>     . attnd wage training age age2 exp exp2, comsup boot reps(100)
>
>
> Authors
>
>         Sascha O. Becker
>         Center for Economic Studies, University of Munich
>
>         Andrea Ichino
>         Department of Economics, European University Institute, Florence
>
>
>     Email so.b@gmx.net or andrea.ichino@iue.it if you observe any problems.
>
>
>
> Acknowledgments
>
>     The way to implement the propensity score estimation in the bootstrap
> procedure
>     has been adapted from the psmatch program written by Barbara Sianesi
> (University
>     College London and Institute for Fiscal Studies) Email: barbara_s@ifs.org.uk.
>
>
>
> Also see
>
>     Online:  help for pscore, atts, attr, attk, attnw (if installed), and
> bs.
>
>               Further details on the analytical formulas and on the algorithms
> used
>               in these programs can be found under http://www.sobecker.de
> or
>               http://www.iue.it/Personal/Ichino.
>
> Thanks
> Jason
>
>
> --- Ursprüngliche Nachricht ---
> Datum: 20.06.2011 20:14:03
> An: "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
>
> Betreff: st: RE: Ousout problem psmatch2
>
> > Jason, your first question seems easy to solve: I think your treatment
> variable
> > and the outcome variable are identical. Recall the syntax for psmatch2,
> see
> > -help psmatch2-.
> >
> > The second point left me a bit confused. Searching for -attnd- using
> -findit
> > attnd- didn't really turn out any hints. Is -attnd- available from SSC?
> Maybe
> > you can clarify your steps there? Maybe differences are due to the issue
>
> > with the first question.
> >
> > Hope it helps
> > Jan Bryla
> >
> >
> > -----Original Message-----
> > From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu]
>
> > On Behalf Of Jason Zarmulski
> > Sent: 20. juni 2011 16:45
> > To: statalist@hsphsun2.harvard.edu
> > Subject: st: Ousout problem psmatch2
> >
> > Dear Statalists,
> > I got two problems, first one:
> >
> > I have a problem understanding the Output of my psmatch2 results. I'm
> new
> > to
> > this so it could be a trivial error, but I'm not sure. I wanted to do
> a
> > nearest neighbor matching with replacement.
> > My results are:
> >
> > . psmatch2 stockprice2010, outcome( stockprice2010) pscore(mypscore)
>
> > neighbor(10) caliper(0.1) ate
> > ----------------------------------------------------------------------------------------
>
> >
> >         Variable     Sample |    Treated     Controls   Difference
> > S.E.   T-stat
> > ----------------------------+-----------------------------------------------------------
>
> >
> >   stockprice2010  Unmatched |      1.628   .479999997    .99964329
> > .000849677  1176.50
> >                         ATT |       1.62   .689999992   .930000013
>
> > .399478823     2.33
> >                         ATU | .479999997   1.79500002   1.31500002
> > .        .
> >                         ATE |                           1.05833335
>
> > .        .
> > ----------------------------+-----------------------------------------------------------
>
> >
> > Note: S.E. does not take into account that the propensity score is
> > estimated.
> >
> >  psmatch2: |   psmatch2: Common
> >  Treatment |        support
> > assignment | Off suppo  On suppor |     Total
> > -----------+----------------------+----------
> >  Untreated |         0          2 |         2
> >    Treated |         1          4 |         5
> >          2 |         0         11 |        11
> >          3 |         0         12 |        12
> >          4 |         0         17 |        17
> >          5 |         0         34 |        34
> >          6 |         0         15 |        15
> >          7 |         0         19 |        19
> >          8 |         0         18 |        18
> >          9 |         0         17 |        17
> >         10 |         0         18 |        18
> >         11 |         0         13 |        13
> >         12 |         0         13 |        13
> >         13 |         0         20 |        20
> >         14 |         0         23 |        23
> >         15 |         0         23 |        23
> >         16 |         0         10 |        10
> >         17 |         0         13 |        13
> >         18 |         0         10 |        10
> >         19 |         0         11 |        11
> >         20 |         0         11 |        11
> >         21 |         0         10 |        10
> >         22 |         0         10 |        10
> >         23 |         0         11 |        11
> >         24 |         0         10 |        10
> >         25 |         0          7 |         7
> >         26 |         0          9 |         9
> >         27 |         0          6 |         6
> >         28 |         0          7 |         7
> >         29 |         0          4 |         4
> >         30 |         0          7 |         7
> >         31 |         0          6 |         6
> >         32 |         0          7 |         7
> >         33 |         0          7 |         7
> >         34 |         0          5 |         5
> >         35 |         0          6 |         6
> >         36 |         0          6 |         6
> >         37 |         0          6 |         6
> >         38 |         0          5 |         5
> >         39 |         0          4 |         4
> >         40 |         0          4 |         4
> >         41 |         0          9 |         9
> >         42 |         0          7 |         7
> >         43 |         0          5 |         5
> >         44 |         0          1 |         1
> >         45 |         0          1 |         1
> >         46 |         0          1 |         1
> >         47 |         0          1 |         1
> >         48 |         0          3 |         3
> >         49 |         0          4 |         4
> >         50 |         0          1 |         1
> >         51 |         0          1 |         1
> >         52 |         0          1 |         1
> >         53 |         0          1 |         1
> >         55 |         0          1 |         1
> >         56 |         0          3 |         3
> >         57 |         0          1 |         1
> >         59 |         0          1 |         1
> >         60 |         0          2 |         2
> >         62 |         0          1 |         1
> >         63 |         0          2 |         2
> >         69 |         0          1 |         1
> >         70 |         0          1 |         1
> >         78 |         0          1 |         1
> >         83 |         0          1 |         1
> >         84 |         0          2 |         2
> >         92 |         0          1 |         1
> > -----------+----------------------+----------
> >      Total |         1        505 |       506
> >
> > Question: Why arent there only the untreated and treated in the table
> and
> >
> > what ist the meaning of the numbers 2-92?
> >
> >
> > Second problem:
> >
> > I've done the nearest neighbor matching of the same data sample with
> the
> >
> > attnd function.
> > My results are:
> >
> > . attnd  stockprice2010 reldummy, pscore(mypscore)
> >
> >
> >  The program is searching the nearest neighbor of each treated unit.
>
> >  This operation may take a while.
> >
> >
> >
> > ATT estimation with Nearest Neighbor Matching method
> > (random draw version)
> > Analytical standard errors
> >
> > ---------------------------------------------------------
> > n. treat.   n. contr.         ATT   Std. Err.           t
> > ---------------------------------------------------------
> >
> >       424          63      -2.255       6.070      -0.371
> >
> > ---------------------------------------------------------
> > Note: the numbers of treated and controls refer to actual
> > nearest neighbour matches
> >
> > Why is this result for ATT so different to the one of psmatch2?
> > Is the attnd matching with replacement?
> >
> > Any suggestions would be much appreciated
> >
> > Thanks
> >
> > Jason
> >
> > --
> > View this message in context: http://statalist.1588530.n2.nabble.com/Ousout-problem-psmatch2-tp6496035p6496035.html
>
> >
> > Sent from the Statalist mailing list archive at Nabble.com.
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/help.cgi?search
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
> >
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/help.cgi?search
> > *   http://www.stata.com/support/statalist/faq
> > *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```