Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
| From | "Jason Zarmulski" <Jason.Zarmulski@mtox.de> |
| To | statalist@hsphsun2.harvard.edu |
| Subject | Re: RE: st: RE: Ousout problem psmatch2 |
| Date | Fri, 24 Jun 2011 15:07:28 +0200 |
Hi again.
I dont know the number selected by attnd.
To test the ATT with caliper, I use attr. This is Radius/Caliper Matching.
another method would be the estimation with the starctification method. (.atts)
Now I have the problem with interpreting the different outcomes.
My new Propensity Score and the different outcomes are:
The balancing property is satisfied
This table shows the inferior bound, the number of treated
and the number of controls for each block
Inferior |
of block | REL(Dummy)
of pscore | 0 1 | Total
-----------+----------------------+----------
.2 | 5 6 | 11
.4 | 16 21 | 37
.6 | 20 17 | 37
.7 | 14 47 | 61
.8 | 19 110 | 129
.9 | 9 223 | 232
-----------+----------------------+----------
Total | 83 424 | 507
. attnd stockprice2010 reldummy, pscore(mypscore)comsup
The program is searching the nearest neighbor of each treated unit.
This operation may take a while.
ATT estimation with Nearest Neighbor Matching method
(random draw version)
Analytical standard errors
---------------------------------------------------------
n. treat. n. contr. ATT Std. Err. t
---------------------------------------------------------
424 65 -0.572 6.828 -0.084
---------------------------------------------------------
Note: the numbers of treated and controls refer to actual
nearest neighbour matches
. attr stockprice2010 reldummy, pscore(mypscore) radius(0.1) comsup
The program is searching for matches of treated units within radius.
This operation may take a while.
ATT estimation with the Radius Matching method
Analytical standard errors
---------------------------------------------------------
n. treat. n. contr. ATT Std. Err. t
---------------------------------------------------------
424 83 1.354 3.588 0.377
---------------------------------------------------------
Note: the numbers of treated and controls refer to actual
matches within radius
. atts stockprice2010 reldummy, pscore(mypscore) comsup blockid(myblock)
ATT estimation with the Stratification method
Analytical standard errors
---------------------------------------------------------
n. treat. n. contr. ATT Std. Err. t
---------------------------------------------------------
424 83 0.273 2.520 0.109
---------------------------------------------------------
My Questions:
-How can I Interpret the different Outcomes?
-Why isnt the .pstest method working with these outcomes?
-Are these Outcomes my the final ones, or do I have to do some robustness tests as well? (If yes, which one?)
Thanks
Jason
--- Ursprüngliche Nachrsht ---
Von: Jan Bryla <JBR@finansraadet.dk>
Datum: 21.06.2011 15:04:02
An: "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
Betreff: RE: st: RE: Ousout problem psmatch2
> Ah, yes. You are right - my mistake.
>
> Regarding -attnd- I see a few reasons why you can get different results:
>
> - using -psmatch2- you "restrict" to 10 neighbours. What is the
> number og nearest neighbours selected by -attnd-?
> - you set caliper to 0.1 using psmatch2. But I don't think a similar "restriction"
> is imposed in attnd?
>
> Maybe this is a start...
>
> /Jan
>
>
> -----Original Message-----
> From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu]
> On Behalf Of Jason Zarmulski
> Sent: 21. juni 2011 12:23
> To: statalist@hsphsun2.harvard.edu
> Subject: Re: st: RE: Ousout problem psmatch2
>
> Hi Jan,
>
> thanks a lot for your support, I changed it and my new output is:
> :-)
>
> . psmatch2 reldummy , outcome( stockprice2010) pscore(mypscore) neighbor(10)
> caliper(0.1) ate
> ----------------------------------------------------------------------------------------
>
> Variable Sample | Treated Controls Difference
> S.E. T-stat
> ----------------------------+-----------------------------------------------------------
>
> stockprice2010 Unmatched | 20.6236969 21.7278571 -1.10416018 2.08916252
> -0.53
> ATT | 20.6236969 20.4730466 .150650303 4.16715274
> 0.04
> ATU | 21.7278571 17.8625509 -3.86530621
> . .
> ATE | -.513405498
> . .
> ----------------------------+-----------------------------------------------------------
>
> Note: S.E. does not take into account that the propensity score is estimated.
>
>
> | psmatch2:
> psmatch2: | Common
> Treatment | support
> assignment | On suppor | Total
> -----------+-----------+----------
> Untreated | 84 | 84
> Treated | 424 | 424
> -----------+-----------+----------
> Total | 508 | 508
>
>
> To the attnd problem : I didnt install a package for attnd. Maybe it was
> with the"nnmatch ado" package
>
> The differences between the psmatch2 and the attnd are still there....
>
> This is the help attnd file:
>
>
> Calculate the average treatment effect on the treated using nearest neighbor
> matching
>
> attnd outcome treatment [varlist] [weight] [if exp] [in range]
> [ ,
> pscore(scorevar) logit index comsup detail bootstrap reps(#)
>
> noisily dots ]
>
> fweights, iweights, and pweights are allowed; see help weights.
>
>
> Description
>
> attnd estimates the average treatment effect on the treated (ATT) using
> nearest
> neighbor matching. attnd should be run after the correct propensity score
>
> specification; i.e., the one satisfying the balancing property has been
> found
> using, for example, pscore. If users do not provide a variable name for
> the
> propensity score, the propensity score is estimated based on the specification
>
> in varlist. Note that in this case the balancing property is not tested.
>
>
> It is left under the responsibility of the user to select the comsup
> option if
> the user provided propensity score has been estimated on a common support
> for
> treated and controls. Otherwise, the ATT is estimated using also the
>
> observations outside the common support for which the propensity score
> may not
> be balanced.
>
> To save on computing time, nearest neighbors are not determined by comparing
>
> treated observations to every single control, but by first sorting all
> records
> by the estimated propensity score and then searching forward and backward
> for
> the closest control unit(s). If a treated unit forward and backward matches
>
> happen to be equally good, this program randomly draws (hence the letters
> "nd"
> for Nearest neighbor and random Draw) either the forward or backward
> matches.
> This approach is one of two computationally feasible options to obtain
>
> analytical standard errors while at the same time exploiting the very
> fast
> forward and backward search strategy. The second possibility is based
> on giving
> equal weight to the groups of forward and backward matches in case of
> equally
> good forward and backward matches and is performed by attnw. In practice,
> the
> case of multiple nearest neighbors should be very rare. In particular,
> if the
> set of X's contains continuous variables, in which case, both attnd and
> attnw
> should give equal results (except for bootstrapped standard errors).
> The
> likelihood of multiple nearest neighbors is further reduced if the propensity
>
> score is estimated and saved in double precision, which is what pscore
> does by
> default.
>
> The ATT is computed by averaging over the unit-level treatment effects
> of the
> treated where the control(s) matched to a treated observation is/are
> those
> observations in the control group that have the closest propensity score.
> If
> there are multiple nearest neighbors, the average outcome of those controls
> is
> used.
>
>
> Options
>
> pscore(scorevar) specifies the name of the user-provided variable name
> for the
> estimated propensity score. If no name is provided the propensity
> score is
> estimated based on the specification in varlist.
>
> logit uses a logit model to estimate the propensity score instead of
> the default
> probit model when the option pscore(scorevar) is not specified by
> the user.
> Otherwise, no effect is produced.
>
> index requires the use of the linear index as the propensity score when
> the
> option pscore(scorevar) is not specified by the user. Otherwise,
> no effect
> is produced.
>
> comsup restricts the computation of the ATT to the region of common support.
>
>
> detail displays more detailed output documenting the steps performed
> to obtain
> the final results.
>
> bootstrap bootstraps the standard error of the treatment effect.
>
> reps(#) specifies the number of bootstrap replications to be performed.
> The
> default is 50. This option produces an effect only if the bootstrap
> option
> is specified.
>
> noisily requests that any output from the replications be displayed.
> This
> option produces an effect only if the bootstrap option is specified.
>
>
> dots requests that a dot be placed on the screen at the beginning of
> each
> replication. This option produces an effect only if the bootstrap
> option is
> specified.
>
>
> Remarks
>
> Please remember to use the update query command before running this program
> to
> make sure you have an up-to-date version of Stata installed. Otherwise,
> this
> program may not run properly.
>
> The treatment has to be binary.
>
> When users do not specify their own previously estimated propensity score,
> the
> bootstrap encompasses the estimation of the propensity score based on
> the
> specification given by varlist. This procedure is actually recommended
> to
> account for the uncertainty associated with the estimation of the propensity
>
> score. Even more so when the comsup option is specified because in this
> case
> the region of common support changes with every bootstrap sample, and
>
> bootstrapped standard errors pick up this uncertainty as well. So, typically
>
> users would first identify a specification satisfying the balancing property
> --
> using pscore -- and then provide exactly this specification in varlist
> and use
> bootstrapped standard errors.
>
>
> Saved results
>
> The program stores the estimated treatment effect, its standard error,
> and the t
> statistic respectively in the scalars r(attnd), r(seattnd), and r(tsattnd).
>
>
> The number of treated and the number of controls are stored respectively
> in the
> scalars r(ntnd) and r(ncnd).
>
> The bootstrapped standard error and t statistic are stored respectively
> in the
> scalars r(bseattnd) and r(btsattnd).
>
>
> Examples
>
> . attnd wage training age age2 exp exp2
>
> . attnd wage training age age2 exp exp2, boot reps(100) dots
>
> . attnd wage training age age2 exp exp2, logit boot reps(100)
>
> . attnd wage training age age2 exp exp2, comsup boot reps(100)
>
>
> Authors
>
> Sascha O. Becker
> Center for Economic Studies, University of Munich
>
> Andrea Ichino
> Department of Economics, European University Institute, Florence
>
>
> Email so.b@gmx.net or andrea.ichino@iue.it if you observe any problems.
>
>
>
> Acknowledgments
>
> The way to implement the propensity score estimation in the bootstrap
> procedure
> has been adapted from the psmatch program written by Barbara Sianesi
> (University
> College London and Institute for Fiscal Studies) Email: barbara_s@ifs.org.uk.
>
>
>
> Also see
>
> Online: help for pscore, atts, attr, attk, attnw (if installed), and
> bs.
>
> Further details on the analytical formulas and on the algorithms
> used
> in these programs can be found under http://www.sobecker.de
> or
> http://www.iue.it/Personal/Ichino.
>
> Thanks
> Jason
>
>
> --- Ursprüngliche Nachricht ---
> Von: Jan Bryla <JBR@finansraadet.dk>
> Datum: 20.06.2011 20:14:03
> An: "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
>
> Betreff: st: RE: Ousout problem psmatch2
>
> > Jason, your first question seems easy to solve: I think your treatment
> variable
> > and the outcome variable are identical. Recall the syntax for psmatch2,
> see
> > -help psmatch2-.
> >
> > The second point left me a bit confused. Searching for -attnd- using
> -findit
> > attnd- didn't really turn out any hints. Is -attnd- available from SSC?
> Maybe
> > you can clarify your steps there? Maybe differences are due to the issue
>
> > with the first question.
> >
> > Hope it helps
> > Jan Bryla
> >
> >
> > -----Original Message-----
> > From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu]
>
> > On Behalf Of Jason Zarmulski
> > Sent: 20. juni 2011 16:45
> > To: statalist@hsphsun2.harvard.edu
> > Subject: st: Ousout problem psmatch2
> >
> > Dear Statalists,
> > I got two problems, first one:
> >
> > I have a problem understanding the Output of my psmatch2 results. I'm
> new
> > to
> > this so it could be a trivial error, but I'm not sure. I wanted to do
> a
> > nearest neighbor matching with replacement.
> > My results are:
> >
> > . psmatch2 stockprice2010, outcome( stockprice2010) pscore(mypscore)
>
> > neighbor(10) caliper(0.1) ate
> > ----------------------------------------------------------------------------------------
>
> >
> > Variable Sample | Treated Controls Difference
> > S.E. T-stat
> > ----------------------------+-----------------------------------------------------------
>
> >
> > stockprice2010 Unmatched | 1.628 .479999997 .99964329
> > .000849677 1176.50
> > ATT | 1.62 .689999992 .930000013
>
> > .399478823 2.33
> > ATU | .479999997 1.79500002 1.31500002
> > . .
> > ATE | 1.05833335
>
> > . .
> > ----------------------------+-----------------------------------------------------------
>
> >
> > Note: S.E. does not take into account that the propensity score is
> > estimated.
> >
> > psmatch2: | psmatch2: Common
> > Treatment | support
> > assignment | Off suppo On suppor | Total
> > -----------+----------------------+----------
> > Untreated | 0 2 | 2
> > Treated | 1 4 | 5
> > 2 | 0 11 | 11
> > 3 | 0 12 | 12
> > 4 | 0 17 | 17
> > 5 | 0 34 | 34
> > 6 | 0 15 | 15
> > 7 | 0 19 | 19
> > 8 | 0 18 | 18
> > 9 | 0 17 | 17
> > 10 | 0 18 | 18
> > 11 | 0 13 | 13
> > 12 | 0 13 | 13
> > 13 | 0 20 | 20
> > 14 | 0 23 | 23
> > 15 | 0 23 | 23
> > 16 | 0 10 | 10
> > 17 | 0 13 | 13
> > 18 | 0 10 | 10
> > 19 | 0 11 | 11
> > 20 | 0 11 | 11
> > 21 | 0 10 | 10
> > 22 | 0 10 | 10
> > 23 | 0 11 | 11
> > 24 | 0 10 | 10
> > 25 | 0 7 | 7
> > 26 | 0 9 | 9
> > 27 | 0 6 | 6
> > 28 | 0 7 | 7
> > 29 | 0 4 | 4
> > 30 | 0 7 | 7
> > 31 | 0 6 | 6
> > 32 | 0 7 | 7
> > 33 | 0 7 | 7
> > 34 | 0 5 | 5
> > 35 | 0 6 | 6
> > 36 | 0 6 | 6
> > 37 | 0 6 | 6
> > 38 | 0 5 | 5
> > 39 | 0 4 | 4
> > 40 | 0 4 | 4
> > 41 | 0 9 | 9
> > 42 | 0 7 | 7
> > 43 | 0 5 | 5
> > 44 | 0 1 | 1
> > 45 | 0 1 | 1
> > 46 | 0 1 | 1
> > 47 | 0 1 | 1
> > 48 | 0 3 | 3
> > 49 | 0 4 | 4
> > 50 | 0 1 | 1
> > 51 | 0 1 | 1
> > 52 | 0 1 | 1
> > 53 | 0 1 | 1
> > 55 | 0 1 | 1
> > 56 | 0 3 | 3
> > 57 | 0 1 | 1
> > 59 | 0 1 | 1
> > 60 | 0 2 | 2
> > 62 | 0 1 | 1
> > 63 | 0 2 | 2
> > 69 | 0 1 | 1
> > 70 | 0 1 | 1
> > 78 | 0 1 | 1
> > 83 | 0 1 | 1
> > 84 | 0 2 | 2
> > 92 | 0 1 | 1
> > -----------+----------------------+----------
> > Total | 1 505 | 506
> >
> > Question: Why arent there only the untreated and treated in the table
> and
> >
> > what ist the meaning of the numbers 2-92?
> >
> >
> > Second problem:
> >
> > I've done the nearest neighbor matching of the same data sample with
> the
> >
> > attnd function.
> > My results are:
> >
> > . attnd stockprice2010 reldummy, pscore(mypscore)
> >
> >
> > The program is searching the nearest neighbor of each treated unit.
>
> > This operation may take a while.
> >
> >
> >
> > ATT estimation with Nearest Neighbor Matching method
> > (random draw version)
> > Analytical standard errors
> >
> > ---------------------------------------------------------
> > n. treat. n. contr. ATT Std. Err. t
> > ---------------------------------------------------------
> >
> > 424 63 -2.255 6.070 -0.371
> >
> > ---------------------------------------------------------
> > Note: the numbers of treated and controls refer to actual
> > nearest neighbour matches
> >
> > Why is this result for ATT so different to the one of psmatch2?
> > Is the attnd matching with replacement?
> >
> > Any suggestions would be much appreciated
> >
> > Thanks
> >
> > Jason
> >
> > --
> > View this message in context: http://statalist.1588530.n2.nabble.com/Ousout-problem-psmatch2-tp6496035p6496035.html
>
> >
> > Sent from the Statalist mailing list archive at Nabble.com.
> > *
> > * For searches and help try:
> > * http://www.stata.com/help.cgi?search
> > * http://www.stata.com/support/statalist/faq
> > * http://www.ats.ucla.edu/stat/stata/
> >
> > *
> > * For searches and help try:
> > * http://www.stata.com/help.cgi?search
> > * http://www.stata.com/support/statalist/faq
> > * http://www.ats.ucla.edu/stat/stata/
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/