Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: rbounds Hodges-Lehmann point estimates and ATT estimates


From   Richard Palmer-Jones <rpjstatalist@googlemail.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: rbounds Hodges-Lehmann point estimates and ATT estimates
Date   Fri, 2 Jul 2010 09:15:21 +0100

Steve (and any others interested)

Reading about sensitivity analysis of PSM estimates suggests that it
is a common view that the Hodges-Lehmann point estimates of difference
which are based on medians are  expected to be smaller that
differences in means which is what the AT is based on. Are you
convinced? Of course we know that medians are less sensitive to
outliers, but why whould that make differences in medians between
treatments smaller?

I have noted that the HL estimates vary considerably with the
algorithm used to compute pscores even though it does not appear that
there are big differences in the actual matches, or the ATTs. Again,
an insight into this would be appreciated.

Richard


On Thu, Jul 1, 2010 at 10:39 PM, Steve Samuels <sjsamuels@gmail.com> wrote:
> Quite right, Richard.  I should really read the command documentation!
> The  difference you note does not appear to be a difference of
> consequence, because  the CI's for ATT from -psmatch2- and
> from-rbounds- overlap substantially and are very wide.
>
> Steve
>
>
>
>
> On Thu, Jul 1, 2010 at 4:52 PM, Richard Palmer-Jones
> <rpjstatalist@googlemail.com> wrote:
>> Thanks, Steve
>>
>> But, unfortunately this is not the problem. he ATT is calculated for
>> the treatment observations - 185, and these are used in the rbounds
>> estimate of the Hodges-Lehman point estimates.
>>
>> Any other ideas?
>>
>> Richard
>> .
>>
>>
>>
>>
>> On Thu, Jul 1, 2010 at 5:45 PM, Steve Samuels <sjsamuels@gmail.com> wrote:
>>> The N's are different.  -psmatch2- has n= 2675.  -bounds reports  185
>>> matched pairs.
>>> Your "diff" variable was set to missing for 2490 observations.
>>>
>>> Steve
>>>
>>> On Thu, Jul 1, 2010 at 12:06 PM, Richard Palmer-Jones
>>> <rpjstatalist@googlemail.com> wrote:
>>>> Dear Readers
>>>>
>>>> When  I run rbounds after psmatch2 I find that the Hodges-Lehman
>>>> minimum and maximum point estimates of impact are (generally)
>>>> substantially different to the estimated ATT when the Gamma =1. Could
>>>> someone explain this?
>>>>
>>>> . use lalonde.dta
>>>>
>>>>
>>>>
>>>> . psmatch2 t age age2 educ educ2 black hisp marr re74 u74 re74 re75 ,
>>>> outcome(re78)
>>>> note: re74 dropped because of collinearity
>>>>
>>>> Probit regression                                 Number of obs   =       2675
>>>>                                                  LR chi2(10)     =     882.99
>>>>                                                  Prob > chi2     =     0.0000
>>>> Log likelihood =  -231.1534                       Pseudo R2       =     0.6564
>>>>
>>>> ------------------------------------------------------------------------------
>>>>           t |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
>>>> -------------+----------------------------------------------------------------
>>>>         age |    .165835   .0597947     2.77   0.006     .0486395    .2830305
>>>>        age2 |  -.0031243   .0009159    -3.41   0.001    -.0049195    -.001329
>>>>        educ |   .4034187   .1637615     2.46   0.014      .082452    .7243854
>>>>       educ2 |  -.0233423   .0081187    -2.88   0.004    -.0392546   -.0074299
>>>>       black |   1.179666   .1692016     6.97   0.000     .8480369    1.511295
>>>>        hisp |   1.200164   .3092023     3.88   0.000     .5941388     1.80619
>>>>        marr |  -1.000713   .1456276    -6.87   0.000    -1.286138    -.715288
>>>>        re74 |  -.0000509   .0000141    -3.62   0.000    -.0000784   -.0000233
>>>>         u74 |    .326188   .1842919     1.77   0.077    -.0350175    .6873935
>>>>        re75 |   -.000103   .0000205    -5.03   0.000    -.0001431   -.0000629
>>>>       _cons |  -4.082649   1.221929    -3.34   0.001    -6.477585   -1.687713
>>>> ------------------------------------------------------------------------------
>>>> Note: 659 failures and 0 successes completely determined.
>>>> There are observations with identical propensity score values.
>>>> The sort order of the data could affect your results.
>>>> Make sure that the sort order is random before calling psmatch2.
>>>> ----------------------------------------------------------------------------------------
>>>>        Variable     Sample |    Treated     Controls   Difference
>>>>    S.E.   T-stat
>>>> ----------------------------+-----------------------------------------------------------
>>>>            re78  Unmatched | 6349.14537   21553.9213  -15204.7759
>>>> 1154.61435   -13.17
>>>>                        ATT | 6349.14537   5387.78028   961.365096
>>>> 1420.27513     0.68
>>>> ----------------------------+-----------------------------------------------------------
>>>> Note: S.E. for ATT does not take into account that the propensity
>>>> score is estimated.
>>>>
>>>>           | psmatch2:
>>>>  psmatch2: |   Common
>>>>  Treatment |  support
>>>> assignment | On suppor |     Total
>>>> -----------+-----------+----------
>>>>  Untreated |     2,490 |     2,490
>>>>   Treated |       185 |       185
>>>> -----------+-----------+----------
>>>>     Total |     2,675 |     2,675
>>>>
>>>>
>>>> . gen diff =  re78- _re78
>>>> (2490 missing values generated)
>>>>
>>>> . rbounds diff, gamma(1(.2)3)
>>>>
>>>> Rosenbaum bounds for diff (N = 185 matched pairs)
>>>>
>>>> Gamma           sig+      sig-    t-hat+    t-hat-       CI+       CI-
>>>> ----------------------------------------------------------------------
>>>>    1        .203558   .203558    498.13    498.13  -618.815   1671.48
>>>>  1.2        .594682   .028425  -109.352   1108.27  -1261.98   2361.57
>>>>  1.4        .873746   .002358   -622.61   1680.15  -1795.44   2936.95
>>>>  1.6        .973397   .000138   -1113.6   2160.85  -2305.18   3440.96
>>>>  1.8        .995808   6.3e-06   -1507.8    2567.3  -2722.57   3920.69
>>>>    2        .999467   2.4e-07  -1811.98    2961.7  -3098.75    4317.7
>>>>  2.2        .999942   8.3e-09  -2171.23      3265  -3463.81   4741.49
>>>>  2.4        .999995   2.6e-10  -2410.04   3622.82  -3749.03   5057.69
>>>>  2.6              1   7.3e-12  -2717.38   3919.67  -4063.75   5380.66
>>>>  2.8              1   2.0e-13  -2960.24      4167  -4308.32   5657.37
>>>>    3              1   5.1e-15  -3195.96   4413.54  -4566.85    5982.9
>>>>
>>>> * gamma  - log odds of differential assignment due to unobserved factors
>>>>  sig+   - upper bound significance level
>>>>  sig-   - lower bound significance level
>>>>  t-hat+ - upper bound Hodges-Lehmann point estimate
>>>>  t-hat- - lower bound Hodges-Lehmann point estimate
>>>>  CI+    - upper bound confidence interval (a=  .95)
>>>>  CI-    - lower bound confidence interval (a=  .95)
>>>>
>>>> .
>>>> end of do-file
>>>>
>>>> My problem is:
>>>>
>>>> ATT = 961
>>>> Gammat-hat+    t-hat-
>>>>  1       498.13    498.13
>>>> *
>>>> *   For searches and help try:
>>>> *   http://www.stata.com/help.cgi?search
>>>> *   http://www.stata.com/support/statalist/faq
>>>> *   http://www.ats.ucla.edu/stat/stata/
>>>>
>>>
>>>
>>>
>>> --
>>> Steven Samuels
>>> sjsamuels@gmail.com
>>> 18 Cantine's Island
>>> Saugerties NY 12477
>>> USA
>>> Voice: 845-246-0774
>>> Fax:    206-202-4783
>>>
>>> *
>>> *   For searches and help try:
>>> *   http://www.stata.com/help.cgi?search
>>> *   http://www.stata.com/support/statalist/faq
>>> *   http://www.ats.ucla.edu/stat/stata/
>>>
>>
>> *
>> *   For searches and help try:
>> *   http://www.stata.com/help.cgi?search
>> *   http://www.stata.com/support/statalist/faq
>> *   http://www.ats.ucla.edu/stat/stata/
>>
>
>
>
> --
> Steven Samuels
> sjsamuels@gmail.com
> 18 Cantine's Island
> Saugerties NY 12477
> USA
> Voice: 845-246-0774
> Fax:    206-202-4783
>
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index