Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: deriving a bootstrap estimate of a difference between two weighted regressions
From
Steve Samuels <[email protected]>
To
[email protected]
Subject
Re: st: deriving a bootstrap estimate of a difference between two weighted regressions
Date
Mon, 9 Aug 2010 22:14:42 -0400
The difference I saw in my last email was due to the small number of
bootstrap replicates I had run for my test cases When I upped the
number of replications to 200 the standard errors that included the
logistic regression were about 19% higher than the standard errors
that did not for ATE and ATT, and about the same for ATC. This is
still important, I think.
The lack of difference for ATC makes sense, as the weights based on
the propensity scores affected only the 2% of the sample in the
"treated" group.
Steve
On Mon, Aug 9, 2010 at 9:39 PM, Steve Samuels <[email protected]> wrote:
> To finish off this topic, perhaps:
>
> I had occasion recently to examine the causal association between a
> continuous outcome and a two category "treatment" variable. The
> "treated" group constituted about 2% of the sample. The method of
> analysis was OLS, and propensity scores had been estimated by logistic
> regression. I computed bootstrap standard errors that included and
> excluded the logistic step.
>
> The result: standard errors for average treatment effects ATE, ATT,
> and ATC that included the logistic regression were roughly double(!)
> the standard errors that did not.
>
> Steve
>
>
>
> On Wed, Aug 4, 2010 at 1:07 PM, Ariel Linden, DrPH
> <[email protected]> wrote:
>> Thank you, Stas. I will take your suggestions under advisement!
>>
>>
>>
>> Date: Tue, 3 Aug 2010 14:37:51 -0500
>> From: Stas Kolenikov <[email protected]>
>> Subject: Re: st: deriving a bootstrap estimate of a difference between two
>> weighted regressions
>>
>> On Tue, Aug 3, 2010 at 1:20 PM, Ariel Linden, DrPH <[email protected]>
>> wrote:
>>> Thank you Stas and Steve for your comments!
>>>
>>> When I stated that the first model's weight would be ATT and the next
>>> ATC, it was already after running the propensity score model and
>>> establishing
>> the
>>> weights for each subject:
>>> ATT = cond(treatvar, 1, propvar/(1- propvar)), and ATC =
>>> cond(treatvar, (1-propvar)/propvar, 1)
>>>
>>> Under these conditions, there should be no negative weights, so that
>>> is
>> not
>>> a concern.
>>
>> The negative weights would come out of Steve's suggestion to entertain the
>> difference in weights (as that's what your procedure boils down to).
>>
>>> I am thinking that the code would look something like this, but I
>>> would appreciate your input:
>>>
>>> 1. bootstrap _b[treatvar] from first regression with [pw=ATT] 2. save
>>> 10,000 samples to file (or tempfile) 3. bootstrap _b[treatvar] from
>>> second regression with [pw=ATC] 4. save 10,000 samples to file (or
>>> tempfile) 5. gen difference = treatvar1-treatvar2 6. bootstrap
>>> r(mean): sum difference, to get bootstrapped CIs
>>>
>>> Does this make sense?
>>
>> 1-2 will produce something very similar to the _se[treatvar] in your basic
>> regression with ATT weights (probably with -robust- option), and
>> 3-4 will produce something very similar to _se[treatvar] in the regression
>> with ATC weights. I outlined the code for you in the previous message -- you
>> need to bootstrap the whole estimation procedure = { the propensity
>> regression (leading to the weights) + two main regressions with two sets of
>> weights }. In other words, for each bootstrap sample, you would need to run
>> everything in the curly brackets to produce your "difference" estimate.
>>
>> I cannot comment on the scientific validity of this procedure; other people
>> more knowledgeable in treatment effect estimation could do that.
>>
>>
>>
>> *
>> * For searches and help try:
>> * http://www.stata.com/help.cgi?search
>> * http://www.stata.com/support/statalist/faq
>> * http://www.ats.ucla.edu/stat/stata/
>>
>
>
>
> --
> Steven Samuels
> [email protected]
> 18 Cantine's Island
> Saugerties NY 12477
> USA
> Voice: 845-246-0774
> Fax: 206-202-4783
>
--
Steven Samuels
[email protected]
18 Cantine's Island
Saugerties NY 12477
USA
Voice: 845-246-0774
Fax: 206-202-4783
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/