Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.
From | Steve Samuels <sjsamuels@gmail.com> |
To | statalist@hsphsun2.harvard.edu |
Subject | Re: st: deriving a bootstrap estimate of a difference between two weighted regressions |
Date | Mon, 9 Aug 2010 22:14:42 -0400 |
The difference I saw in my last email was due to the small number of bootstrap replicates I had run for my test cases When I upped the number of replications to 200 the standard errors that included the logistic regression were about 19% higher than the standard errors that did not for ATE and ATT, and about the same for ATC. This is still important, I think. The lack of difference for ATC makes sense, as the weights based on the propensity scores affected only the 2% of the sample in the "treated" group. Steve On Mon, Aug 9, 2010 at 9:39 PM, Steve Samuels <sjsamuels@gmail.com> wrote: > To finish off this topic, perhaps: > > I had occasion recently to examine the causal association between a > continuous outcome and a two category "treatment" variable. The > "treated" group constituted about 2% of the sample. The method of > analysis was OLS, and propensity scores had been estimated by logistic > regression. I computed bootstrap standard errors that included and > excluded the logistic step. > > The result: standard errors for average treatment effects ATE, ATT, > and ATC that included the logistic regression were roughly double(!) > the standard errors that did not. > > Steve > > > > On Wed, Aug 4, 2010 at 1:07 PM, Ariel Linden, DrPH > <ariel.linden@gmail.com> wrote: >> Thank you, Stas. I will take your suggestions under advisement! >> >> >> >> Date: Tue, 3 Aug 2010 14:37:51 -0500 >> From: Stas Kolenikov <skolenik@gmail.com> >> Subject: Re: st: deriving a bootstrap estimate of a difference between two >> weighted regressions >> >> On Tue, Aug 3, 2010 at 1:20 PM, Ariel Linden, DrPH <ariel.linden@gmail.com> >> wrote: >>> Thank you Stas and Steve for your comments! >>> >>> When I stated that the first model's weight would be ATT and the next >>> ATC, it was already after running the propensity score model and >>> establishing >> the >>> weights for each subject: >>> ATT = cond(treatvar, 1, propvar/(1- propvar)), and ATC = >>> cond(treatvar, (1-propvar)/propvar, 1) >>> >>> Under these conditions, there should be no negative weights, so that >>> is >> not >>> a concern. >> >> The negative weights would come out of Steve's suggestion to entertain the >> difference in weights (as that's what your procedure boils down to). >> >>> I am thinking that the code would look something like this, but I >>> would appreciate your input: >>> >>> 1. bootstrap _b[treatvar] from first regression with [pw=ATT] 2. save >>> 10,000 samples to file (or tempfile) 3. bootstrap _b[treatvar] from >>> second regression with [pw=ATC] 4. save 10,000 samples to file (or >>> tempfile) 5. gen difference = treatvar1-treatvar2 6. bootstrap >>> r(mean): sum difference, to get bootstrapped CIs >>> >>> Does this make sense? >> >> 1-2 will produce something very similar to the _se[treatvar] in your basic >> regression with ATT weights (probably with -robust- option), and >> 3-4 will produce something very similar to _se[treatvar] in the regression >> with ATC weights. I outlined the code for you in the previous message -- you >> need to bootstrap the whole estimation procedure = { the propensity >> regression (leading to the weights) + two main regressions with two sets of >> weights }. In other words, for each bootstrap sample, you would need to run >> everything in the curly brackets to produce your "difference" estimate. >> >> I cannot comment on the scientific validity of this procedure; other people >> more knowledgeable in treatment effect estimation could do that. >> >> >> >> * >> * For searches and help try: >> * http://www.stata.com/help.cgi?search >> * http://www.stata.com/support/statalist/faq >> * http://www.ats.ucla.edu/stat/stata/ >> > > > > -- > Steven Samuels > sjsamuels@gmail.com > 18 Cantine's Island > Saugerties NY 12477 > USA > Voice: 845-246-0774 > Fax: 206-202-4783 > -- Steven Samuels sjsamuels@gmail.com 18 Cantine's Island Saugerties NY 12477 USA Voice: 845-246-0774 Fax: 206-202-4783 * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/