Notice: On March 31, it was **announced** that Statalist is moving from an email list to a **forum**. The old list will shut down on April 23, and its replacement, **statalist.org** is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
"Colette.Grey" <Colette.Grey@ul.ie> |

To |
<statalist@hsphsun2.harvard.edu> |

Subject |
RE: st: RE: psmatch2 and p-value |

Date |
Thu, 2 Dec 2010 13:27:10 -0000 |

Thanks a million Ariel for such a comprehensive respone, I really appreciate the help and guidance. I ran the pscore command in Stata and it said 'The balancing property is satisfied'. I did notice a question on this list about how many covariates one should use. I get the above even after dropping some of the covariates. In contrast, I have read the more covariates the better. When calculating the _pscore, I used psmatch2 in Stata and generated a variable ps2 to be equal to _pscore so from then on, I used the following: psmatch2 TARG, outcome (RAW) pscore (ps2) logit neighbor(1) noreplacement I totally understand your concerns regarding the methodology, that is my next task, I was trying first to come to terms with the commands and outputs in Stata. Again thanks, Colette -----Original Message----- From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] On Behalf Of Ariel Linden, DrPH Sent: 01 December 2010 18:11 To: statalist@hsphsun2.harvard.edu Subject: Re:st: RE: psmatch2 and p-value Hi Collette, First, to the tables: The first line indicates what the unmatched/unadjusted values look like (ie., comparison of all the treated to all of the untreated). Thus the t-stat indicates this is not significant (t-stat of 1.96 equals p-value of 0.05, so you are looking for t-stat values equal to or greater than 1.96). The second line shows the average treatment effect on the treated: here, the 140 matched treated are compared to the 140 matched untreated. Here the t-stat is even lower (-0.97), suggesting there is no statstically significant difference in RAW between matched treated and controls. The table below that indicates that there were 140 matched pairs, and 293 treated subjects for which controls could not be found (because they were beyond the common support range). Conceptual issues: What I don't see here is what covariates you used to generate the pscore, or if balance was achieved on those covariates? This is perhaps the most important aspect of observational studies, and should not be disregarded. The next issue (and my biggest area of concern) is that you are running this algorthm for each year and then adding up the scores in the end. I am not sure I agree with that. These are the same firms, and so there is going to be repeated measurements on the same firms over time. Your methodology does not account for the longitudinal nature of the data and the auto-correlation that would be expected within firm. A more appropriate methodology would be to generate propensity score based weights for the baseline period (preintervention) for each group in the dataset, and then use those weights within a longitudinal model (ie., GEE with the appropriate family and link). These are not trivial issues, and I suggest you consult a statistician or read the appropriate literature on these topics (longitudinal data, causal inference from observational data, etc.). I hope this helps Ariel Date: Tue, 30 Nov 2010 16:12:35 -0000 From: "Colette.Grey" <Colette.Grey@ul.ie> Subject: st: RE: psmatch2 and p-value Hi Ariel, I appreciate you taking time to help me with these queries, I am learning and making some progress I hope. I have two questions about the output from psmatch2. I have data for three years so I want to estimate the propensity score annually and match the firms annually and am using psmatch2 for this. Then I use the results from the three years together to asses the ATT. Yes, I do have more treatment than control firms. I am starting by just looking at the matched firms. (1) What I have is a result below where RAW is the outcome of interest and TARG is the treatment: . psmatch2 TARG, outcome (RAW) pscore (ps2) logit neighbor(1) noreplacement - ---------------------------------------------------------------------------- ------------ Variable Sample | Treated Controls Difference S.E. T-stat - ----------------------------+------------------------------------------- ----------------------------+---- ------------ RAWAWCA Unmatched | .005222726 .015996705 -.010773979 .006727021 -1.60 ATT | .007248377 .015996705 -.008748328 .009040694 -0.97 - ----------------------------+------------------------------------------- ----------------------------+---- ------------ Note: S.E. does not take into account that the propensity score is estimated. What I don't know form the above is whether or not the T-stat is significant, I need a p-value. Do you know please how I can get a p-value? (2) The below is also printed when I run the above command: psmatch2: | psmatch2: Common Treatment | support assignment | Off suppo On suppor | Total - -----------+----------------------+---------- Untreated | 0 140 | 140 Treated | 293 140 | 433 - -----------+----------------------+---------- Total | 293 280 | 573 My question is if the above is calculated on the 280 matched firms or the 573 firms? How do I tell Stata which I want, assuming I am able to do so please? Any help would be great, thanks, Colette * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/ * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**Re: st: RE: psmatch2 and p-value***From:*"Dimitriy V. Masterov" <dvmaster@gmail.com>

**References**:**Re:st: RE: psmatch2 and p-value***From:*"Ariel Linden, DrPH" <ariel.linden@gmail.com>

- Prev by Date:
**Re: st: Predicting number of cases** - Next by Date:
**st: .ado and .plugin** - Previous by thread:
**Re:st: RE: psmatch2 and p-value** - Next by thread:
**Re: st: RE: psmatch2 and p-value** - Index(es):