Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
"Ariel Linden, DrPH" <ariel.linden@gmail.com> |

To |
<statalist@hsphsun2.harvard.edu> |

Subject |
RE: Re: st: problem with the interpretation of pstest after psmatch2, t-tests and percentage of bias provide conflicting results, which one should I follow? |

Date |
Mon, 14 Jan 2013 10:03:16 -0500 |

Simone, Adam gave you good advice and pointed you to a good article that discusses why you shouldn't use t-tests for testing covariate balance. Rather than re-asking the same question, perhaps you should read the article and learn why t-tests are not a good measure of balance (hint: we are not making inferences, and sample size can affect the results), whereas standardized differences, variance ratios, and a whole host of graphic plots (q-q plots, box plots, kernel density plots, etc.) are more suitable. You will likely not be asking this question again, once you've done your homework. Additional suggested reading (a few among hundreds of articles): Caliendo M, Kopeinig S. Some practical guidance for the implementation of propensity score matching. Journal of Economic Surveys 2008;22:31- Stuart EA. Matching methods for causal inference: a review and a look forward. Statistical Science 2010;25(1):1-21. Flury BK, Reidwyl H. Standard distance in univariate and multivariate analysis. The American Statistician 1986;40:249-251 Rubin DB. Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services & Outcomes Research Methodology 2001;2:169-188. Iacus SM, King G, Porro G. Multivariate matching methods that are monotonic imbalance bounding. Journal of the American Statistical Association 2011;106:345-361. Ariel Date: Sun, 13 Jan 2013 12:21:06 +0100 From: simone ferro <simone.ferro88@gmail.com> Subject: Re: st: problem with the interpretation of pstest after psmatch2, t-tests and percentage of bias provide conflicting results, which one should I follow? Thank you so much for your answer, it was very clear and helpful, I followed your suggestion and I got better results, so thank you very much, by the way I'm using a kernel matching based on a probit regression, variables are mainly continuous. Now I get a better balance, but as expected t-tests refuse the null-hypotesis of equal means, and chi2 test refuse the null hypotesis of balance as well, looking at the means it seems to me that this is a good balance, do you think I can simply don't look at t-tests and chi2 test? - --------------------------------------------------------- | Mean | t-test Variable | Treated Control %bias | t p>|t| - -------------+--------------------------+---------------- cost_per_e~l | 60.95 60.053 4.4 | -3.48 0.001 asset | 193.35 260.33 -5.3 | -1.41 0.159 employees | 398.82 551.06 -7.9 | -0.97 0.333 av_age | 54.424 54.951 -6.9 | 1.69 0.092 donne_perc | .29114 .28214 4.3 | 3.86 0.000 laureati_p~c | .23952 .23494 1.9 | 0.11 0.914 bluecollar~c | .35553 .37312 -5.8 | -2.94 0.004 occupaz_min | .16463 .13253 9.1 | -0.13 0.895 donne_perc | .29114 .28214 4.3 | 3.86 0.000 - ---------------------------------------------------------- Pseudo R2 LR chi2 p>chi2 MeanB MedB - ---------------------------------------------------------- 0.163 86.83 0.000 5.6 5.3 - ---------------------------------------------------------- thank you again, Regards, Simone Il giorno 13/gen/2013, alle ore 05:18, Adam Olszewski <adam.olszewski@gmail.com> ha scritto: > Hi Simone, > t-test based comparisons after PS matching are highly controversial. > t-test makes a lot of usually untenable assumptions (are the variables > normally distributed?), and moreover is too sensitive to sample size. > A non-significant test might just mean a small sample, while a > minuscule difference in means might be "significant" in a very large > sample. > Standardized differences of means seem to be more accepted, although > for continuous covariates comparing actual distributions may be most > persuasive. > From your message it is not clear what type of matching you used and > whether all the variables are continuous or some are categorical. If > you question the test statistics calculation, you can run the t-test > manually and check. While I'm not knowledgeable about practices in > sociology and economics, you may want to look at SDM's rather than the > pstest results. See e.g.: > Austin, P. C. (2009). "Balance diagnostics for comparing the > distribution of baseline covariates between treatment groups in > propensity-score matched samples." Statistics in Medicine 28(25): > 3083-3107. > > Good luck, > Adam Olszewski * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/faqs/resources/statalist-faq/ * http://www.ats.ucla.edu/stat/stata/

- Prev by Date:
**Re: st: RE: Fixed Effects estimation with time-invariant variables** - Next by Date:
**Re: st: Regression Discontinuity Designs with rd package and binary outcome** - Previous by thread:
**Re: st: problem with the interpretation of pstest after psmatch2, t-tests and percentage of bias provide conflicting results, which one should I follow?** - Next by thread:
**st: Mixed effects model with zero-inflated negative binomial outcome for repeated measures data** - Index(es):