What's this about?
Treatment-effects models extract experimental-style causal effects from
In experimental data, treatment groups must be assigned randomly,
meaning characteristics across groups will be approximately equal.
Treatment-effects estimators reweight the observational data
in hopes of achieving experimental-like
balanced data results.
If the reweighting is successful, then
the weighted distribution of each covariate should be the same
across treatment groups. In such cases, we say that the
treatment model "balanced" the covariates.
We can examine whether the treatment model balanced the covariates
and perform a statistical test.
Three diagnostics and one test are provided.
One diagnostic reports, for each covariate, the model-adjusted
difference in means in the treatment groups and the ratio of
Another diagnostic graphs the model-adjusted
estimated pdfs of covariates; these pdfs can be examined
visually to verify that they are approximately equal.
The third graphical diagnostic is the same as the second but
uses box plots rather than smoothed pdfs.
The statistical test is an overidentification test.
It tests whether the model-adjusted means of the
covariates are the same between groups.
Let's see it work
Say that we estimate the effect of smoking during pregnancy on infant
birthweight using an inverse-probability-weighted (IPW) treatment-effects
We assume that treatment (smoking during pregnancy) is determined by
marital status, the mother's age, attendance to prenatal care during
the first quarter of pregnancy, and whether this is the mother's first
. teffects ipw (bweight) (mbsmoke mmarried mage prenatal1 fbaby
c.mage#(c.mage i.mmarried prenatal1))
Iteration 0: EE criterion = 9.365e-20
Iteration 1: EE criterion = 3.141e-26
Treatment-effects estimation Number of obs = 4,642
Estimator : inverse-probability weights
Outcome model : weighted mean
Treatment model: logit
| || || Robust|
| bweight || Coef. Std. Err. z P>|z| [95% Conf. Interval]|
| mbsmoke |
|(smoker vs nonsmoker) || -239.6875 26.43427 -9.07 0.000 -291.4977 -187.8773|
| mbsmoke |
| nonsmoker || 3403.638 9.56792 355.73 0.000 3384.885 3422.39|
We find that the average treatment effect (ATE) is -240 grams.
Have we done an adequate job of balancing the covariates so that we can trust
the estimated treatment effect?
We could use the overidentification test:
. tebalance overid, nolog
Overidentification test for covariate balance
H0: Covariates are balanced:
chi2(8) = 11.8612
Prob > chi2 = 0.1575
We cannot reject the null hypothesis that the covariates are balanced,
and that's good.
We can look at the various diagnostics (and in real life, we probably would
have used the diagnostics before using the statistical test).
tebalance summarize reports the model-adjusted difference in means and
ratio of variances between the treated and untreated for each covariate:
. tebalance summarize
Covariate balance summary
| || |
| ||Number of obs = 4,642 4,642.0|
| ||Treated obs = 864 2,329.1|
| ||Control obs = 3,778 2,312.9|
| || ||Standardized differences Variance ratio|
| || Raw Weighted Raw Weighted|
| mmarried || -.5953009 .0053497 1.335944 .9953184|
| mage || -.300179 .0410889 .8818025 1.076571|
| prenatal1 || -.3242695 .0009807 1.496155 .9985165|
| fbaby || -.1663271 -.0130638 .9430944 .9965406|
| mage || -.3028275 .0477465 .8274389 1.109134|
| mage |
| married || -.6329701 .0197209 1.157026 1.034108|
| mage |
| Yes || -.4053969 .0182109 1.226363 1.032561|
Ignore the raw columns, at least to begin, and focus on the weighted
columns. Differences in weighted means are negligible, and variance
ratios are all near one. The Raw columns show where we started, and,
before weighting, differences were large.
tebalance can show us pdfs or box plots so that we can examine the
entire distribution. Below we have put the graphs produced
by tebalance density and tebalance box together:
Tests and diagnostics confirm that our model balances the covariates.
Tell me more
To find out more about checking for balance after teffects or stteffects, see [TE] tebalance.
Read the overview from the Stata News.