Home  /  Stata News  /  Vol 30 No 3 (2015 quarter 3)  /  Treatment effects
The Stata News
  Previous   Home   Next

In the spotlight: Treatment effects

A delicate balancing act

Treatment-effects modeling is a fundamental tool to obtain experimental-style causal effects from observational data. Ideally, we would conduct an experiment, but for ethical or financial reasons, an experiment sometimes is not feasible.

A good example is the effect of cigarette smoking (the treatment) on the birthweight of infants (the outcome). In an experiment, we would first obtain a representative sample of pregnant women. Then, some would be told not to smoke (the control group), while others would be forced to smoke an arbitrary number of cigarettes per day (the treatment group). Clearly, such an experiment is unethical and would not be allowed. However, we can still answer our question of interest using Stata’s suite of parametric, semiparametric, and nonparametric treatment-effects estimators.

Suppose we want to tackle this question using teffects. For our estimates to be trustworthy, we have to guarantee that once we control for observable characteristics, it is as if pregnant mothers had been randomly assigned to control and treatment groups.

In an experiment, it is easy to inspect whether the characteristics of the treatment and control groups are equivalent. We simply need to look at the data as observed. For instance, the mothers in both groups should have the same age and level of education on average, and if we plotted the density of both groups, they should look the same.

However, this is not the case with observational data. Instead, we inspect whether our treatment-effects model reweights the data in such a way that the model-adjusted distribution of the mothers’ characteristics is equivalent across groups.

The balancing act in action

We model the birthweight (bweight) as a function of the number of prenatal visits (nprenatal), whether the mother is married (mmarried), and whether this baby is her first pregnancy (fbaby). The treatment, smoking during pregnancy (mbsmoke), is modeled as a function of the same variables and with regard to whether the mother consumed alcohol during her pregnancy. We type

. webuse cattaneo2, clear 
(Excerpt from Cattaneo (2010) Journal of Econometrics 155: 138-154)

. teffects ipwra (bweight nprenatal i.mmarried i.fbaby) 
                 (mbsmoke i.mmarried i.alcohol i.fbaby nprenatal)

We do not show the output, but suffice it to say that the effect of smoking is large and decidedly significant.

To obtain balancing diagnostics of the averages and variances of the mothers’ characteristics across groups, we type

. tebalance summarize
  Covariate balance summary
Raw Weighted
Number of obs = 4,642 4,642.0
Treated obs = 864 2,318.7
Control obs = 3,778 2,323.3
Standardized differences Variance ratio
Raw Weighted Raw Weighted
married -.5953009 -.0002835 1.335944 1.000247
1 .3222725 -.0031106 4.509207 .9838918
Yes -.1663271 .0131381 .9430944 1.003143
nprenatal -.2837987 -.0154989 1.430129 1.044148

The values in the Raw columns show that without controlling for covariates, the groups are very different. The values in the Weighted columns show the differences in means and the ratio of the variances of the control and treatment groups after reweighting for the covariates. The mean differences are all near zero, and the variance ratios are all close to one. These diagnostics suggest that after we control for the covariates, it is as if we had randomly assigned the mothers to either the control group or the treatment group.

We can also inspect this graphically by plotting the distribution before fitting our model and the distribution after weighting. We do this for the number of prenatal visits.

. tebalance density nprenatal


The density graphs confirm what we observe from our diagnostics.

Can we do a test?

What we have described so far is qualitative: we have diagnostics but not a formal test. We can, however, do a test. Intuitively, the score equations for the treatment and control groups should be the same. We can test whether this is the case by using the score equations as moments in an overidentification test. The null hypothesis is that our covariates are balanced. We type

. tebalance overid
Overidentification test for covariate balance
         H0: Covariates are balanced:

         chi2(5)      =   4.0425
         Prob > chi2  =   0.5433

We cannot reject the null hypothesis. This implies that there is no evidence that our covariates remain imbalanced after reweighting.

Parting words

Sometimes, we cannot conduct experiments, but we can obtain experimental-style causal effects from observational data. For this to happen, we need to be able to say that our treatment-effects model reweights the data in such a way that the model-adjusted distribution of the covariates is equivalent across treatment groups. We can verify this with the postestimation diagnostic tests provided in teffects.

—Enrique Pinzon
Senior Econometrician, StataCorp