[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: st: logit and mfx for randomized experiments
Maarten buis <firstname.lastname@example.org>
Re: st: logit and mfx for randomized experiments
Mon, 2 May 2011 17:59:50 +0100 (BST)
--- Daniel Schwartz wrote:
> 1. Is it more common to report this type of results just using
> As reporting the results using your first example of your
paper (page 2,
> stata tip 87).
> xi: logit y i.x1*x2 baseline, nocons or
That is really (sub-(sub-))discipline specific.
I work in a sub-discipline of Sociology (social stratification and social
mobility research) where the odds ratios are the norm. This makes sense given
the typical research question in this field. Typically they want to look at
the association between family background and educational and/or occupational
attainment of the offspring while controlling for structural changes like more
people continuing after high school or more people entering service
occupations in more recent cohorts. These structural changes are captured by
changes in the baseline odds, and they thus drop out of the odds ratios.
Marginal effects are contaminated by these structural changes and are thus not
appropriate for answering this question(*).
On the other hand, my impression of many branches of economics is that it tends
to regard odds ratios and ratios of odds ratios as too hard to understand, and
thus tend to focus more on the marginal effects. If your audience finds it hard
to understand your statistics than you need to take that seriously, even though
I do not think that this reputation is justified. One way of taking these
concerns seriously without using marginal effects is to just start with
explaining the baseline odds, move to the odds ratios, and than to ratios of
odds ratios. This does not take much extra space and it gradually and in a natural
way eases the audience into what your results mean.
> 2. At the end of your email, you indicate to use inteff, that in my case it
> would be:
> xi: inteff y i.x1*x2
> (I guess it's fine that doesn’t allow to add 'baseline' in this case)
> Should I use this s.e. and z-value for the interaction to report results,
> keeping the odds from the previous point?
No, you must also use the different interaction effects produced by -inteff-.
Typically this means that you will have many different interaction effects,
some significantly positive, some non-significant, and others significantly
negative, which will make it hard to draw meaningful conclusions. But that
is the price you must pay for using an effect size that is not "natural" for
It is easiest to see what is natural to a model by looking at what it does
to a continuous variable: If you add a continuous variable (x) to a linear
regression model it says that a unit change in x always leads to b units y
increase in the expected value of y, regardless of how much y you had to
begin with. So in a linear regression the natural way of looking at effects
is that we compare groups by computing the differences in expected value of
In logistic regression the effect of a continuous variable says that a unit
change in x always leads to a change in odds of success by a factor of
exp(b), regardless of how high the odds was to begin with. So in logistic
regression the natural way of looking at effects is that we compare groups
by computing the ratio of odds.
We can use "unnatural effect sizes" but than we must be willing to live
with the resulting complications.
> 3. Related to the previous points, in your paper you found that the
> wasn't significant (p = .16), but when you compute the
'margins', you find
> significant results. I understand that your
calculating different things here.
In my case the rather odd (pun intended) marginal effects in terms of odds
differences rather than the more conventional risk differences is actually
insignificant, as you can see here:
sysuse nlsw88, clear
gen byte high_occ = occupation < 3 if occupation < .
gen byte black = race == 2 if race < .
drop if race == 3
gen byte baseline = 1
logit high_occ black##collgrad baseline, or nocons
margins , over(black collgrad) expression(exp(xb())) post
lincom (0.black#1.collgrad - 0.black#0.collgrad) - ///
(1.black#1.collgrad - 1.black#0.collgrad)
The things I computed in my article where the effects of collgrad for blacks
and whites separately. These are, unsurprisingly, significant. To turn these
into an interaction effect you need to compute the difference (in this case not
a ratio because now we are talking marginal effects) between these effects as I
have done above.
(*) Within my discipline I have taken the position that we should not always
control for these structural changes. I did not propose to do away with the
odds ratio, but rather to explicitly study the link between odds ratios and
marginal effects as a way to study how structural changes and "real" changes
(the odds ratios) together influence the outcome, e.g.:
Maarten L. Buis
Institut fuer Soziologie
* For searches and help try: