Re: st: logit and mfx for randomized experiments

[Maarten buis <maartenbuis@yahoo.co.uk>]

--- Daniel Schwartz wrote:
> 1. Is it more common to report this type of results just using 
odds-ratio?
> As reporting the results using your first example of your 
paper (page 2,
> stata tip 87). 
> 
> xi: logit y i.x1*x2 baseline, nocons or


That is really (sub-(sub-))discipline specific. 


I work in a sub-discipline of Sociology (social stratification and social 

mobility research) where the odds ratios are the norm. This makes sense given 

the typical research question in this field. Typically they want to look at 

the association between family background and educational and/or occupational 

attainment of the offspring while controlling for structural changes like more 

people continuing after high school or more people entering service 

occupations in more recent cohorts. These structural changes are captured by 

changes in the baseline odds, and they thus drop out of the odds ratios. 

Marginal effects are contaminated by these structural changes and are thus not 

appropriate for answering this question(*).

On the other hand, my impression of many branches of economics is that it tends
to regard odds ratios and ratios of odds ratios as too hard to understand, and
thus tend to focus more on the marginal effects. If your audience finds it hard
to understand your statistics than you need to take that seriously, even though
I do not think that this reputation is justified. One way of taking these 

concerns seriously without using marginal effects is to just start with 

explaining the baseline odds, move to the odds ratios, and than to ratios of 

odds ratios. This does not take much extra space and it gradually and in a natural
way eases the audience into what your results mean.


> 2. At the end of your email, you indicate to use inteff, that in my case it
> would be:
> 
> xi: inteff y i.x1*x2
> (I guess it's fine that doesn’t allow to add 'baseline' in this case)


Yes


> Should I use this s.e. and z-value for the interaction to report results, 
> keeping the odds from the previous point?

No, you must also use the different interaction effects produced by -inteff-.
Typically this means that you will have many different interaction effects, 

some significantly positive, some non-significant, and others significantly 

negative, which will make it hard to draw meaningful conclusions. But that
is the price you must pay for using an effect size that is not "natural" for 

that model. 


It is easiest to see what is natural to a model by looking at what it does 

to a continuous variable: If you add a continuous variable (x) to a linear 

regression model it says that a unit change in x always leads to b units y 

increase in the expected value of y, regardless of how much y you had to 

begin with. So in a linear regression the natural way of looking at effects 

is that we compare groups by computing the differences in expected value of 

y. 


In logistic regression the effect of a continuous variable says that a unit 

change in x always leads to a change in odds of success by a factor of 

exp(b), regardless of how high the odds was to begin with. So in logistic
regression the natural way of looking at effects is that we compare groups
by computing the ratio of odds. 


We can use "unnatural effect sizes" but than we must be willing to live
with the resulting complications.


> 3. Related to the previous points, in your paper you found that the 
interaction 

> wasn't significant (p = .16), but when you compute the 
'margins', you find
> significant results. I understand that your 
calculating different things here. 


In my case the rather odd (pun intended) marginal effects in terms of odds
differences rather than the more conventional risk differences is actually
insignificant, as you can see here:


sysuse nlsw88, clear
gen byte high_occ = occupation < 3 if occupation < .
gen byte black = race == 2 if race < .
drop if race == 3
gen byte baseline = 1
logit high_occ black##collgrad baseline, or nocons 
margins , over(black collgrad) expression(exp(xb())) post
lincom (0.black#1.collgrad - 0.black#0.collgrad) - ///
       (1.black#1.collgrad - 1.black#0.collgrad)


The things I computed in my article where the effects of collgrad for blacks
and whites separately. These are, unsurprisingly, significant. To turn these
into an interaction effect you need to compute the difference (in this case not
a ratio because now we are talking marginal effects) between these effects as I
have done above.


-- Maarten

(*) Within my discipline I have taken the position that we should not always 
control for these structural changes. I did not propose to do away with the
odds ratio, but rather to explicitly study the link between odds ratios and
marginal effects as a way to study how structural changes and "real" changes
(the odds ratios) together influence the outcome, e.g.:
<http://www.maartenbuis.nl/dissertation/chap_6.pdf>
 
--------------------------
Maarten L. Buis
Institut fuer Soziologie
Universitaet Tuebingen
Wilhelmstrasse 36
72074 Tuebingen
Germany


http://www.maartenbuis.nl
--------------------------


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/