Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

RE: st: Testing for program effectiveness with heckman


From   Maarten buis <[email protected]>
To   [email protected]
Subject   RE: st: Testing for program effectiveness with heckman
Date   Sat, 31 May 2008 09:37:40 +0100 (BST)

--- Austin Nichols wrote:
> The position you espouse that "group comparisons are, if interpreted
> correctly as the difference between groups, likely to be much less
> biased than so called causal models" seems very strange to me.  If
you
> observe the treatment group having much better outcomes than the
> control, but treatment is nonrandomly assigned, to what correct
> interpretation of difference between groups would you assign the
> difference between groups? 

The whole point is that I would not assign it to anything, a difference
between groups is just that. Which is why the bias is less: it aims at
a much less ambitious goal. There is almost always a tradeoff between
how right you want to be and how useful you want to be. With these
group comparison models you are closer to the being right part of that
spectrum and further way from the being useful part. The causal models
typically use assumptions that we know to be wrong, e.g. variable z
influences selection into treatment group but not the outcome (IV),
whatever is unobserved is uncorrelated with the observed variabels
(random effects), or whatever is unobserved is constant over time/space
(fixed effects). So when using these methods we move away from the
being right part towards the being useful part of the spectrum. It is
very rare that you can be both right and useful, and I find too many
people that treat method X as a magic bullet and are unaware that they
are making this tradeoff.

> The position you espouse that "group comparisons are, if interpreted
> correctly as the difference between groups, likely to be much less
> biased than so called causal models" seems very strange to me.  If
you
> observe the treatment group having much better outcomes than the
> control, but treatment is nonrandomly assigned, to what correct
> interpretation of difference between groups would you assign the
> difference between groups? 

I may not have been clear: the models I described as measuring
differences between groups I include models that control for measured
confounders. 

-- Maarten

-----------------------------------------
Maarten L. Buis
Department of Social Research Methodology
Vrije Universiteit Amsterdam
Boelelaan 1081
1081 HV Amsterdam
The Netherlands

visiting address:
Buitenveldertselaan 3 (Metropolitan), room Z434

+31 20 5986715

http://home.fsw.vu.nl/m.buis/
-----------------------------------------


      __________________________________________________________
Sent from Yahoo! Mail.
A Smarter Email http://uk.docs.yahoo.com/nowyoucan.html
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2024 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index