Re: st: criticisms of classical model selection methods

[Maarten buis <maartenbuis@yahoo.co.uk>]

--- On Thu, 19/8/10, Sam Brilleman wrote:
> Is anyone able to point me in the direction of references
> dealing with the discussion for and against traditional
> methods of model selection. I am particularly interested in
> criticisms of the assumptions on which commonly used model
> selection criteria are based (eg. AIC, BIC, etc).

Andrew Gelman just posted on his blog on a closely related issue.
<http://www.stat.columbia.edu/~cook/movabletype/archives/2010/08/the_eitheror_fa.html>

I would have put this slightly differently: there may or may
not be a true world out there (I'll leave that question to the
philosophers), but the purpose of a model and model selection
is _not_ to find that true world, put to find a simplified 
version of it that helps us find the answer to a specific 
research question. 

As a consequence we need to have an idea of what the key parts
are that we need to get right in our model in order to answer 
our question, and check those. This really depends on the 
question, there can be no universaly correct checklist/cookbook
we can work through in order to get the "correct/scientific"
model. A generic measure, like goodnes of fit, just distracts 
us from the main issue, which is to answer our research 
question based on stuff we have seen, heard, felt, or otherwise
observed.

-- Maarten

--------------------------
Maarten L. Buis
Institut fuer Soziologie
Universitaet Tuebingen
Wilhelmstrasse 36
72074 Tuebingen
Germany

http://www.maartenbuis.nl
--------------------------


      

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/