Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: criticisms of classical model selection methods


From   Maarten buis <[email protected]>
To   [email protected]
Subject   Re: st: criticisms of classical model selection methods
Date   Thu, 19 Aug 2010 11:39:04 +0000 (GMT)

--- On Thu, 19/8/10, Sam Brilleman wrote:
> Is anyone able to point me in the direction of references
> dealing with the discussion for and against traditional
> methods of model selection. I am particularly interested in
> criticisms of the assumptions on which commonly used model
> selection criteria are based (eg. AIC, BIC, etc).

Raftery, Adrian E. 1995. "Bayesian Model Selection in Social Research." Sociological Methodology 25:111–163.
And in particular the comments that follow this article

G. Claeskens and Nils Lid Hjort (2008) Model Selection and Model 
Averaging, Cambridge University Press

Burnham, K. P., and D. R. Anderson.  2002.  Model selection and 
multimodel inference: a practical information-theoretic approach. 
2nd Edition. Springer-Verlag, New York, New York.
 
> I'm also quite interested to hear what measures people
> often use when selecting between competing non-nested
> models, particularly when the models are non-linear. Is AIC
> reasonable when the models differ only by one covariate, or
> is it always worthwhile making judgement on a number of GOF
> tests?

This very much depends on what the aim of the model selection is:

If the comparison of models is done to test a specific hypothesis
then it is nice to stick to a single logic on which to base your
decision to choose one model over the other. All these logics are
flawed, so the best thing you can do is choose the one that is
most commonly used in your (sub-sub-)discipline.

If the comparison of models is done to find the "best" model, 
then you need to start with defining what "best" means. Almost
always you find that there is no way you can cover that with
one goodness of fit statistic, or even many of these statistics.

Hope this helps,
Maarten

--------------------------
Maarten L. Buis
Institut fuer Soziologie
Universitaet Tuebingen
Wilhelmstrasse 36
72074 Tuebingen
Germany

http://www.maartenbuis.nl
--------------------------


      

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index