Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: criticisms of classical model selection methods


From   Maarten buis <maartenbuis@yahoo.co.uk>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: criticisms of classical model selection methods
Date   Thu, 19 Aug 2010 11:39:04 +0000 (GMT)

--- On Thu, 19/8/10, Sam Brilleman wrote:
> Is anyone able to point me in the direction of references
> dealing with the discussion for and against traditional
> methods of model selection. I am particularly interested in
> criticisms of the assumptions on which commonly used model
> selection criteria are based (eg. AIC, BIC, etc).

Raftery, Adrian E. 1995. "Bayesian Model Selection in Social Research." Sociological Methodology 25:111–163.
And in particular the comments that follow this article

G. Claeskens and Nils Lid Hjort (2008) Model Selection and Model 
Averaging, Cambridge University Press

Burnham, K. P., and D. R. Anderson.  2002.  Model selection and 
multimodel inference: a practical information-theoretic approach. 
2nd Edition. Springer-Verlag, New York, New York.
 
> I'm also quite interested to hear what measures people
> often use when selecting between competing non-nested
> models, particularly when the models are non-linear. Is AIC
> reasonable when the models differ only by one covariate, or
> is it always worthwhile making judgement on a number of GOF
> tests?

This very much depends on what the aim of the model selection is:

If the comparison of models is done to test a specific hypothesis
then it is nice to stick to a single logic on which to base your
decision to choose one model over the other. All these logics are
flawed, so the best thing you can do is choose the one that is
most commonly used in your (sub-sub-)discipline.

If the comparison of models is done to find the "best" model, 
then you need to start with defining what "best" means. Almost
always you find that there is no way you can cover that with
one goodness of fit statistic, or even many of these statistics.

Hope this helps,
Maarten

--------------------------
Maarten L. Buis
Institut fuer Soziologie
Universitaet Tuebingen
Wilhelmstrasse 36
72074 Tuebingen
Germany

http://www.maartenbuis.nl
--------------------------


      

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index