Re: st: fit index for ordered logistic regression

[Darcy Hannibal <dlhannibal@ucdavis.edu>]

While I prefer the Bayesian approach, if you haven't used it before, 
this might be more than you want to tackle for a manuscript that is 
pretty much done. You could just respond to the comment by the reviewer 
and to say you are not using Bayesian methodology. To do this right, you 
typically need many variables to build a useful model, not just a few 
variables to test a specific hypothesis.

On 12/14/2013 2:13 PM, Lucas wrote:
> David Hoaglin makes a good point.  In addition, you might look at:
>
> Raftery, Adrian E.  1995.  "Bayesian Model Selection in Social
> Research." *Sociological Methodology* 25: 111-163.
>
> He has a nice discussion of what you might regard as cut-offs for a
> meaningful difference between models.  At the risk of undermining
> incentive to read Raftery's very informative paper, check out, for
> example, Tables 8 and 9 on page 141. The text further discusses what
> counts as weak, strong, and very strong evidence in a comparison of
> models.
>
> Take care
> Sam
>
> On Sat, Dec 14, 2013 at 2:03 PM, David Hoaglin <dchoaglin@gmail.com> wrote:
> > Hi, Julie.
> >
> > I'm not sure what you mean by "imbedded."
> >
> > AIC is not an absolute measure.  If you have a set of models that
> > should be good models on substantive grounds, you can choose the model
> > that has the smallest AIC.  Those models do not need to be nested, and
> > they do not have to be based on the same family of distributions (as
> > long as the calculation of AIC includes all the constants in the
> > likelihood).  The usual definition of AIC applies to large samples.
> > If the ratio of the sample size (n) to the total number of parameters
> > (K) is not "large," say n/K < 40, it is better to use a modified
> > version of AIC.
> >
> > Are you able to use deviance (and DIC) to compare each of your models
> > against the corresponding "saturated" model?
> >
> > Also, Agresti (2010) discusses ways of assessing goodness of fit.
> >
> > David Hoaglin
> >
> > Agresti, A. (2010).  Analysis of Ordinal Categorical Data, 2nd ed.
> > John Wiley & Sons.
> >
> >
> > On Sat, Dec 14, 2013 at 2:48 PM, Julie Lamoureux <drjuliel@comcast.net> wrote:
> > > Good day,
> > >
> > > I am looking for "normed" goodness of fit indices for ordered logistic
> > > regression.  We submitted a manuscript with the results of 2 OLR (on two
> > > different unrelated outcomes) and the reviewer is asking how we assessed
> > > model goodness of fit and its usefulness.  I know AIC and BIC let us compare
> > > models that are "imbedded" but can we use AIC and BIC to determine how
> > > useful the model is?  If so, what "cut-off" for those indices can be
> > > considered "good"?  Is there something else I am not aware of to answer this
> > > question?
> > >
> > > Thank you for your time
> > >
> > > Julie Lamoureux
> > *
> > *   For searches and help try:
> > *   http://www.stata.com/help.cgi?search
> > *   http://www.stata.com/support/faqs/resources/statalist-faq/
> > *   http://www.ats.ucla.edu/stat/stata/
> *
> *   For searches and help try:
> *   http://www.stata.com/help.cgi?search
> *   http://www.stata.com/support/faqs/resources/statalist-faq/
> *   http://www.ats.ucla.edu/stat/stata/


---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/