Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: st: Hosmer-Lemeshow and other Pseudo Rsquares


From   "Lachenbruch, Peter" <[email protected]>
To   "[email protected]" <[email protected]>
Subject   RE: st: Hosmer-Lemeshow and other Pseudo Rsquares
Date   Mon, 14 May 2012 07:26:58 -0700

I just had reason to check Frank Harrell's book (REgression MOdeling Strategies, Springer 2001) on variable selection and his advice was fit the full model with variables taht are defined by their contextual (biological, sociological) importance.  Don't believe any R^2 or other values as they are distorted by the selection process. 

More recently, the LASSO (Tibshirani - don't have reference, but google scholar can provide one) or LARS (Efron, Hastie) should be helpful.

Tony

________________________________________
From: [email protected] [[email protected]] On Behalf Of Nick Cox [[email protected]]
Sent: Monday, May 14, 2012 7:10 AM
To: [email protected]
Subject: Re: st: Hosmer-Lemeshow and other Pseudo Rsquares

I suggest a few meta-rules for yourself:

1. Whatever you calculate should be defined and calculated
consistently across different models.

2. Whatever you calculate you promise to use with extreme caution
always flagging precisely how it is calculated.

3. You don't decide which model is "best" from these measures; you
just treat them as descriptive statistics.

#1 sounds easy but can bite quite hard. I find the idea of R^2 as

square of correlation between observed and predicted

as the sense of R^2 that I like best but this grows out of a long
personal history of working with correlation and regression and one
that is dominated by working with continuous outcomes. People with a
long history the other way round might want you to look for

1 - (log likelihood for model) / (log likelihood for same model with
only a constant term)

and could have similar warm feelings for that. Others would find the
whole idea of looking at goodness of fit without also assessing number
of parameters or model complexity in general quite misguided, but
those others can't agree on which of various *IC you should use, and
even those who have a favourite often say, "You should use ?IC except
that it usually favours over-simplified models" or some such.

On the first option see

FAQ     . . . . . . . . . . . . . . . . . . . . . . . Do-it-yourself R-squared
        . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  N. J. Cox
        9/03    How can I get an R-squared value when a Stata command
                does not supply one?
                http://www.stata.com/support/faqs/stat/rsquared.html

On Mon, May 14, 2012 at 2:47 PM, Joseph Padgett <[email protected]> wrote:

> I am working with a data set where students are nested within school.
>
> I have completed a thorough run of models starting with nulls and
> ending with full fixed- and random-effects with all controls and
> predictors and several models in between with various combinations of
> controls.  My dependent variable is a binary outcome.
>
> I have Haussman tests, LR tests, and Wald taken care of, but I would
> like to report some goodness-of-fit results for my models.  I am aware
> of the Hosmer-Lemeshow test statistic and it's interpretation, but I'm
> having a difficult time finding out how to compute it from my model
> results.  I would also like to consider alternatives such as Cox and
> Snell.
>
> I have run my models with each of xtlogit, xtmelogit, and gllamm.  I
> did this mostly to be able to learn a bit about the post estimation
> commands and different options with each command.  That being said, I
> don't know how to get the pseudo Rsquare measures after any of these
> and most explanations that I find refer only to the logit command and
> give examples using very simplistic models.
>
> I'm fairly certain there's something terribly obvious that I'm
> overlooking.  Any help would be greatly appreciated.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index