Ben,
Thank you very much for thinking about this.
I have been trying to figure out how my index relates to the between-model
difference in the area under the ROC curves. The difference in AUCs are
notorious for sometimes being very small even when otherwise seemingly
important predictors are added to a model. I know that one interpretation
of the AUC is as the probability that a subject with the outcome of
interest will be ranked higher than a subject without it, but I’m having a
hard time figuring out whether there is a direct transformation between AUC
differences and the index that I propose, which would offer a quantitative
measure of the number (or proportion) of patients correctly reassigned with
the new model.
Dan
***********
From: Chapman, Ben P [mailto:Ben_Chapman@URMC.Rochester.edu]
Sent: Saturday, September 29, 2007 9:39 AM
To: dan@amplecat.com
Subject: net proportion of subjects with improved prediction
Hi Dan--not sure if this will be helpful or not; I've run into a similar
issue of trying to demonstrate that added variable(s) enhance overall
predictive accuracy in logit models.
This analysis at the UCLA web site shows a very interesting method
(apparently published in the stata journal) for demonstrating that added
predictor(s) enhance AUC--if you consider improvements in sensitivity v.
1-specificity as a net or overall measure of improved prediction--not sure
if it is exactly what you're looking for but thought I'd point it out...
http://www.ats.ucla.edu/stat/stata/faq/roc.htm
Best,
Ben Chapman
Benjamin Chapman, PhD
NRSA Postdoctoral Fellow
MPH Candidate
University of Rochester School of Medicine and Dentistry
Department of Psychiatry
Laboratory of Personality and Development
Rochester Center for Mind Body Research
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.5.488 / Virus Database: 269.13.33/1036 - Release Date: 9/28/2007
3:40 PM
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.5.488 / Virus Database: 269.13.33/1036 - Release Date: 9/28/2007
3:40 PM
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
