Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: Area under the ROC curve less than 0.5?


From   <[email protected]>
To   <[email protected]>
Subject   st: Area under the ROC curve less than 0.5?
Date   Fri, 21 Jan 2011 17:56:04 +0800

Hi!

I just want to ask, after running a logistic regression on a development
sample, I ran the lroc command. The corresponding area under the ROC
curve is .9226. When I wished to check the robustness of the model on a
holdout sample, I keyed in "lroc if sample==2". The software was able to
run it on this separate sample, however, the resulting AROC is now
.4060. I thought AROC values are only from 0.5 to 1.0, or was I
mistaken?

Also, when I tried running roccomp on the two samples, Stata was again
able to churn out the same ROCs (.9226 and .4060), but the chi-square
p-value is .0663, indicating that at alpha=.05, the null hypothesis of
equal areas under the curves cannot be rejected.  Does it make sense
that even though the AROC of the second sample is less than half of the
first sample, the test still says that the 2 ROC curves have equal
areas? Is this test perhaps affected by the number of observations?
(sample 1 : 26 positive, 174 negative, while sample 2 contains 5(+) and
102(-)).

Would really appreciate anyone's help. Thanks in advance! 


- Reggae




*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index