st: Area under the ROC curve less than 0.5?

[<reggae.benigno@metrobank.com.ph>]

Hi!

I just want to ask, after running a logistic regression on a development
sample, I ran the lroc command. The corresponding area under the ROC
curve is .9226. When I wished to check the robustness of the model on a
holdout sample, I keyed in "lroc if sample==2". The software was able to
run it on this separate sample, however, the resulting AROC is now
.4060. I thought AROC values are only from 0.5 to 1.0, or was I
mistaken?

Also, when I tried running roccomp on the two samples, Stata was again
able to churn out the same ROCs (.9226 and .4060), but the chi-square
p-value is .0663, indicating that at alpha=.05, the null hypothesis of
equal areas under the curves cannot be rejected.  Does it make sense
that even though the AROC of the second sample is less than half of the
first sample, the test still says that the 2 ROC curves have equal
areas? Is this test perhaps affected by the number of observations?
(sample 1 : 26 positive, 174 negative, while sample 2 contains 5(+) and
102(-)).

Would really appreciate anyone's help. Thanks in advance! 


- Reggae




*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/