Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Area under the ROC curve less than 0.5?


From   Roger Newson <[email protected]>
To   "[email protected]" <[email protected]>
Subject   Re: st: Area under the ROC curve less than 0.5?
Date   Mon, 24 Jan 2011 10:27:39 +0000

This is a common problem. The solution is to exit Stata after installing -somersd- and to launch Stata again. The -somersd- package should then work.

The reason for this problem is that -somersd- uses a Mata library, containing a suite of Mata programs. When Stata is launched, it creates a list of Mata libraries in which to look for Mata programs. If the user then installs a package with a Mata library (such as -somersd-), then some versions of Stata will not update this list by adding the new Mata library to the list. However, if the user exits Stata and then launches Stata again, then the list of Mata libraries will be created again, and should include the Mata library in the -somersd- package.

I hope this helps. Let me know if it doesn't.

Best wishes

Roger


Roger B Newson BSc MSc DPhil
Lecturer in Medical Statistics
Respiratory Epidemiology and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton Campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7352 8121 ext 3381
Fax: +44 (0)20 7351 8322
Email: [email protected]
Web page: http://www.imperial.ac.uk/nhli/r.newson/
Departmental Web page:
http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/

Opinions expressed are those of the author, not of the institution.

On 24/01/2011 04:33, [email protected] wrote:
Thank you very much Professor for taking the time to answer my queries. It really helped me comprehend better how AROC works. I also did as you suggested, and was able to install the -somersd- package by typing in "ssc install somersd". However, when I tried using the program, stata returned an error message:

tidotforsomersd():  3499  panelsetup() not found
	<istmt>:     -  function returned error

Can you also help me figure out how to fix this?


Very much obliged,


Reggae




 From :   Roger Newson<[email protected]>
To : "[email protected]"<[email protected]>
Subject : Re: st: Area under the ROC curve less than 0.5?
Date :  Fri, 21 Jan 2011 13:54:56 +0000

In reply to your first query, the area under the ROC can be anything from 0 to 1. Predictors with a ROC area less than 0.5 are negative prwedictors, and predictors with a ROC area between 0.5 and 1 are positive predictors.

In reply to your second query, it is possible for the difference between 2 ROC areas to be statistically non-significant in a small sample, even if one ROC area is less than half the other. This is because the confidence interval for the difference between 2 ROC areas may be wide, and include a zero difference and a range of positive and negative differences. To get a confidence interval for the difference between 2 ROC areas (also known as Harrell's c statistics), use the -somersd- package, which you can download from SSC and which has 3 .pdf manuals, distributed with the package as ancillary files. The on-line help contains hyper-references to the Stata Journal articles explaining the -somersd- package, which you can use with -lincom- to get confidence intervals for the difference between 2 Harrell's c statistics.

I hope this helps.

Best wishes

Roger


Roger B Newson BSc MSc DPhil
Lecturer in Medical Statistics
Respiratory Epidemiology and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton Campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7352 8121 ext 3381
Fax: +44 (0)20 7351 8322
Email: [email protected]
Web page: http://www.imperial.ac.uk/nhli/r.newson/
Departmental Web page:
http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/


Opinions expressed are those of the author, not of the institution.

	On 21/01/2011 09:56, reggae.benigno wrote:

	Hi!

	I just want to ask, after running a logistic regression on a 	development sample, I ran the lroc command. The corresponding area 	under  the ROC curve is .9226. When I wished to check the robustness 	of the model on a holdout sample, I keyed in "lroc if sample==2". The 	software was able to run it on this separate sample, however, the 	resulting AROC is now .4060. I thought AROC values are only from 0.5 	to 1.0, or was I mistaken?

	Also, when I tried running roccomp on the two samples, Stata was again
	able to churn out the same ROCs (.9226 and .4060), but the chi-square
	p-value is .0663, indicating that at alpha=.05, the null hypothesis of
	equal areas under the curves cannot be rejected.  Does it make sense
	that even though the AROC of the second sample is less than half of 	the first sample, the test still says that the 2 ROC curves have equal
	areas? Is this test perhaps affected by the number of observations?
	(sample 1 : 26 positive, 174 negative, while sample 2 contains 5(+) 	and 102(-)).

	Would really appreciate anyone's help. Thanks in advance!


	- Reggae





*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index