Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: Cut-off point for ROC curve using parametric and non-parametric method


From   "Roger B. Newson" <r.newson@imperial.ac.uk>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: Cut-off point for ROC curve using parametric and non-parametric method
Date   Mon, 21 Jan 2013 16:02:10 +0000

The Youden index can be estimated using -somersd-, for a given binary predictor (or binary predictors), by calculating Somers' D of the binary predictor (or binary predictors) with respect to the outcome. For instance, if the binary (0,1) predictor is -binpred-, and the binary (0,1) outcome is -disease-, then you might type:

somersd disease binpred, tdist transf(z)

to get an asymmetric confidence interval for the Youden index. And, if you have 2 binary predictors (-binpred1- and -binpred2-), then you might type:

somersd disease binpred1 binpred2, tdist
lincom (binpred1-binpred2)/2

to get a symmetric confidence interval for half the difference between 2 Youden indices. This half-difference can tell us whether -binpred1- or -binpred2- is a better predictor of disease.

In your case, you seem to be deriving binary predictors by applying a threshold to a continuous predictor (-p-). This is a trivial extension to the method illustrated above.

However, in your case, the continuous predictor is calculated (using -predict-) from a model fitted to the same data that you later use to estimate the Youden index. If you do this, then the Youden index will be correct, but the confidence limits may not (because there is no rigorous proof that the coverage probability will be correct). If you want to compute confidence limits for Youden indices, and their differences, which can be taken seriously, then you should probably split the data into a training set and a test set, fit the model to the training set, compute the continuous predictor using out-of-sample prediction in the test set, and estimate the Youden indices and their differences in the test set. The issues involved are discussed in Newson (2010).

I hope this helps.

Best wishes

Roger

References

Newson RB. Comparing the predictive power of survival models using Harrell’s c or Somers’ D. The Stata Journal 2010; 10(3): 339–358. Download pre-publication draft from
http://www.imperial.ac.uk/nhli/r.newson/papers.htm#papers_in_journals

Roger B Newson BSc MSc DPhil
Lecturer in Medical Statistics
Respiratory Epidemiology and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton Campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7352 8121 ext 3381
Fax: +44 (0)20 7351 8322
Email: r.newson@imperial.ac.uk
Web page: http://www.imperial.ac.uk/nhli/r.newson/
Departmental Web page:
http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/

Opinions expressed are those of the author, not of the institution.

On 21/01/2013 13:25, Pham Ngoc Minh wrote:
Dear Roger,
Thank you very much for your helpful suggestion. While I am reading
how to use -somersd- package for estimating Youden index, I found
-senspec- package convenient to calculate sensitivity (sn) and
specificity (sp). However, I'm not sure about programs below are
accurate for estimating age-adjusted sn and sp and their subsequent
youden index.

logit metabo bmi age
predict p if e(sample)
senspec metabo p, sensitivity(sn) specificity(sp)
gen youdenid= sn-(1-sp)
egen youdenidmax= max(youdenid)
gen distance = sqrt((1-sn)^2 + (1-sp)^2)
egen mindistance = min(distance)
/* Identify cut-off point */
list sn sp  youdenidmax   distance bmi if abs(youdenid -youdenidmax)<0.000001
list sn sp  youdenid     mindistance bmi  if abs(distance -
mindistance)<0.000001

And evaluating threshold of BMI based on Youden index or shortest
distance from the curve to upper-left corner.

I'd appreciate your responses to ensure the above commands.
sincerely,
Pham Ngoc Minh

On Mon, Jan 21, 2013 at 1:30 AM, Roger B. Newson
<r.newson@imperial.ac.uk> wrote:
If your aim is to estimate the Youden index, then you can use the -somersd-
package, which you can download from SSC. The Youden index is an alternative
name for Somers' D between 2 binary variables. The -somersd- package can
also be used, together with -lincom-, to calculate the difference between 2
Youden indices for 2 diagnostic tests for the same disease.

I hope this helps.

Best wishes

Roger

Roger B Newson BSc MSc DPhil
Lecturer in Medical Statistics
Respiratory Epidemiology and Public Health Group
National Heart and Lung Institute
Imperial College London
Royal Brompton Campus
Room 33, Emmanuel Kaye Building
1B Manresa Road
London SW3 6LR
UNITED KINGDOM
Tel: +44 (0)20 7352 8121 ext 3381
Fax: +44 (0)20 7351 8322
Email: r.newson@imperial.ac.uk
Web page: http://www.imperial.ac.uk/nhli/r.newson/
Departmental Web page:
http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/

Opinions expressed are those of the author, not of the institution.


On 19/01/2013 03:38, Pham Ngoc Minh wrote:

Dear Statalist,

I want to construct age-adjusted (age is a continuous variable) optimal
cut-off points for body mass index (bmi)  and waist circumference (wc) in
determining metabolic syndrome (metabo) for a large study. My question is
how can I calculate sensitivity and specificity for bmi and wc with
adjustment for age, based on which I can calculate Youden index. For
non-parametric method, I know only about using command: roctab metabo bmi,
detail or senspec metabo bmi, se(varname1) spe(varname2)  (without
covariate adjustment). Moreover, how can I calculate age-adjusted
sensitivity and specificity using parametric ROC analysis. It is because
   using non-parametric method showed P less than 0.01, although AUC for
bmi and wc was somewhat similar.
Thank you for your guidance in advance.

Pham Ngoc Minh
Thai Nguyen University Faculty of Public Health, Vietnam
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index