Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: comparison of diagnostic procedures


From   Roger Harbord <roger.harbord@bristol.ac.uk>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: comparison of diagnostic procedures
Date   Mon, 12 Sep 2005 17:45:35 +0100

-serrbar- should plot point estimates as well as CIs automatically so i'm a bit confused. With -twoway rcap- you'd have to overlay a scatter of the point estimates e.g. something like

. twoway rcap sens_lci sens_uci test || scatter sens_pt test


By the way i just noticed this in current issue of Statistics in Medicine which may be of relevance (haven't read it yet) :

Andriy I. Bandos, Howard E. Rockette, David Gur. A permutation test sensitive to differences in areas for comparing ROC curves from a paired design. Statistics in Medicine Volume 24, Issue 18 , Pages 2873 - 2893


--On 11 September 2005 19:40 +0100 Roger Newson <roger.newson@kcl.ac.uk> wrote:


You might possibly want to use the -senspec- package (downloadable from
SSC) to calculate sensitivities and specificities, and then calculate
their standard errors using the standard formulas.. A good overall
performance indicator for comparing two ROC curves is the ROC area, and
the difference between ROC areas (with confidence limits) can be
calculated using the -somersd- package (also downloadable from SSC)
together with -lincom-.

I hope this helps.

Roger


At 19:20 10/09/2005, you wrote:
Αρχικό μήνυμα από  Roger Harbord <roger.harbord@bristol.ac.uk>:
i tried to save the estimates with "parmest" but it is not possible
with "diagt". The graph options "serrbar" and "twoway rcap" can produce a
graph with the confidence intervals of the estimates but it is not
possible to
include the values of sensitivity or specificity.Any advice will be very
helpful.

Thank you a lot!



> Try -serrbar- or -twoway rcap-. However you'd need to first save the
> estimates and CIs as variables. Roger Newson's -parmest- package could
> be one way to do that, after which you could use his -eclplot- package
> (both available on SSC) as an alternative to -serrbar- or -twoway
> rcap-.
>
> Roger H.
>
> --On 08 September 2005 16:15 +0300 htzavara@med.uoa.gr wrote:
>
> > Αρχικό μήνυμα από  Roger Harbord <roger.harbord@bristol.ac.uk>:
> >
> > thank you a lot for your help. This is the solution and i have
> > already find it  in a related article. I would like to ask you if
> > you know how i will produce  an error graph (graph of sensitivities
> > and their confidence interval) for  every diagnostic test? Does
> > stata 8 support a graph like this?
> >
> > thank you a lot in advance!!!!!!
> >
> >
> >> As Pepe mentions on p43, you can test the null hypothesis of equal
> >> sensitivity or  of equal specificity of two binary tests done on the
> >> same  people using McNemar's test (-symmetry- or -mcc- commands in
> >> Stata). I  think something like:
> >>
> >> . symmetry test1 test2 if disease==1   /* for sensitivity */
> >> . symmetry test1 test2 if disease==0   /* for specificity */
> >>
> >>
> >> However with 12 tests there are a lot of comparisons (66 for each of
> >> sens &  spec) so some allowance for multiple testing does seem a
> >> good idea.
> >>
> >> A Bayesian approach seems quite attractive for this sort of problem
> >> as you  can then meaningfully ask "what is the probability that
> >> test X has the  highest sensitivity?", which you can't in a
> >> frequentist framework. You'd  need to switch to something like
> >> WinBUGS to get an answer to that though.
> >>
> >> If one test has higher sensitivity than another but lower
> >> specificity or vice-versa then which is better also depends on the
> >> disbenefits of false positives compared to false negatives of
> >> course.
> >>
> >> Roger.
> >>
> >> --
> >> Roger Harbord
> >> roger.harbord@bristol.ac.uk MRC Health Services Research
> >> Collaboration & Dept. of Social Medicine University of Bristol
> >> http://www.epi.bris.ac.uk/staff/rharbord
> >>
> >> --On 07 September 2005 15:07 -0400 "Michael P. Mueller"
> >> <michael.mueller@utoronto.ca> wrote:
> >>
> >> > You might want to take a look at this book: Pepe, M.S. (2003).
> >> > Statistical Evaluation of Medical Tests for Classification and
> >> > Prediction. Dr. Pepe has Stata programs on her webpage you can
> >> > download. Hope this helps,
> >> > Michael
> >> >
> >> > htzavara@med.uoa.gr wrote:
> >> >
> >> >> Αρχικό μήνυμα από  Svend Juul <SJ@SOCI.AU.DK>:
> >> >>
> >> >>
> >> >>
> >> >>> htzvara (?) wrote:
> >> >>>
> >> >>> i have one variable which represents if the patient has the
> >> >>> disease (coding: 0-
> >> >>> 1)--and this is standard.
> >> >>> Additionally i have 12 more variables which represents the
> >> >>> outcome of 12 different diagnostic procedures (coding: 0-1 for
> >> >>> all of them).I want to find which is the best diagnostic
> >> >>> procedure. I calculate the sensitivity and specificity and
> >> >>> their confidence intervals for each of them. If the confidence
> >> >>> interval for the sensitivity of one diagnostic procedure do not
> >> >>> overlap the confidence interval for the Se of another diagnostic
> >> >>> procedure then the difference is significant.
> >> >>> Is there any test to perform and give p_value? Is there a need
> >> >>> to make a correction for multiple comparisons.?
> >> >>>
> >> >>> ----
> >> >>>
> >> >>> It is not quite clear to me what you want. If it is to find the
> >> >>> single test that has the "best" predictive value, try Paul
> >> >>> Seed's -diagt- (findit diagt). However, you must look at both
> >> >>> sensitivity and specificity to get a meaningful assessment.
> >> >>>
> >> >>> I am not sure why you want to test whether the sensitivity of
> >> >>> two tests are significantly different. And the confidence
> >> >>> interval comparison you describe is quite insensititive.
> >> >>>
> >> >>> Would this show what you need:
> >> >>> Make a logistic regression followed by -lroc- (ROC analysis):
> >> >>>   . logistic disease test1-test12
> >> >>>   . lroc
> >> >>>
> >> >>> You might then try to remove tests to see whether removal makes
> >> >>> a difference to the AUC (area under curve).
> >> >>>
> >> >>> Hope this helps
> >> >>> Svend
> >> >>>
> >> >>>
> >> >>> Thank you very much for your help.
> >> >>>
> >> >>>
> >> >> I know about diagt and i used it to obtain the sensitivity -
> >> >> specificity. Roc analysis cannot help as the variables which
> >> >> represents the diagnostic  tests are not continuous but
> >> >> dichotomous (0-1).Even if i can see, which test  has the best
> >> >> se-sp i want to perform a test to prove it.
> >> >>
> >> >> thank you again.

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index