Notice: On March 31, it was **announced** that Statalist is moving from an email list to a **forum**. The old list will shut down at the end of May, and its replacement, **statalist.org** is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
"Bains, Lauren" <Lauren.Bains@insel.ch> |

To |
"statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu> |

Subject |
AW: st: bootstrapping with senspec |

Date |
Tue, 4 Sep 2012 11:49:16 +0000 |

Thanks, Roger. I did notice that senspec is not ideal for the reason that you suggest (that it does not directly store them in r() or e()). However, being relatively new to Stata, I am having trouble finding another function that does return what I need as an r() or e(). What I need are: Sensitivity = TP/(TP+FN) Specificity = TN/(TN+FP) Accuracy = (TP+TN)/(TP+TN+FP+FN) Or, alternatively, something that returns the numbers of true positives, true negatives, false positives, and false negatives so that I can calculate the above. Diagt does the job for sensitivity and specificity, but not accuracy. Does anyone have a suggestion for a Stata module that would be better suited to this problem than senspec? Thanks and best regards, Lauren Bains -----Ursprüngliche Nachricht----- Von: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu] Im Auftrag von Roger B. Newson Gesendet: Dienstag, 4. September 2012 12:57 An: statalist@hsphsun2.harvard.edu Betreff: Re: st: bootstrapping with senspec As Nick seems to suspect, -senspec- was not really designed for use with -bootstrap-, because, as stated in -help senspec-, -senspec- does not save many Saved Results (just the sample number and the numbers of opositive and negative observations). If you want to bootstrap a sensitivity or specificity (or any other proportion), then you should use a command that estimates proportions and stores them in Saved Results in -r()- and/or -e()-. Best wishes Roger Roger B Newson BSc MSc DPhil Lecturer in Medical Statistics Respiratory Epidemiology and Public Health Group National Heart and Lung Institute Imperial College London Royal Brompton Campus Room 33, Emmanuel Kaye Building 1B Manresa Road London SW3 6LR UNITED KINGDOM Tel: +44 (0)20 7352 8121 ext 3381 Fax: +44 (0)20 7351 8322 Email: r.newson@imperial.ac.uk Web page: http://www.imperial.ac.uk/nhli/r.newson/ Departmental Web page: http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/ Opinions expressed are those of the author, not of the institution. On 04/09/2012 10:38, Nick Cox wrote: > Roger Newson, the author of -senspec-, is a member of this list and > no doubt will comment. But this looks wrong to me. In essence, > -senspec- generates lots of variables. But you are trying to force > them into scalars. In practice what that will mean is that the first > value of each variable, and only the first value, will be carried > over. I think you need another approach. > > In future postings, please note details that the FAQ explains: > > 1. No; you should not attach the dataset. Attachments should not be > sent to Statalist. > > 2. For "STATA" read "Stata" throughout. > > Nick > > On Tue, Sep 4, 2012 at 10:23 AM, Bains, Lauren <Lauren.Bains@insel.ch> wrote: > >> I am trying to use bootstrapping in STATA 12.1 to calculate 95% confidence intervals of "sensitivity", "specificity", and "accuracy" on a clustered dataset of diagnosing positive and negative lymph node metastases clustered by pelvic side (right and left pelvic sides). I am new to programming with STATA, and am having some problems with the CIs, which I assume are likely related to my initial programming attempts. >> >> I am using the module senspec to return the true positives (TP), false negatives (FN), TN, FP, calculate accuracy, and return the sensitivity, specificity, and accuracy, which I downloaded from: >> >> http://ideas.repec.org/c/boc/bocode/s439801.html >> >> My bootstrapping program looks like this (apologies for what is likely an inelegant attempt): >> >> capture program drop bootstrap_sens_spec_da >> program define sens_spec_da, rclass >> tempvar s_calc_sens s_calc_spec fp1 fn1 tp1 tn1 >> senspec `1' `2', sensitivity(`s_calc_sens') specificity(`s_calc_spec') nfpos(`fp1') nfneg(`fn1') ntpos(`tp1') ntneg(`tn1') >> return scalar calc_da = (`tp1'+`tn1')/(`tp1'+`tn1'+`fp1'+`fn1') >> return scalar calc_sens =`s_calc_sens' >> return scalar calc_spec =`s_calc_spec' >> end >> >> Then, I am using bootstrapping to calculate the confidence intervals: >> >> bootstrap r(calc_sens) r(calc_spec) r(calc_da), reps(1000) cluster(side): sens_spec_da histo_LN_ bin_R3_LN_ >> estat bootstrap, all >> >> Some of the time this seems to work although the CIs seem large, compared with the results that one gets for sensitivity and specificity when not accounting for clustering using, for example, diagt. Sometimes it does not work at all. Using diagt to find the sensitivity and specificity for the 3rd reader works fine, but the bootstrapping fails. Here is the output of diagt: >> >> . diagt histo_LN_ bin_R3_LN_ >> >> | bin_R3_LN_ >> histo_LN_ | Pos. Neg. | Total >> -----------+----------------------+---------- >> Abnormal | 25 19 | 44 >> Normal | 25 171 | 196 >> -----------+----------------------+---------- >> Total | 50 190 | 240 >> >> True abnormal diagnosis defined as histo_LN_ = 1 >> >> >> [95% Confidence Interval] >> --------------------------------------------------------------------------- >> Prevalence Pr(A) 18.3% 13.6% 23.8% >> --------------------------------------------------------------------------- >> Sensitivity Pr(+|A) 56.8% 41.0% 71.7% >> Specificity Pr(-|N) 87.2% 81.7% 91.6% >> >> >> >> And here is STATA's output of bootstrapping on the readings for R3 (the third reader): >> >> . bootstrap r(calc_sens) r(calc_spec) r(calc_da), reps(1000) cluster(side): sens_spec_da histo_LN_ bin_R3_LN_ >> .... >> >> Bootstrap results Number of obs = 240 >> Replications = 1000 >> >> command: sens_spec_da histo_LN_ bin_R3_LN_ >> _bs_1: r(calc_sens) >> _bs_2: r(calc_spec) >> _bs_3: r(calc_da) >> >> (Replications based on 2 clusters in side) >> ------------------------------------------------------------------------------ >> | Observed Bootstrap Normal-based >> | Coef. Std. Err. z P>|z| [95% Conf. Interval] >> -------------+---------------------------------------------------------------- >> _bs_1 | 1 . . . . . >> _bs_2 | 0 (omitted) >> _bs_3 | .1833333 .0235188 7.80 0.000 .1372373 .2294294 >> ------------------------------------------------------------------------------ >> >> (notice that the first two results, for sensitivity and specificity, fail to match with diagt) >> >> This is my first time posting to the STATA listserv, so I give my apologies in advance if I have provided too much (or not enough) detail. I can attach the dataset if that would be helpful. Any suggestions would be much appreciated! > > * > * For searches and help try: > * http://www.stata.com/help.cgi?search > * http://www.stata.com/support/statalist/faq > * http://www.ats.ucla.edu/stat/stata/ > * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/ * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**References**:**st: bootstrapping with senspec***From:*"Bains, Lauren" <Lauren.Bains@insel.ch>

**Re: st: bootstrapping with senspec***From:*Nick Cox <njcoxstata@gmail.com>

**Re: st: bootstrapping with senspec***From:*"Roger B. Newson" <r.newson@imperial.ac.uk>

- Prev by Date:
**Re: st: Problem with the results of "mfx" command** - Next by Date:
**st: replacing zero length strings with missing values** - Previous by thread:
**Re: st: bootstrapping with senspec** - Next by thread:
**st: onespell** - Index(es):