Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: bootstrapping with senspec


From   "Bains, Lauren" <Lauren.Bains@insel.ch>
To   "statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu>
Subject   st: bootstrapping with senspec
Date   Tue, 4 Sep 2012 09:23:19 +0000

Hello,

I am trying to use bootstrapping in STATA 12.1 to calculate 95% confidence intervals of "sensitivity", "specificity", and "accuracy" on a clustered dataset of diagnosing positive and negative lymph node metastases clustered by pelvic side (right and left pelvic sides).  I am new to programming with STATA, and am having some problems with the CIs, which I assume are likely related to my initial programming attempts.  

I am using the module senspec to return the true positives (TP), false negatives (FN), TN, FP, calculate accuracy, and return the sensitivity, specificity, and accuracy, which I downloaded from:

http://ideas.repec.org/c/boc/bocode/s439801.html

My bootstrapping program looks like this (apologies for what is likely an inelegant attempt):

capture program drop bootstrap_sens_spec_da
program define sens_spec_da, rclass
	tempvar s_calc_sens s_calc_spec fp1 fn1 tp1 tn1
	senspec `1' `2', sensitivity(`s_calc_sens') specificity(`s_calc_spec') nfpos(`fp1') nfneg(`fn1') ntpos(`tp1') ntneg(`tn1')
	return scalar calc_da = (`tp1'+`tn1')/(`tp1'+`tn1'+`fp1'+`fn1')
	return  scalar calc_sens =`s_calc_sens'
	return scalar calc_spec =`s_calc_spec'
end

Then, I am using bootstrapping to calculate the confidence intervals:

bootstrap r(calc_sens) r(calc_spec) r(calc_da), reps(1000) cluster(side): sens_spec_da histo_LN_ bin_R3_LN_
estat bootstrap, all

Some of the time this seems to work although the CIs seem large, compared with the results that one gets for sensitivity and specificity when not accounting for clustering using, for example, diagt.  Sometimes it does not work at all.  Using diagt to find the sensitivity and specificity for the 3rd reader works fine, but the bootstrapping fails.  Here is the output of diagt:

. diagt histo_LN_ bin_R3_LN_

           |      bin_R3_LN_
 histo_LN_ |      Pos.       Neg. |     Total
-----------+----------------------+----------
  Abnormal |        25         19 |        44 
    Normal |        25        171 |       196 
-----------+----------------------+----------
     Total |        50        190 |       240 

True abnormal diagnosis defined as histo_LN_ = 1


                                                  [95% Confidence Interval]
---------------------------------------------------------------------------
Prevalence                         Pr(A)     18.3%     13.6%      23.8%
---------------------------------------------------------------------------
Sensitivity                      Pr(+|A)     56.8%     41.0%     71.7%
Specificity                      Pr(-|N)     87.2%     81.7%     91.6%



And here is STATA's output of bootstrapping on the readings for R3 (the third reader):

. bootstrap r(calc_sens) r(calc_spec) r(calc_da), reps(1000) cluster(side): sens_spec_da histo_LN_ bin_R3_LN_
....

Bootstrap results                               Number of obs      =       240
                                                Replications       =      1000

      command:  sens_spec_da histo_LN_ bin_R3_LN_
        _bs_1:  r(calc_sens)
        _bs_2:  r(calc_spec)
        _bs_3:  r(calc_da)

                                    (Replications based on 2 clusters in side)
------------------------------------------------------------------------------
             |   Observed   Bootstrap                         Normal-based
             |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       _bs_1 |          1          .        .       .            .           .
       _bs_2 |          0  (omitted)
       _bs_3 |   .1833333   .0235188     7.80   0.000     .1372373    .2294294
------------------------------------------------------------------------------

 (notice that the first two results, for sensitivity and specificity, fail to match with diagt)

This is my first time posting to the STATA listserv, so I give my apologies in advance if I have provided too much (or not enough) detail.  I can attach the dataset if that would be helpful.  Any suggestions would be much appreciated!
Thanks,

Lauren Bains

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index