Bema,
I had the same problem a while back, and I found that performing -lroc-
after -bs- performs the ROC test using information from the final
bootstrap iteration. I assume this applies to -lfit- as well, though I
have not tested it.
To solve this, I wrote (with help from several people on this list) a
-bstrap- function which included an -lroc- call in every iteration.
Though it's specific to the issue I was looking at, I can email it to
you as an example function if you'd like. I should also mention that I
adopted in my function Efron's suggestion of bootstrapping
"over-optimism" in an accuracy index (like the ROC curve); Frank
Harrell's Regression Modeling Strategies has a good explanation of this
in Section 5.2.5.
Regards,
AJ Bostian
>Hello listers,
>
>I have been working on a logistic regression model of some data that I
>have. The model fits well as assessed by the Hosmer-Lemeshow
>goodness-of-fit statistic [lfit, group(10) command], and has good
>discrimination (lroc command). Local regression diagnostics are also
>good.
>
>After constructing the model, I ran 1000 bootstrap repetitions using
the
>bs command to determine the amount of over-fitting in the model. I then
>retyped the lfit and lroc commands as I had before, and got very
>different numbers from those that I had calculated earlier.
>
>I am curious to know and would be grateful if anyone could tell me if:
>1. it is legitimate to run the lroc & lfit commands after a bootstrap
>procedure in Stata, and if so
>2. If typing these commands after running a bootstrap procedure,
>reports these statistics for all the bootstrapped samples, the last
>bootstrapped sample, or some average of those samples.
>
>Thanks a great deal
>
>Sincerely,
>
>Bema Bonsu
>
>*
>* For searches and help try:
>* http://www.stata.com/support/faqs/res/findit.html
>* http://www.stata.com/support/statalist/faq
>* http://www.ats.ucla.edu/stat/stata/
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
