[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]
st: Re: lroc and lfit after bootstrapping a logistic model
I had the same problem a while back, and I found that performing -lroc-
after -bs- performs the ROC test using information from the final
bootstrap iteration. I assume this applies to -lfit- as well, though I
have not tested it.
To solve this, I wrote (with help from several people on this list) a
-bstrap- function which included an -lroc- call in every iteration.
Though it's specific to the issue I was looking at, I can email it to
you as an example function if you'd like. I should also mention that I
adopted in my function Efron's suggestion of bootstrapping
"over-optimism" in an accuracy index (like the ROC curve); Frank
Harrell's Regression Modeling Strategies has a good explanation of this
in Section 5.2.5.
>I have been working on a logistic regression model of some data that I
>have. The model fits well as assessed by the Hosmer-Lemeshow
>goodness-of-fit statistic [lfit, group(10) command], and has good
>discrimination (lroc command). Local regression diagnostics are also
>After constructing the model, I ran 1000 bootstrap repetitions using
>bs command to determine the amount of over-fitting in the model. I then
>retyped the lfit and lroc commands as I had before, and got very
>different numbers from those that I had calculated earlier.
>I am curious to know and would be grateful if anyone could tell me if:
>1. it is legitimate to run the lroc & lfit commands after a bootstrap
>procedure in Stata, and if so
>2. If typing these commands after running a bootstrap procedure,
>reports these statistics for all the bootstrapped samples, the last
>bootstrapped sample, or some average of those samples.
>Thanks a great deal
>* For searches and help try:
* For searches and help try: