Maarten is right. This is one way of validating the fitted model in case of
possible unreliability caused by overfitting. This approach was described by
Frank Harrell in his book "Regression Modeling Strategies."
The solution provided by Maarten works. Thanks a lot.
Andrzej
On 9/4/09 11:23 AM, "Maarten buis" <[email protected]> wrote:
> --- Andrzej Niemierko wrote:
>>> I am trying to validate my logistic regression model by calculating a
>>> distribution of area under the ROC curve for bootstrap samples of my
>>> original data.
>
> --- Nick Cox wrote:
>> Let's take this step by step:
>>
>> 1. If you want a new ROC area for each new bootstrap sample that you
>> draw, you can only do that after the sample is drawn.
>>
>> 2. That must therefore be done within your program, and all under the
>> aegis of -simulate-.
>>
>> Thus -mysim- must call -logistic-.
>
> I think the key word in Andrzej question is "validate". In some areas
> this term means estimate a model in a sample, compute a fit statistic
> of that model in (many) alternative sample, and use these fit statistics
> as a measure of fit. If I remember correctly this is supposed to protect
> against overfitting or capitalizing on chance: You are trying to look at
> how well the model generalizes to other populations. I (barely) know of
> its existence, so I can't comment on its merrit, but this would seem to
> me to make Andrzej request reasonable (and I gave a sugestion on how to
> do it in an earlier post).
>
> Hope this helps,
> Maarten
>
> --------------------------
> Maarten L. Buis
> Institut fuer Soziologie
> Universitaet Tuebingen
> Wilhelmstrasse 36
> 72074 Tuebingen
> Germany
>
> http://www.maartenbuis.nl
> --------------------------
>
>
>
>
> *
> * For searches and help try:
> * http://www.stata.com/help.cgi?search
> * http://www.stata.com/support/statalist/faq
> * http://www.ats.ucla.edu/stat/stata/
The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
