Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

st: how evaluate the accuracy of models in a resampling process?

From   Albert Navarro <>
Subject   st: how evaluate the accuracy of models in a resampling process?
Date   Mon, 18 Apr 2011 20:02:20 +0100 (BST)

Dear all,

we are conducting a study to identify the associated distribution with the generating process of a particular phenomenon (survival data). Briefly:

1. We fitted Weibull, log-normal and log-logistic models to 1000 resamples (null models, without covariates)
2. We compared the AIC of the models in each resample. We selected the model with lowest AIC in the higher number of resamples.
3. Estimated parameters: we selected the median of the estimations in the 1000 resamples, for the selected model.

The next step would be to get evidence on the accuracy of the model selected. The best model is not necessarily a good model ...

Because we are working with 1000 resamples, graphical methods aren’t very practical. We prefer not using a pseudo R2 (“A perfectly adequate model may have what, at face value, seems like a terrible low R2 due to a high percent of censored data” [Hosmer-Lemeshow]).

Can anyone help us, please? For weeks we are thinking on this issue and we fail to find a good solution.

Thank you very much,

Best regards,

Albert Navarro

*   For searches and help try:

© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index