st: Generating logistic regression models for forecasting

["Brent McSharry (ADHB)" <BrentM@adhb.govt.nz>]

Dear statalisters

The question is more of a statistics than a Stata question, so apologies if this is the wrong forum (please feel free to redirect me to a more appropriate forum).

Years ago, when developing logistic regression models for forecasting, data would be randomly split into development and testing data sets (using something like .gen develop = runiform() < 1/3). This is much less popular now, and using Rick Thompson's crossval.ado I have a very useful tool for testing fit and discrimination of variables included in the final model.

However, in the stages of model development, It would be logical that I should no longer limit included data to a randomly defined development data set (eg logit x y z if develop). The whole point of the old technique (to my understanding) was to avoid overfitting when choosing which covariates to include. Is it reasonable to develop the model on the whole data set, and then test the final model using crossval, or should I be limiting the data set used for model development some how (or indeed use some tool for multiple resampling to obtain model coefficients and estimates, while in the model development phase)?

Thank you very much.

Brent McSharry MBBS BSc(med) FCICM(paed)
Paediatric Intensivist
Starship Children's Hospital
Private Bag 92024
Auckland 1142
New Zealand 


*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/