We had a similar discussion on the list this week. In that case, the topic
was the R2 for Multiple Imputation (MI). Maarten proposed (for R2 case) to
compute the geometric average instead of arithmetic one, based on Donald
Rubin's reply somewhere else. Hansen J test is asymptotically distributed as
chi-square, maybe a similar suggestion applies for your case.
My own suggestion for the R2 was to report your simple average and write a
small note with the min/max R2 along your regressions. In your case, I
suggest to analyze more in deep the figures for Hansen J tests and the
p-values associated with these. I think that is perfectly OK to have pvalues
of 0.01 0.008, etc. (similar magnitud)... and I don't expect to see very
different values for Hansen J test as well. If so... then you have problems
with the model and/or the method of MI.
All this works if your # of missing over the total observations is few and
if you imputed all the variables (including the variables used in the first
step) at once. Finally, MI methods are based on simulations then in practice
I generate more than 5 datasets and play with some combinations of 5
datasets (2nd to 6th, etc) and with more datasets (8, 10 or 12) to see if
the results change.
R
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
