Bookmark and Share

Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down at the end of May, and its replacement, statalist.org is already up and running.


[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: How to compare RMSE of two models


From   "JVerkuilen (Gmail)" <jvverkuilen@gmail.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: How to compare RMSE of two models
Date   Tue, 27 Nov 2012 09:14:13 -0500

On Tue, Nov 27, 2012 at 7:49 AM, rasool.bux <rasool.bux@aku.edu> wrote:
> Dear Maarten,
>
> Thanks for reply. May be I am not conveying my question clearly. I would like to compare the two models against the same yvariable, which model is performing better and how we could say that model 1 is better than model 2 i.e. in terms of p-values of RMSE.
>

I assume you don't have nested models, because otherwise you would
simply use -testparm- to consider whether the additional coefficients
are jointly 0. Non-nested model tests are tricky. The Vuong test is
one example. http://en.wikipedia.org/wiki/Vuong's_closeness_test

However, if your y variables are the same (i.e., no missing data) you
can probably get most of what you want using AIC or by comparing
adjusted R-square changes between models. If you must have a p-value I
suspect that the only reasonable way is to use bootstrapping:

http://stata.com/statalist/archive/2010-03/msg01845.html

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/


© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   Site index