[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: st: finite mixture models with the EM algorithm

From   "Verkuilen, Jay" <[email protected]>
To   "'[email protected]'" <[email protected]>
Subject   RE: st: finite mixture models with the EM algorithm
Date   Thu, 17 Sep 2009 18:02:01 -0400

As I recall, EM is really good when you're quite far away from the solution but is not so good near it, and thus it is very useful for getting a decent solution to refine with Newton. But nothing is going to be all that great when you have a multimodal likelihood, which is why "protect" optimizations and multi-starts, going all the way to simulated annealing and other such methods are good ideas. Even in the comparatively benign world of SEM doing protect optimizations (which Stata does for you upon request in the exploratory factor analysis program -factor-) is a really good idea, and something many programs don't enable. 

Partha Deb already indicated that big problems with -fmm- (and indeed other latent variable procedures) are most likely to happen when you have a poor model. Example: I know on some data I use for class to illustrate a factor analysis (and hence has been analyzed A LOT, both by me and dozens of students) there is a valid solution that's a local optimum while what I believe the global optimum to be is a boundary solution with a variance = 0. This model is mis-specified because it asks for an extra factor and so it's where trouble is likely. I found it by using something like 1000 random protect optimizations. Stata usually finds the interior point but every once in a while.... 

I'd say that one area where there's been a lot of attention to this issue is in the multidimensional scaling world, because the objective functions in nonmetric MDS are wretched and plagued by local optima. The book by Patrick Groenen and Ingwer Borg (Modern Multidimensional Scaling, 2nd Edition, Springer, 2005) has a lot of discussion of this issue. I believe what they recommend is to try many different random configurations for a relatively small number of iterations (cheap to compute) and refine from the more promising locations. 


*   For searches and help try:

© Copyright 1996–2024 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index