[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: finite mixture models with the EM algorithm

From   Partha Deb <[email protected]>
To   [email protected]
Subject   Re: st: finite mixture models with the EM algorithm
Date   Thu, 17 Sep 2009 16:22:06 -0400


Stas is correct in that EM would have to be written from scratch and implemented case by case.

In my experience, using models with covariates and sample sizes in the thousands generally, EM has been *much* slower than ML, in part because of the number of M-steps needed for convergence. Also, while EM might be able to deal with multimodalities better than ML (esp. Stata's -ml-), there is nothing inherent about EM that makes it "better". Perhaps one ought to try simulated annealing if one is really worried. In my experience, messing around with the starting values has been sufficiently comforting.

Although the convergence problems you are describing are well within the realm of possibilities of an otherwise well specified finite mixture model, in my experience, such severe convergence issues are almost always indication of an otherwise misspecified model (or a poorly specified model vis-a-vis the assumed mixture). I've observed that convergence is more difficult when one has indicator variables with low frequencies, especially when the mixing probabilities are "lopsided" - perhaps this is an issue. What happens if you estimate a sparsely specified model? What does the kernel density of the outcome look like?



László Sándor wrote:
Thanks very much, Stas!

I just heard today in lecture from Guido Imbens that EM might solve
the problem when the likelihood is multimodal, as in the case of
mixtures. Can we reasonably expect Stata's ML procedure to solve this
problem anyway?

With -fmm-, convergence seemed to be really tricky for me. I had to
dig deep and enable -ml search- and set a large number of repetitions
for it. Plus I have the difficult option enabled. Even then, sometimes
I get no convergence just telling me that the likelihood is concave,
or some of the models converged to a case with basically only one
component present out of the possible two -- which is not impossible,
but unlikely in my case.

Thanks again,


On Thu, Sep 17, 2009 at 2:49 PM, Stas Kolenikov <[email protected]> wrote:
The EM algorithm might converge faster, that's all. You will get the
same estimates, although without the standard errors; they are the
pain in the lower back with the EM. If -fmm- works for you, there is
little reason to try and code the EM algorithm from scratch, unless
you'd be lucky to find an implementation floating around. Unlike the
general -ml-, the EM algorithm is problem-specific and depends on what
your missing data are. In -xtmixed-, the missing data are continuous
random effects, while in mixture models, the missing data are discrete
class indicators. The E-steps are entirely different in those two

2009/9/17 László Sándor <[email protected]>:
Dear fellow statalisters,

I am estimating a model where I assume that the dependent variable
came from either of two classical normal linear models, with some
probabilities for the one and the other. This is a finite mixture
model, and I had some success with Partha Deb's -fmm- command.

However, I heard it might be very much worthwhile to try the
estimation using the EM algorithm, not standard (though fine-tuned á
la Stata) ML.  Before any more Google searches getting lost among
latent class models and various related topics, I would ask you
whether you happen to know a tool (or way) that can do this estimation
using the EM algorithm.

(I know that there are different software packages claiming to do
this, but I would rather stick to Stata. Especially because Stata does
EM for -xtmixed-, for instance, so maybe it is also easily applicable
to my problem somehow.)

Thank you very much, you're help is very much appreciated, as always.


*   For searches and help try:

Stas Kolenikov, also found at
Small print: I use this email account for mailing lists only.

*   For searches and help try:

*   For searches and help try:

Partha Deb
Professor of Economics
Hunter College
ph:  (212) 772-5435
fax: (212) 772-5398

Emancipate yourselves from mental slavery
None but ourselves can free our minds.
	- Bob Marley

*   For searches and help try:

© Copyright 1996–2024 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index