# Re: st: question with MLE and complex model

 From Partha Deb To statalist@hsphsun2.harvard.edu Subject Re: st: question with MLE and complex model Date Sat, 02 Aug 2008 23:16:05 -0400

Jonathan,

3100 is not a particularly large sample in the context of a finite mixture model. So, it does not surprise me entirely that your model has numerical problems when you introduce covariates into the mixing probabilities. That said, it's possible that the issue is somewhere else. If you send me your dataset and code off list, I'll spend a bit of time investigating.

You ask "By the way, how does one obtain the estimated probability weight on each component for an observation?"

For estimates of prior probabilities, after estimation -predict varname1, eq(component1) prior- , etc. For posterior probabilities, -predict varname1, eq(component1) posterior- , etc.

Jonathan Hanson wrote:

```Partha,

On Jul 30, 2008, at 10:30 PM, Partha Deb wrote:

```
Jonathan,

There are a number of likely reasons why starting values are not feasible.

1. If you have a small sample, it may be too ambitious to model the probabilities as functions of covariates. You may simply be too close to a boundary for some observations. What is your sample size?
```My sample is about 3,100.

```

2. Although Stata is quite good about handling scaling issues, scaling can matter. If your z's are scaled very differently, that could be a source of the problem.
```Thanks, they are not dramatically different, but I will try some rescaling.

```
What happens when you estimate 2 and 3 component mixtures without covariates in the probabilities?
I can get results with no covariates in the probabilities when starting with a 2 component mixture and then moving to 3 component mixture. Usually, as soon as I add one covariate in the probabilities, Stata cannot find initial values, whether there are 2 or three components.

By the way, how does one obtain the estimated probability weight on each component for an observation? I've read the help file but can't figure it out. I'd like to examine how the observations are being weighted.

Your generous help is greatly appreciated!

Jonathan

On your question about why your original effort fell short - obviously I don't know for sure, but the likelihood you had written out was not the likelihood for a mixture model - beyond that, I'm afraid I didn't look particularly hard at the formulation.

Partha

Jonathan Hanson wrote:

Partha,
Thanks again for your help -- I really appreciate it. Your tip did help me get up to a three-component model. I am not consistently able to do so, however, especially when using variables to estimate the component probabilities. When adding one of these variables, even to a two-component model, Stata often reports that it cannot find feasible values at the beginning of the estimation. Perhaps I need to supply my own starting values?
Also, since I hope to learn how to use ml, I am still puzzling over why my original effort is falling short. Are there some obvious things that I should think about as I work on it?
Thanks!
Jonathan
On Jul 30, 2008, at 2:08 PM, Partha Deb wrote:

```.

Correction, start here:

fmm y x1 x2 x3, components(2) mixtureof(normal) prob(z1 z2 z3)

fmm y x1 x2 x3, components(3) mixtureof(normal) prob(z1 z2 z3)

Partha Deb wrote:
```
Jonathan,
The easiest thing to do is try the following:
fmm y x1 x2 x3, components(2) mixtureof(normal) search(on) prob(z1 z2 z3)
fmm y x1 x2 x3, components(3) mixtureof(normal) search(on) prob(z1 z2 z3) from(?)
If a 3-component model follows a 2-component model, it gets "smart" starting values based on the parameters in the 2-component model.
If you prefer to use -from()-, enter parameter values (separated by spaces) in the other they would appear in the output. Obviously this is hard to guess if you've never seen any output from -fmm- so I recommend the "two-step" approach first.
Best.
Partha
Jonathan Hanson wrote:

Partha,

Thank you for your suggestion. I've downloaded the fmm procedure and am experimenting with it. One question: I get a message stating that I should provide starting values, which appears to be done using the from() option. What is the form with which starting values are provided?

For example, if I use:

fmm y x1 x2 x3, components(3) mixtureof(normal) search(on) prob(z1 z2 z3) from(?)

what replaces the question mark?

Many thanks!

Jonathan

On Jul 30, 2008, at 11:56 AM, Partha Deb wrote:

Jonathan,

It appears that the model you are trying to estimate (in principle) is a finite mixture of 3 normal densities - your code is not quite that, however. Unless you are sure yours is the model you want, I suggest you estimate a standard finite mixture model for the problem you describe. You can code that up yourself or use -fmm- . -findit fmm- will get you to the link to install it.

Best.

Partha

Jonathan Hanson wrote:

Greetings,
I am working on an MLE procedure to use in situations where there may be distinct, or at least mostly distinct, causal processes at work for different parts of the sample. For example, suppose there are three different states of the world, and the coefficients on key explanatory variables vary across these states. Additionally, suppose that there is a set of variables that determines (probabilistically) the extent to which to a particular case falls into each state.
In other words, I have three linear models: mod1, mod2, and mod3. Also, I have a weighting function, similar to that used in multinomial logit, that estimates a set of weights that sum to 1: p1 + p2 + p3.
I am fairly new to ML programming, so I started with the ML version of a standard linear regression (with adjustments to s_e suggested by Gould and Sribney) and incorporated the three linear models with their corresponding weighting functions. The trouble is, when I try to estimate the model, Stata goes through thousands of iterations, nearly all of which report "not concave". Convergence is achieved only rarely, and it depends very much upon specification.
program define stage3ml
args lnf mod1 st1 mod2 st2 mod3 ctrls s_e
tempvar den p1 p2 p3
quietly gen double `den' = 1 + exp(`st1') + exp(`st2')
quietly gen double `p1' = exp(`st1')/`den'
quietly gen double `p2' = exp(`st2')/`den'
quietly gen double `p3' = 1/`den'
quietly replace `lnf'=ln(normalden((\$ML_y1 - `p1'*`mod1' - `p2'*`mod2' - `p3'*`mod3' - `ctrls')/exp(`s_e'))) - `s_e'
end
ml model lf stage3ml (mod1: y = x1 x2 x3) (p1: z1 z2 z3) (mod2: x1 x2 x3) (p2: z1 z2 z3) (mod3: x1 x2 x3) (ctrs: ) ()
Any advice on what steps I should next take would be greatly appreciated!
Many thanks,
Jonathan Hanson
Assistant Professor of Political Science
Maxwell School of Citizenship and Public Affairs
100 Eggers Hall
Syracuse University
Syracuse, NY 13244
johanson@maxwell.syr.edu
*
* For searches and help try:
* http://www.stata.com/help.cgi?search
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
```--
Partha Deb
Department of Economics
Hunter College
ph:  (212) 772-5435
fax: (212) 772-5398
http://urban.hunter.cuny.edu/~deb/

Emancipate yourselves from mental slavery
None but ourselves can free our minds.
- Bob Marley

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```
```_______________________________________
Jonathan Hanson
Assistant Professor of Political Science
Maxwell School of Citizenship and Public Affairs
100 Eggers Hall
Syracuse University
Syracuse, NY 13244
johanson@maxwell.syr.edu

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```
```--
Partha Deb
Department of Economics
Hunter College
ph:  (212) 772-5435
fax: (212) 772-5398
http://urban.hunter.cuny.edu/~deb/

Emancipate yourselves from mental slavery
None but ourselves can free our minds.
- Bob Marley

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```
```_______________________________________
Jonathan Hanson
Assistant Professor of Political Science
Maxwell School of Citizenship and Public Affairs
100 Eggers Hall
Syracuse University
Syracuse, NY 13244
johanson@maxwell.syr.edu
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```
```--
Partha Deb
Department of Economics
Hunter College
ph:  (212) 772-5435
fax: (212) 772-5398
http://urban.hunter.cuny.edu/~deb/

Emancipate yourselves from mental slavery
None but ourselves can free our minds.
- Bob Marley

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```
```____________________________________
Jonathan Hanson
Assistant Professor of Political Science
Maxwell School of Citizenship and Public Affairs
100 Eggers Hall
Syracuse University
Syracuse, NY 13244
johanson@maxwell.syr.edu

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```
```--
Partha Deb
Department of Economics
Hunter College
ph:  (212) 772-5435
fax: (212) 772-5398
http://urban.hunter.cuny.edu/~deb/

Emancipate yourselves from mental slavery
None but ourselves can free our minds.
- Bob Marley

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```