Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at statalist.org.

# st: Nelder Mead in mata optimize: restart

 From Matthew J Baker <[email protected]> To [email protected] Subject st: Nelder Mead in mata optimize: restart Date Fri, 8 Apr 2011 18:13:36 -0400 (EDT)

```Dear listers:

I have observed that the Nelder-Mead optimization algorithm
sometimes produces different results when run twice. That is, if
one maximizes a function using option "nm" and then reruns the
optimization starting from the maximizing parameters, often a
new optimum is found. I'm curious if anyone knows why this
happens: is it a property of the algorithm in general?

Below is a simple example where this happens in a probit
estimation using a frequency simulator. The code shows that the
simulated estimator gets it right, but when maximization is run
twice, a lower value of the likelihood is found.

clear all
set more off
set seed 8675309
mata

// fictional data
X=invnormal(runiform(1000,1)):>.5
Y=(-.2:+.5:*X:+invnormal(runiform(1000,1))):>0
X=X,J(1000,1,1)
E=invnormal(runiform(1000,1000))

// simulated log likelihood-freq. simulator

void crit(todo,b,y,X,E,crit,g,H)
{
real matrix Us
Us=X*b':+E
crit=y:*ln(rowsum(Us:>0)/cols(Us)):+
(1:-y):*ln(rowsum(Us:<=0)/cols(Us))
crit=colsum(crit)
}
S=optimize_init()
optimize_init_evaluator(S,&crit())
optimize_init_evaluatortype(S,"d0")
optimize_init_params(S,J(1,cols(X),0))
optimize_init_technique(S,"nm")
optimize_init_nmsimplexdeltas(S,J(1,cols(X),1))
optimize_init_argument(S,1,Y)
optimize_init_argument(S,2,X)
optimize_init_argument(S,3,E)
b=optimize(S)
V=optimize_result_V(S)

T=optimize_init()
optimize_init_evaluator(T,&crit())
optimize_init_evaluatortype(T,"d0")
optimize_init_params(T,b)
optimize_init_technique(T,"nm")
optimize_init_nmsimplexdeltas(T,J(1,cols(X),1))
optimize_init_argument(T,1,Y)
optimize_init_argument(T,2,X)
optimize_init_argument(T,3,E)
bb=optimize(T)

bb \ b

end
getmata (X*)=X
getmata Y
probit Y X1

/* end example */

Incidentally, running the nm optimizer a third time does what
one expects: it returns the starting values as parameters.

Matt Baker

Dr. Matthew J. Baker
Department of Economics
Hunter College and the Graduate Center, CUNY
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```