st: Nelder Mead in mata optimize: restart

[Matthew J Baker <matthew.baker@hunter.cuny.edu>]

Dear listers:

I have observed that the Nelder-Mead optimization algorithm 
sometimes produces different results when run twice. That is, if 
one maximizes a function using option "nm" and then reruns the 
optimization starting from the maximizing parameters, often a 
new optimum is found. I'm curious if anyone knows why this 
happens: is it a property of the algorithm in general?

Below is a simple example where this happens in a probit 
estimation using a frequency simulator. The code shows that the 
simulated estimator gets it right, but when maximization is run 
twice, a lower value of the likelihood is found. 

clear all
set more off
set seed 8675309
mata

// fictional data
X=invnormal(runiform(1000,1)):>.5
Y=(-.2:+.5:*X:+invnormal(runiform(1000,1))):>0
X=X,J(1000,1,1)
E=invnormal(runiform(1000,1000))

// simulated log likelihood-freq. simulator

void crit(todo,b,y,X,E,crit,g,H)
{
	real matrix Us
	Us=X*b':+E
	crit=y:*ln(rowsum(Us:>0)/cols(Us)):+
		(1:-y):*ln(rowsum(Us:<=0)/cols(Us))
	crit=colsum(crit)
}	
S=optimize_init()
optimize_init_evaluator(S,&crit())
optimize_init_evaluatortype(S,"d0")
optimize_init_params(S,J(1,cols(X),0))
optimize_init_technique(S,"nm")
optimize_init_nmsimplexdeltas(S,J(1,cols(X),1))
optimize_init_argument(S,1,Y)
optimize_init_argument(S,2,X)
optimize_init_argument(S,3,E)
b=optimize(S)
V=optimize_result_V(S)

T=optimize_init()
optimize_init_evaluator(T,&crit())
optimize_init_evaluatortype(T,"d0")
optimize_init_params(T,b)
optimize_init_technique(T,"nm")
optimize_init_nmsimplexdeltas(T,J(1,cols(X),1))
optimize_init_argument(T,1,Y)
optimize_init_argument(T,2,X)
optimize_init_argument(T,3,E)
bb=optimize(T)

bb \ b

end
getmata (X*)=X
getmata Y
probit Y X1

/* end example */

Incidentally, running the nm optimizer a third time does what 
one expects: it returns the starting values as parameters.

Thank you in advance!

Matt Baker

Dr. Matthew J. Baker
Department of Economics
Hunter College and the Graduate Center, CUNY
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/