Re: st: Nelder Mead optimization in Mata

[jpitblado@stata.com (Jeff Pitblado, StataCorp LP)]

Timothy Mak <timothy.mak07@imperial.ac.uk> is using -optimize()- and wants to
specify a negative value for one of the deltas for Nelder-Mead optimization:

> Can I just clarify if it is always not possible to put in a negative value
> for delta when we use Nelder-Mead in optimize()?
> 
> And if so, why is there such a restriction? 
> 
> I'm using Stata 10.1. 
> 
> See below for a demonstration: 

> mata
> 
> real scalar toy (todo, p, lnf, g, H) {
> 	y = -p[1]^2 - p[2]^2
> 	return(y)
> }
> 
> S = optimize_init()
> optimize_init_evaluator(S, &toy())
> optimize_init_technique(S, "nm")
> optimize_init_params(S, (1,2))
> optimize_init_nmsimplexdeltas(S, (1,-1))
> optimize(S)
> 
> end
> 
> Error message: 
>          opt__validate():  3498  simplex delta value too small (must be greater than 10*ptol)
>               optimize():     -  function returned error
>                  <istmt>:     -  function returned error

Before answering the original question, we need to change Timothy's 'toy()'
function since it will not work with -optimize()-.  'toy()' must be a 'void'
function that returns its results in the 'lnf' argument since the default
evaluator type is "d0" and Nelder-Mead does not make use of derivatives.  Here
is a commented version of 'toy()' that works:

	void toy(todo, p, v, g, H) {
		// INPUT:
		// 	todo	-- ignored, it is always 0
		// 	p	-- parameter vector
		// OUTPUT:
		// 	v	-- value to be optimized
		// 	g	-- ignored, gradient vector
		// 	H	-- ignored, Hessian matrix

		v = -p[1]^2 - p[2]^2
	}

Timothy initialized the parameters at

	(1, 2)

Let's say we want to set the simplex deltas to

	(d1, d2)

then this would result in the following simplex:

	(1, 2)
	(1+d1, 2)
	(1, 2+d2)

Timothy specified the deltas as

	(1, -1)

so, in theory, we should get the following simplex:

	(1, 2)
	(2, 2)
	(1, 1)

With the current restriction on the deltas it is not possible to get this
simplex.

We'll update -optimize()- to require that the deltas are greater that 10*ptol
"in absolute value".

This should be available in a future Stata 11 ado-file update.

Timothy mentioned he is using Stata 10.1, but Stata 10 is not longer being
updated.

In lieu of upgrading to Stata 11, Timothy could change the initial values and
deltas to construct a simplex that contains the one above.  If the initial
values are

	(1, 1)

and the deltas are

	(2, 2)

then the resulting simplex will be

	(1, 1)
	(3, 1)
	(1, 3)

--Jeff
jpitblado@stata.com
*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/