Notice: On March 31, it was announced that Statalist is moving from an email list to a forum. The old list will shut down on April 23, and its replacement, statalist.org is already up and running.

# st: numerical derivatives and -ml- command

 From Sun Yutao To Subject st: numerical derivatives and -ml- command Date Mon, 5 Nov 2012 22:26:22 +0100

```Hello,

I'm trying to write an maximum likelihood algorithm that can handle a
particularly large number of fixed effects (which is not possible with Stata's
#var and matsize limitation). I have been comparing the performance of my ml
with the Stata -ml- these days and I find something really strange. More
specifically, does anyone know how the Newton-Raphson algorithm in Stata -ml-
command works in detail? Because what I see from a small experiment is that
the -ml- command is just way too fast so it's not possible to iteratively
update the Hessian by numerical differentiation.

Here the story goes:

For an original Newton-Raphson(nr), one needs to update the gradients and
Hessian in each iteration.  And suppose we have a likelihood function(llf), 5
variables, and a given number of observations. And a single evaluation of the
llf takes 0.0475 seconds, so for a 2-point approximation of the gradient
vector one need 5*2=10 evaluations, which gives roughly 0.475 seconds. And the
Hessian is a bit complicated: every off-diagonal element will need 4 llf
evaluations and you have  4+3+2+1=10 of them, while the diagonal elements need
2 evaluations each, plus 1 for the common f(x), which gives 5*2+1 llf
evaluations and hence for the Hessian one needs 10*4+5*2+1=51 llf evaluations
which gives  2.4225 seconds.

Based on that, one iterations in this setting should roughly take 3 seconds.
However, the Stata -ml- command only needs 1 second per iteration, which is
particularly uncommon. And even if my calculations for the Hessian was wrong,
I'm still convinced that the Hessian needs more llf evaluations and hence more
time to compute than the gradients. So it cannot possibly be 1 second per
iteration. And in fact, in terms of performances, the Stata nr behaves a
little bit like a quasi-Newton + a line search... Or is It possible that Stata
secretly have a feature that can take analytical derivatives on a
user-specified function?

Does anyone know the answer to this problem? Or if there just is a better way
to compute numerical derivatives that I do not know?

Best regards,
Sun Yutao

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/faqs/resources/statalist-faq/
*   http://www.ats.ucla.edu/stat/stata/
```