Notice: On March 31, it was **announced** that Statalist is moving from an email list to a **forum**. The old list will shut down at the end of May, and its replacement, **statalist.org** is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
Sun Yutao <yutao.sun.statalist@outlook.com> |

To |
<statalist@hsphsun2.harvard.edu> |

Subject |
st: numerical derivatives and -ml- command |

Date |
Mon, 5 Nov 2012 22:26:22 +0100 |

Hello, I'm trying to write an maximum likelihood algorithm that can handle a particularly large number of fixed effects (which is not possible with Stata's #var and matsize limitation). I have been comparing the performance of my ml with the Stata -ml- these days and I find something really strange. More specifically, does anyone know how the Newton-Raphson algorithm in Stata -ml- command works in detail? Because what I see from a small experiment is that the -ml- command is just way too fast so it's not possible to iteratively update the Hessian by numerical differentiation. Here the story goes: For an original Newton-Raphson(nr), one needs to update the gradients and Hessian in each iteration. And suppose we have a likelihood function(llf), 5 variables, and a given number of observations. And a single evaluation of the llf takes 0.0475 seconds, so for a 2-point approximation of the gradient vector one need 5*2=10 evaluations, which gives roughly 0.475 seconds. And the Hessian is a bit complicated: every off-diagonal element will need 4 llf evaluations and you have 4+3+2+1=10 of them, while the diagonal elements need 2 evaluations each, plus 1 for the common f(x), which gives 5*2+1 llf evaluations and hence for the Hessian one needs 10*4+5*2+1=51 llf evaluations which gives 2.4225 seconds. Based on that, one iterations in this setting should roughly take 3 seconds. However, the Stata -ml- command only needs 1 second per iteration, which is particularly uncommon. And even if my calculations for the Hessian was wrong, I'm still convinced that the Hessian needs more llf evaluations and hence more time to compute than the gradients. So it cannot possibly be 1 second per iteration. And in fact, in terms of performances, the Stata nr behaves a little bit like a quasi-Newton + a line search... Or is It possible that Stata secretly have a feature that can take analytical derivatives on a user-specified function? Does anyone know the answer to this problem? Or if there just is a better way to compute numerical derivatives that I do not know? Best regards, Sun Yutao * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/faqs/resources/statalist-faq/ * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**Re: st: numerical derivatives and -ml- command***From:*Stas Kolenikov <skolenik@gmail.com>

**Re: st: numerical derivatives and -ml- command***From:*Maarten Buis <maartenlbuis@gmail.com>

- Prev by Date:
**st: Dynamic factor model** - Next by Date:
**Re: st: xtlogit, margins** - Previous by thread:
**st: Lerner Index using Stata** - Next by thread:
**Re: st: numerical derivatives and -ml- command** - Index(es):