[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: Re: ML estimation and gradient

From   "Stas Kolenikov" <>
Subject   Re: st: Re: ML estimation and gradient
Date   Tue, 21 Aug 2007 10:33:07 -0500

Moreover, as long as -ml- needs to take numerical derivatives in a lot
of MLE routines (I'd be curious to see the breakdown, if one is
available, of the official routines -- my understanding is that 3/4 or
so would be -ml lf-), it needs to increase the steps sizes thus
sacrificing the precision even further -- when computing numerical
derivatives, one has to sacrifice a few orders of magnitude in
precision. The [ML] book from Stata Corp explains it all real well --

On 8/21/07, Kit Baum <> wrote:
> In any numerical optimization procedure with numerical derivatives,
> the derivative is an approximation to the slope along an
> infinitesimal segment of the function. If you took an arbitrarily
> small epsilon around the optimum, and the optimum was expressed to a
> precision beyond 15 digits, you would get something closer to zero.
> In something like -ml-, applying a tighter convergence criterion
> (i.e. the norm of the gradient must be no more than 10^-8) you may
> not find an optimum at all, or it may take a long time. Thus any
> optimization routine trades off precision for speed and likelihood of
> convergence. Stata's behavior in this regard is similar to that of
> any other software I have used.
> Kit Baum, Boston College Economics and DIW Berlin
> An Introduction to Modern Econometrics Using Stata:
> On Aug 21, 2007, at 2:33 AM, statalist-digest wrote:
> > I would like to know why, when using maximum likelihood estimation
> > in Stata,
> >   the gradient in the last iteration is often numerically different
> > from
> > zero whereas it should be theoretically equal to zero.
> *
> *   For searches and help try:
> *
> *
> *

Stas Kolenikov, also found at

Small print: Please do not reply to my Gmail address as I don't check
it regularly.
*   For searches and help try:

© Copyright 1996–2015 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index