Statalist


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: Re: ML estimation and gradient


From   "Stas Kolenikov" <skolenik@gmail.com>
To   statalist@hsphsun2.harvard.edu
Subject   Re: st: Re: ML estimation and gradient
Date   Tue, 21 Aug 2007 10:33:07 -0500

Moreover, as long as -ml- needs to take numerical derivatives in a lot
of MLE routines (I'd be curious to see the breakdown, if one is
available, of the official routines -- my understanding is that 3/4 or
so would be -ml lf-), it needs to increase the steps sizes thus
sacrificing the precision even further -- when computing numerical
derivatives, one has to sacrifice a few orders of magnitude in
precision. The [ML] book from Stata Corp explains it all real well --
see http://www.stata.com/bookstore/mle.html.

On 8/21/07, Kit Baum <baum@bc.edu> wrote:
> In any numerical optimization procedure with numerical derivatives,
> the derivative is an approximation to the slope along an
> infinitesimal segment of the function. If you took an arbitrarily
> small epsilon around the optimum, and the optimum was expressed to a
> precision beyond 15 digits, you would get something closer to zero.
> In something like -ml-, applying a tighter convergence criterion
> (i.e. the norm of the gradient must be no more than 10^-8) you may
> not find an optimum at all, or it may take a long time. Thus any
> optimization routine trades off precision for speed and likelihood of
> convergence. Stata's behavior in this regard is similar to that of
> any other software I have used.
>
>
> Kit Baum, Boston College Economics and DIW Berlin
> http://ideas.repec.org/e/pba1.html
> An Introduction to Modern Econometrics Using Stata:
> http://www.stata-press.com/books/imeus.html
>
>
> On Aug 21, 2007, at 2:33 AM, statalist-digest wrote:
>
> > I would like to know why, when using maximum likelihood estimation
> > in Stata,
> >   the gradient in the last iteration is often numerically different
> > from
> > zero whereas it should be theoretically equal to zero.
>
> *
> *   For searches and help try:
> *   http://www.stata.com/support/faqs/res/findit.html
> *   http://www.stata.com/support/statalist/faq
> *   http://www.ats.ucla.edu/stat/stata/
>


-- 
Stas Kolenikov, also found at http://stas.kolenikov.name

Small print: Please do not reply to my Gmail address as I don't check
it regularly.
*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index