Search
   >> Home >> Products >> Features >> Maximum likelihood >> Maximum likelihood estimation

Maximum likelihood estimation

In addition to providing built-in commands to fit many standard maximum likelihood models, such as logistic, Cox, Poisson, etc., Stata can maximize user-specified likelihood functions. To demonstrate, say Stata could not fit logistic regression models. The logistic likelihood function is

    f(y, Xb) = 1/(1+exp(-Xb))                 if y = 1
             = exp(-Xb)/(1+exp(-Xb))          if y = 0

We might first write a program in Stata to calculate the log of the likelihood function given y ($ML_y1 in the code below) and Xb:

program define mylogit args lnf Xb replace `lnf' = -ln(1+exp(-`Xb')) if $ML_y1==1 replace `lnf' = -`Xb' - ln(1+exp(-`Xb')) if $ML_y1==0 end

That done, we can fit a logistic-regression model of dependent variable foreign on mpg and displ by typing

. ml model lf mylogit (foreign=mpg weight) . ml maximize

You will be surprised when you see the output:

. ml model lf mylogit (foreign=mpg weight) . ml maximize initial: log likelihood = -51.292891 alternative: log likelihood = -46.081697 rescale: log likelihood = -45.181365 Iteration 0: log likelihood = -45.181365 Iteration 1: log likelihood = -29.420506 Iteration 2: log likelihood = -27.210884 Iteration 3: log likelihood = -27.175196 Iteration 4: log likelihood = -27.175156 Iteration 5: log likelihood = -27.175156 Number of obs = 74 Wald chi2(2) = 17.78 Log likelihood = -27.175156 Prob > chi2 = 0.0001
foreign Coef. Std. Err. z P>|z| [95% Conf. Interval]
mpg -.1685869 .0919175 -1.83 0.067 -.3487418 .011568
weight -.0039067 .0010116 -3.86 0.000 -.0058894 -.001924
_cons 13.70837 4.518709 3.03 0.002 4.851859 22.56487

Stata automatically generated this neatly formatted output, complete with significance levels and confidence intervals.

The following features are worth noting:

  • We define a program to calculate the log likelihood generically—in terms of y and Xb.
  • Once the program to calculate the log likelihood has been defined, we can fit any particular model. The syntax of the ml model statement is
    ml model methodname programname ( model )
    To fit a model of foreign on mpg and weight using our program mylogit (and method lf), we typed
    ml model lf mylogit (foreign=mpg weight)
  • We do not have to specify initial values for the parameters, although we may. When we do not specify initial values, ml finds starting values on its own.
  • Stata provides four optimization methods: lf, d0, d1, and d2. Optimization method lf is easiest to use because (1) you do not have to program derivatives and (2) the likelihood is assumed to be of the form L(Xb). Method d0 allows the function to be more general, L(X,b), but similarly requires no programming of derivatives. Method d1 requires programming first derivatives, and method d2 requires programming first and second derivatives.

Stata’s likelihood-maximization procedures have been designed for both quick-and-dirty work and writing prepackaged estimation routines that obtain results quickly and robustly. For instance, Stata fits negative binomial regressions (a variation on Poisson regression) and Heckman selection models. We wrote those routines using Stata's ml command, although most users are not aware of that. They think that negative binomial and Heckman selection are just two more things Stata can do.

If you are serious about maximizing likelihood functions, you will want to obtain the text Maximum Likelihood Estimation with Stata, Fourth Edition by William Gould, Jeffrey Pitblado, and Brian Poi (2010). The first chapter provides a general overview of maximum likelihood estimation theory and numerical optimization methods, with an emphasis on the practical applications of each for applied work. The middle chapters detail, step by step, the use of Stata to maximize user-written likelihood functions. The final chapters explain, for those interested, how to add new estimation commands to Stata.

The Stata Blog: Not Elsewhere Classified Find us on Facebook Follow us on Twitter LinkedIn Google+ Watch us on YouTube