Maximum likelihood estimation

In addition to providing built-in commands to fit many standard maximum likelihood models, such as logistic, Cox, Poisson, etc., Stata can maximize user-specified likelihood functions. To demonstrate, say Stata could not fit logistic regression models. The logistic likelihood function is

    f(y, Xb) = 1/(1+exp(-Xb))                 if y = 1
             = exp(-Xb)/(1+exp(-Xb))          if y = 0

We might first write a program in Stata to calculate the log of the likelihood function given y ($ML_y1 in the code below) and Xb:

  program define mylogit
          args lnf Xb
          replace `lnf' = -ln(1+exp(-`Xb')) if $ML_y1==1
          replace `lnf' = -`Xb' - ln(1+exp(-`Xb')) if $ML_y1==0
  end

That done, we can fit a logistic-regression model of dependent variable foreign on mpg and displ by typing

  . ml model lf mylogit (foreign=mpg weight)
  . ml maximize

You will be surprised when you see the output:

 . ml model lf mylogit (foreign=mpg weight)

 . ml maximize

 initial:       log likelihood = -51.292891
 alternative:   log likelihood = -46.081697
 rescale:       log likelihood = -45.181365
 Iteration 0:   log likelihood = -45.181365
 Iteration 1:   log likelihood = -29.420506
 Iteration 2:   log likelihood = -27.229801
 Iteration 3:   log likelihood = -27.175287
 Iteration 4:   log likelihood = -27.175156
 Iteration 5:   log likelihood = -27.175156

                                                   Number of obs   =         74
                                                   Wald chi2(2)    =      17.78
 Log likelihood = -27.175156                       Prob > chi2     =     0.0001

 ------------------------------------------------------------------------------
      foreign |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
 -------------+----------------------------------------------------------------
          mpg |  -.1685869   .0919175    -1.83   0.067    -.3487418     .011568
       weight |  -.0039067   .0010116    -3.86   0.000    -.0058894    -.001924
        _cons |   13.70837   4.518709     3.03   0.002     4.851859    22.56487
 ------------------------------------------------------------------------------

Stata automatically generated this neatly formatted output, complete with significance levels and confidence intervals.

The following features are worth noting:

• We define a program to calculate the log likelihood generically—in terms of y and Xb.
• Once the program to calculate the log likelihood has been defined, we can fit any particular model. The syntax of the ml model statement is

  ml model methodname programname ( model )

  To fit a model of foreign on mpg and weight using our program mylogit (and method lf), we typed

  ml model lf mylogit (foreign=mpg weight)

• We do not have to specify initial values for the parameters, although we may. When we do not specify initial values, ml finds starting values on its own.
• Stata provides four optimization methods: lf, d0, d1, and d2. Optimization method lf is easiest to use because (1) you do not have to program derivatives and (2) the likelihood is assumed to be of the form L(Xb). Method d0 allows the function to be more general, L(X,b), but similarly requires no programming of derivatives. Method d1 requires programming first derivatives, and method d2 requires programming first and second derivatives.

Stata’s likelihood-maximization procedures have been designed for both quick-and-dirty work and writing prepackaged estimation routines that obtain results quickly and robustly. For instance, Stata fits negative binomial regressions (a variation on Poisson regression) and Heckman selection models. We wrote those routines using Stata's ml command, although most users are not aware of that. They think that negative binomial and Heckman selection are just two more things Stata can do.

If you are serious about maximizing likelihood functions, you will want to obtain the text Maximum Likelihood Estimation with Stata, second edition by William Gould, Jeffrey Pitblado, and William Sribney (2003). The first chapter provides a general overview of maximum likelihood estimation theory and numerical optimization methods, with an emphasis on the practical applications of each for applied work. The middle chapters detail, step by step, the use of Stata to maximize user-written likelihood functions. The final chapters explain, for those interested, how to add new estimation commands to Stata.

See New in Stata 10 for more about what was added in Stata Release 10.