Home  /  Products  /  Stata 18  /  Corrected and consistent AICs

<- See Stata 18's new features

Highlights

• Model selection

• Corrected AIC

• Consistent AIC

By popular request, the existing estat ic and estimates stats commands now support two new model-selection criteria: corrected Akaike information criterion (AICc) and consistent AIC (CAIC). The new option all displays all available information criteria. The new option df() specifies the degrees of freedom to compute the information criteria.

Model selection is fundamental to any statistical analysis, and information criteria have been and remain some of the more common statistical techniques for model selection. In Stata, after any estimation command that reports a log likelihoood, which includes most estimation commands, simply type

. estat ic, aiccorrected


or

. estat ic, aicconsistent


0 to compute AICc or CAIC, respectively.

To report all four information criteria (AIC, BIC, AICc, and CAIC), type

. estat ic, all


Sometimes, in analyses such as linear mixed models, we need to manually specify the degrees of freedom or the number of observations to be used in the calculation of the criterion. We can do this by specifying options n() and df():

. estat ic, n(500) df(10) all


These same new criteria and options are also available with the estimates stats command.

##### Using information criteria for a small sample size

We start by exploring information criteria for a dataset with a small sample size. In such datasets, AICc is considered a more reliable criterion than AIC. We compare two multinomial models for insurance type without and with the site dummy variable. The dummy variable site indicates the site of study. We also include the age<30 condition to reduce the sample size to only 87 observations.

. webuse sysdsn1
(Health Insurance data)

. mlogit insure age i.male i.nonwhite if age < 30
(output omitted)

. estat ic, all

Information criteria

Model            N   ll(null)  ll(model)        df

.           87  -76.93025  -70.36684         8

Note: BIC, AICc, and CAIC use N = number of observations.
See [R] IC note.

Model            AIC          BIC          AICc          CAIC

.       156.7337     176.4609      158.5798      184.4609

Legend: AIC is Akaike's information criterion.
BIC is Bayesian information criterion.
AICc is corrected Akaike's information criterion.
CAIC is consistent Akaike's information criterion.

. mlogit insure age i.male i.nonwhite i.site if age < 30
(output omitted)

. estat ic, all

Information criteria

Model            N   ll(null)  ll(model)        df

.           87  -76.93025  -66.03298        12

Note: BIC, AICc, and CAIC use N = number of observations.
See [R] IC note.

Model            AIC          BIC          AICc          CAIC

.        156.066     185.6569      160.2822      197.6569

Legend: AIC is Akaike's information criterion.
BIC is Bayesian information criterion.
AICc is corrected Akaike's information criterion.
CAIC is consistent Akaike's information criterion.


The AIC suggests that the model with the site dummies is preferred, whereas the AICc suggests the opposite.

##### Specifying degrees of freedom

As we mentioned earlier, when fitting linear mixed models using restricted maximum likelihood, care must be taken when comparing models, especially when the fixed-effects specification differs across models. We show how to use the n() and df() options to modify the default values of the number of observations and degrees of freedom that are used for the computation of information criteria. Suppose we want to compare the following two models:

. webuse productivity
(Public capital productivity)

. mixed gsp private emp hwy water other unemp || region: || state:, reml
(output omitted)

. estimates store model1

. mixed gsp private emp hwy unemp || region: hwy || state: unemp, reml
(output omitted)

. estimates store model2


The two models differ in both their fixed-effects and random-effects specifications. Therefore, it is not reliable to compare models with the standard information criteria. Below, we manually specify n() and df() to make models comparable. For each model, the value of n() is computed by subtracting the number of fixed-effects parameters from the number of observations, and df() indicates the number of random-effects parameters.

. estimates restore model1
(results model1 are active now)

. estat ic, n(809) df(3)

Akaike's information criterion and Bayesian information criterion

Model            N   ll(null)  ll(model)      df        AIC        BIC

model1          809          .    1404.71       3   -2803.42  -2789.333

. estimates restore model2
(results model2 are active now)

. estat ic, n(811) df(5)

Akaike's information criterion and Bayesian information criterion

Model            N   ll(null)  ll(model)      df        AIC        BIC

model2          811          .   1413.557       5  -2817.114  -2793.623



Both AIC and BIC indicate that the second model is preferable.

#### Tell me more

Read more about corrected AIC and consistent AIC in the Stata Base Reference Manual; see [R] estat ic.

View all the new features in Stata 18.