Why should I not do a likelihood-ratio test after an ML estimation
(e.g., logit, probit) with clustering or pweights?
| Title |
|
Likelihood-ratio test after survey/robust ML estimation |
| Author |
William Sribney, StataCorp
|
| Date |
August 1997; updated September 2005
|
The “likelihood” for pweighted or clustered MLEs is not a true
likelihood; i.e., it is NOT the distribution of the sample. When there is
clustering, individual observations are no longer independent, and the
“likelihood” does not reflect this. Where there are pweights,
the “likelihood” does not fully account for the
“randomness” of the weighted sampling.
The “likelihood” for pweighted or clustered MLEs is used only
for the computation of the point estimates and should not be used for
variance estimation using standard formulas. Thus the standard
likelihood-ratio test should NOT be used after estimating pweighted or
clustered MLEs.
Instead of likelihood-ratio tests (the
lrtest command), Wald tests (the
test command) should be used.
The svy commands
allow the use of the
test command, which
computes an adjusted Wald test. This adjustment is useful when the total
number of clusters is small (<∼100).
The test command also has a mtest(bonferroni) option. Some
statisticians argue that the mtest(bonferroni) option of test
gives a better test than an adjusted Wald test. See
Korn, E. L., and B. I. Graubard. 1990.
Simultaneous testing of regression coefficients with complex survey
data: use of Bonferroni t statistics.
The American Statistician 44: 270–276.
for such an argument.
The Bonferroni adjustment carries the tacit assumption that the
multidimensional test (let k = the dimension) being conducted
consists of k hypotheses that individually make sense as different
research questions—or said more precisely, a priori you should have no
knowledge that the individual hypotheses are highly collinear. If the
individual hypotheses ARE highly collinear, then the Bonferroni adjustment
can be overly conservative. So if you suspect this in advance, you might
want to stay away from the Bonferroni adjustment. But, in statistics at
least, being conservative is safe, so doing the Bonferroni adjustment should
be fine.
|