Stata 15 help for _mtest


[P] _mtest -- Adjustment for multiple (simultaneous) testing


_mtest query

_mtest syntax, mtest[(name)]

_mtest adjust matname, mtest[(name)] [pindex(#) if(#) replace append]

_mtest footer col type [symbol]


When testing multiple hypotheses simultaneously based on a series of p-values for tests of the individual hypotheses, adjustments should be made to these p-values to bound the probability of falsely accepting at least one of these hypotheses (see Miller [1986] for details). _mtest implements several adjustment methods and is designed to easily incorporate others.

_mtest has the following subcommands:

_mtest query returns in r(methods) the names of the supported methods. Programmers can use this feature to produce messages (for instance, an error message) that list the available methods for the mtest() option that they are adding to their command. When new methods are added to _mtest, programs that use the results of _mtest query will automatically have those new methods listed.

_mtest syntax returns in r(method) the unabbreviated name of the method implied by name. It returns noadjust if mtest was specified without an argument, it returns nothing if mtest was not specified, and it displays an error message otherwise.

_mtest adjust computes p-values that are adjusted for multiple (simultaneous) testing, based on p-values of n tests (p_1, ..., p_n), stored in a column of a matrix. It expects an n x k matrix and returns in r(result) a column vector of adjusted p-values. Alternatively, it returns the input matrix with the adjusted p-values replacing the original p-values (option replace) or appended as an extra column (option append).

_mtest footer displays a footer for a table describing how the p-values were adjusted. col specifies the column to which the message should be right aligned. symbol is a character used to display what p-values were adjusted.


mtest(name) specifies the adjustment method. The following methods are supported:

bonferroni (Bonferroni's method)

p' = min(1, np)

holm (Holm's method)

p' = min(1,n(p)*p), where n(p) is the number of p-values smaller than p

sidak (Sidak's method)

p' = 1 - (1-p)^n


p' = p; that is, no adjustment is made

Caller commands should typically allow option mtest to be specified without an argument, implying that multiple testing be performed but that the p-values not be adjusted for multiple testing. This option is equivalent to specifying mtest(noadjust). _mtest implements this behavior as well.

pindex(#) specifies the index of the column in matname that contains the p-values.

if(#) specifies the index of the column in matname that indicates which rows of matname are to be included or excluded in the computations. Those rows where the value in this column is unequal to 0 and not missing are included. The elements in the returned matrix that correspond to excluded tests are set to missing. If if is not specified, all rows are used.

replace specifies that _mtest returns in r(result) the matrix matname with the column of p-values replaced by the adjusted p-values.

append specifies that _mtest returns in r(result) the matrix matname with a column of adjusted p-values added at the end of the matrix.


The following code fragment demonstrates how to write a command that supports multiple testing via _mtest. This code need not be modified in any way if new methods for dealing with multiple testing are added in _mtest ("inheritance").

program define mycmd, rclass syntax ... [, Mtest Mtest2(passthru) ... ]

_mtest syntax, `mtest2' `mtest' local mtest `r(method)'

if "`mtest'" != "" { ... tempname tests mat `tests' = J(n,3,0) ...

fill the matrix tests, with p-value in column 3

_mtest adjust `tests', pindex(3) replace displays multiple tests, with adjusted p-value in column 3

mat return test r(result) }

do any further computations and display simultaneous test results


Beware that you specify the options Mtest and Mtest2(passthru) in this order. The trick to implementing options that take an optional argument is to specify the argument-accepting option after the nonaccepting option. If you list them in the other order, the code will not work.

The _mtest syntax subcommand is provided so that the correct syntax of the mtest or mtest2() option can be verified before indulging in possibly time-consuming computations for the multiple tests. If early syntax checking is not performed, _mtest adjust will still capture the problem, but only after a delay that may irritate the user.


_mtest was written by Jeroen Weesie, Department of Sociology, Utrecht University, The Netherlands.


Benjamini, Y., and Y. Hochberg. 1995. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B 57: 289-300.

Holm, S. 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6: 65-70.

Jaccard, J., M. Becker, and G. Wood. 1984. Pairwise multiple comparisons: A review. Psychological Bulletin 96: 589-596.

Miller, R. G., Jr. 1986. Simultaneous Statistical Inference. 2nd ed. New York: Springer.

Shaffer J. P. 1986. Modified sequentially rejective multiple test procedures. Journal of the American Statistical Association 81: 826-831.

Wright, S. P. 1992. Adjusted P-values for simultaneous inference. Biometrics 48: 1005-1013.

© Copyright 1996–2018 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index