__Title__

**[P] _mtest** -- Adjustment for multiple (simultaneous) testing

__Syntax__

**_mtest** __q__**uery**

**_mtest** __s__**yntax,** __m__**test**[**(***name***)**]

**_mtest** __a__**djust** *matname***,** __m__**test**[**(***name***)**] [__p__**index(***#***)** **if(***#***)** **replace** **append**]

**_mtest** __f__**ooter** *col* *type* [*symbol*]

__Description__

When testing multiple hypotheses simultaneously based on a series of
p-values for tests of the individual hypotheses, adjustments should be
made to these p-values to bound the probability of falsely accepting at
least one of these hypotheses (see Miller [1986] for details). **_mtest**
implements several adjustment methods and is designed to easily
incorporate others.

**_mtest** has the following subcommands:

**_mtest query**
returns in **r(methods)** the names of the supported methods.
Programmers can use this feature to produce messages (for
instance, an error message) that list the available methods for
the **mtest()** option that they are adding to their command. When
new methods are added to **_mtest**, programs that use the results of
**_mtest query** will automatically have those new methods listed.

**_mtest syntax**
returns in **r(method)** the unabbreviated name of the method implied
by *name*. It returns **noadjust** if **mtest** was specified without an
argument, it returns nothing if **mtest** was not specified, and it
displays an error message otherwise.

**_mtest adjust**
computes p-values that are adjusted for multiple (simultaneous)
testing, based on p-values of n tests (p_1, ..., p_n), stored in
a column of a matrix. It expects an n x k matrix and returns in
**r(result)** a column vector of adjusted p-values. Alternatively, it
returns the input matrix with the adjusted p-values replacing the
original p-values (option **replace**) or appended as an extra column
(option **append**).

**_mtest footer**
displays a footer for a table describing how the p-values were
adjusted. *col* specifies the column to which the message should be
right aligned. *symbol* is a character used to display what
p-values were adjusted.

__Options__

**mtest(***name***)** specifies the adjustment method. The following methods are
supported:

__b__**onferroni** (Bonferroni's method)

p' = min(1, np)

__h__**olm** (Holm's method)

p' = min(1,n(p)*p), where n(p) is the number of p-values
smaller than p

__s__**idak** (Sidak's method)

p' = 1 - (1-p)^n

__no__**adjust**

p' = p; that is, no adjustment is made

Caller commands should typically allow option **mtest** to be specified
without an argument, implying that multiple testing be performed but
that the p-values not be adjusted for multiple testing. This option
is equivalent to specifying **mtest(noadjust)**. **_mtest** implements this
behavior as well.

**pindex(***#***)** specifies the index of the column in *matname* that contains the
p-values.

**if(***#***)** specifies the index of the column in *matname* that indicates which
rows of *matname* are to be included or excluded in the computations.
Those rows where the value in this column is unequal to 0 and not
missing are included. The elements in the returned matrix that
correspond to excluded tests are set to missing. If **if** is not
specified, all rows are used.

**replace** specifies that **_mtest** returns in **r(result)** the matrix *matname*
with the column of p-values replaced by the adjusted p-values.

**append** specifies that **_mtest** returns in **r(result)** the matrix *matname* with
a column of adjusted p-values added at the end of the matrix.

__Example__

The following code fragment demonstrates how to write a command that
supports multiple testing via **_mtest**. This code need not be modified in
any way if new methods for dealing with multiple testing are added in
**_mtest** ("inheritance").

**program define mycmd, rclass**
** syntax** *...* **[, Mtest Mtest2(passthru)** *...* **]**

** _mtest syntax, `mtest2' `mtest'**
** local mtest `r(method)'**

** if "`mtest'" != "" {**
*...*
** tempname tests**
** mat `tests' = J(n,3,0)**
*...*

fill the matrix tests, with p-value in column 3

** _mtest adjust `tests', pindex(3) replace**
displays multiple tests, with adjusted p-value in column 3

** mat return test r(result)**
** }**

do any further computations
and display simultaneous test results

**end**

Beware that you specify the options **Mtest** and **Mtest2(passthru)** in this
order. The trick to implementing options that take an optional argument
is to specify the argument-accepting option after the nonaccepting
option. If you list them in the other order, the code will not work.

The **_mtest syntax** subcommand is provided so that the correct syntax of
the **mtest** or **mtest2()** option can be verified before indulging in possibly
time-consuming computations for the multiple tests. If early syntax
checking is not performed, **_mtest adjust** will still capture the problem,
but only after a delay that may irritate the user.

__Acknowledgment__

**_mtest** was written by Jeroen Weesie, Department of Sociology, Utrecht
University, The Netherlands.

__References__

Benjamini, Y., and Y. Hochberg. 1995. Controlling the false discovery
rate: A practical and powerful approach to multiple testing. *Journal*
*of the Royal Statistical Society, Series B* 57: 289-300.

Holm, S. 1979. A simple sequentially rejective multiple test procedure.
*Scandinavian Journal of Statistics* 6: 65-70.

Jaccard, J., M. Becker, and G. Wood. 1984. Pairwise multiple
comparisons: A review. *Psychological Bulletin* 96: 589-596.

Miller, R. G., Jr. 1986. *Simultaneous Statistical Inference*. 2nd ed.
New York: Springer.

Shaffer J. P. 1986. Modified sequentially rejective multiple test
procedures. *Journal of the American Statistical Association* 81:
826-831.

Wright, S. P. 1992. Adjusted P-values for simultaneous inference.
*Biometrics* 48: 1005-1013.