help _mtest undocumented
-------------------------------------------------------------------------------
Title
[P] _mtest -- Adjustment for multiple (simultaneous) testing
Syntax
_mtest query
_mtest syntax, mtest[(name)]
_mtest adjust matname, mtest[(name)] [pindex(#) if(#) replace append]
_mtest footer col type [symbol]
Description
When testing multiple hypotheses simultaneously based on a series of
p-values for tests of the individual hypotheses, adjustments should be
made to these p-values to bound the probability of falsely accepting at
least one of these hypotheses (see Miller 1986 for details). _mtest
implements several adjustment methods and is designed to easily
incorporate others.
_mtest has the following subcommands:
_mtest query
returns in r(methods) the names of the supported methods.
Programmers can use this feature to produce messages (for
instance, an error message) that list the available methods for
the mtest() option that they are adding to their command. When
new methods are added to _mtest, programs that use the results of
_mtest query will automatically have those new methods listed.
_mtest syntax
returns in r(method) the unabbreviated name of the method implied
by name. It returns noadjust if mtest was specified without an
argument, it returns nothing if mtest was not specified, and it
displays an error message otherwise.
_mtest adjust
computes p-values that are adjusted for multiple (simultaneous)
testing, based on p-values of n tests (p_1, ..., p_n), stored in
a column of a matrix. It expects an n x k matrix and returns in
r(result) a column vector of adjusted p-values. Alternatively, it
returns the input matrix with the adjusted p-values replacing the
original p-values (option replace) or appended as an extra column
(option append).
_mtest footer
displays a footer for a table describing how the p-values were
adjusted. col specifies the column to which the message should be
right aligned. symbol is a character used to display what
p-values were adjusted.
Options
mtest(name) specifies the adjustment method. The following methods are
supported:
bonferroni (Bonferroni's method)
p' = min(1, np)
holm (Holm's method)
p' = min(1,n(p)*p), where n(p) is the number of p-values
smaller than p
sidak (Sidak's method)
p' = 1 - (1-p)^n
noadjust
p' = p; i.e., no adjustment is made
Caller commands should typically allow option mtest to be specified
without an argument, implying that multiple testing be performed but
that the p-values not be adjusted for multiple testing. This option
is equivalent to specifying mtest(noadjust). _mtest implements this
behavior as well.
pindex(#) specifies the index of the column in matname that contains the
p-values.
if(#) specifies the index of the column in matname that indicates which
rows of matname are to be included or excluded in the computations.
Those rows where the value in this column is unequal to 0 and not
missing are included. The elements in the returned matrix that
correspond to excluded tests are set to missing. If if is not
specified, all rows are used.
replace specifies that _mtest returns in r(result) the matrix matname
with the column of p-values replaced by the adjusted p-values.
append specifies that _mtest returns in r(result) the matrix matname with
a column of adjusted p-values added at the end of the matrix.
Example
The following code fragment demonstrates how to write a command that
supports multiple testing via _mtest. This code need not be modified in
any way if new methods for dealing with multiple testing are added in
_mtest ("inheritance").
program define mycmd, rclass
syntax ... [, Mtest Mtest2(passthru) ... ]
_mtest syntax, `mtest2' `mtest'
local mtest `r(method)'
if "`mtest'" != "" {
...
tempname tests
mat `tests' = J(n,3,0)
...
fill the matrix tests, with p-value in column 3
_mtest adjust `tests', pindex(3) replace
displays multiple tests, with adjusted p-value in column 3
mat return test r(result)
}
do any further computations
and display simultaneous test results
end
Beware that you specify the options Mtest and Mtest2(passthru) in this
order. The trick to implementing options that take an optional argument
is to specify the argument-accepting option after the nonaccepting
option. If you list them in the other order, the code will not work.
The _mtest syntax subcommand is provided so that the correct syntax of
the mtest or mtest2() option can be verified before indulging in possibly
time-consuming computations for the multiple tests. If early syntax
checking is not performed, _mtest adjust will still capture the problem,
but only after a delay that may irritate the user.
Acknowledgment
_mtest was written by Jeroen Weesie, Department of Sociology, Utrecht
University, The Netherlands.
References
Benjamini, Y., and Y. Hochberg. 1995. Controlling the false discovery
rate: A practical and powerful approach to multiple testing. Journal
of the Royal Statistical Society, Series B 57: 289-300.
Holm, S. 1979. A simple sequentially rejective multiple test procedure.
Scandinavian Journal of Statistics 6: 65-70.
Jaccard, J., M. Becker, and G. Wood. 1984. Pairwise multiple
comparisons: A review. Psychological Bulletin 96: 589-596.
Miller, Rupert G., Jr. 1986. Simultaneous Statistical Inference. 2nd
ed. New York: Springer.
Shaffer J. P. 1986. Modified sequentially rejective multiple test
procedures. Journal of the American Statistical Association 81:
826-831.
Wright, S. P. 1992. Adjusted P-values for simultaneous inference.
Biometrics 48: 1005-1013.
Also see
Manual: undocumented
Help: [R] oneway; [R] hettest, [R] test, [R] testnl
