Notice: On March 31, it was **announced** that Statalist is moving from an email list to a **forum**. The old list will shut down on April 23, and its replacement, **statalist.org** is already up and running.

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

From |
Roger Newson <r.newson@imperial.ac.uk> |

To |
"statalist@hsphsun2.harvard.edu" <statalist@hsphsun2.harvard.edu> |

Subject |
Re: st: strange -multproc- results |

Date |
Wed, 17 Mar 2010 20:28:49 +0000 |

I hope this helps. Best wishes Roger References

http://www.stata-journal.com/article.html?article=st0035 On 17/03/2010 20:07, Feiveson, Alan H. (JSC-SK311) wrote:

Hi - I have been using -multproc- to control the false discovery rate (FDR) on 27 significance tests. As given in the help file, there are several methods to chose from for controlling the FDR: liu1,liu2,simes,yekutieli, and krieger. method() Step type FWER/FDR Definition or source userspecified One-step Either pcor() option bonferroni One-step FWER pcor=puncor/m sidak One-step FWER pcor=1-(1-puncor)^(1/m) (or Sidak, 1967) holm Step-down FWER Holm, 1979 holland Step-down FWER Holland and Copenhaver, 1987 liu1 Step-down FDR Benjamini and Liu, 1999a liu2 Step-down FDR Benjamini and Liu, 1999b hochberg Step-up FWER Hochberg, 1988 rom Step-up FWER Rom, 1990 simes Step-up FDR Benjamini and Hochberg, 1995 (or Benjamini and Yekutieli, 2001 (first method)) yekutieli Step-up FDR Benjamini and Yekutieli, 2001 (second method) krieger Step-up FDR Benjamini, Krieger and Yekutieli, 2001 So I tried liu1, liu2, simes, yekutieli, and krieger to see what difference it would make with a specified FDR of 0.05. The two liu's and yekutieli were about the same (4 rejections, critical P-value about 0.002. But the simes and krieger were completely different (simes: 14 rejections, critical p-vlaue = 0.026) and (krieger: 15 rejections, critical p-value = 0.051). The latter two look too good to be true, especially the Krieger, where the critical p-value is actually higher than the specified FDR rate. Anyone know what's going on here? Am I doing this correctly? What assumptions are there for Krieger, for example, that do not hold for the first three? Al Feiveson If anyone wants to try it - here's the data: h se z pv 1761.754 419.83 4.196352 .0000271 .0613758 .0171379 3.58129 .0003419 .0431283 .0134256 3.212402 .0013163 .0503242 .0159218 3.160711 .0015738 .0662939 .0223807 2.962102 .0030555 .0388915 .0133944 2.903562 .0036894 .0793423 .0274955 2.885645 .0039061 .0353006 .0129654 2.722682 .0064754 868.2667 323.8542 2.681042 .0073393 .0491057 .0184865 2.6563 .0079003 893.4875 341.7166 2.614703 .0089305 .0310786 .0131919 2.355878 .018479 .034885 .0150222 2.322223 .0202209 .032349 .0144647 2.236412 .0253248 .0302972 .0139816 2.166937 .0302396 .0295493 .0165115 1.789618 .0735153 -.0201654 .0129031 -1.562831 .1180923 .0255772 .017776 1.438857 .1501909 .0150236 .0122768 1.223738 .2210511 .0187261 .0165989 1.128154 .2592548 -.013579 .0127488 -1.065118 .2868224 .0142208 .0142817 .9957331 .3193798 .0099778 .0117025 .8526207 .3938697 .0071485 .0095379 .7494824 .4535665 .0130484 .021209 .6152284 .5384039 -.0067718 .0137157 -.4937261 .6214996 .0028293 .0090162 .3138045 .7536695 . multproc ,method(liu1) pvalue(pv) puncor(.05) Method: liu1 Uncorrected overall critical P-value: .05 Number of P-values: 27 Corrected overall critical P-value: .00262649 Number of rejected P-values: 4 . multproc ,method(liu2) pvalue(pv) puncor(.05) Method: liu2 Uncorrected overall critical P-value: .05 Number of P-values: 27 Corrected overall critical P-value: .00255198 Number of rejected P-values: 4 . multproc ,method(simes) pvalue(pv) puncor(.05) Method: simes Uncorrected overall critical P-value: .05 Number of P-values: 27 Corrected overall critical P-value: .02592593 Number of rejected P-values: 14 . multproc ,method(yekutieli) pvalue(pv) puncor(.05) Method: yekutieli Uncorrected overall critical P-value: .05 Number of P-values: 27 Corrected overall critical P-value: .00190351 Number of rejected P-values: 4 . multproc ,method(krieger) pvalue(pv) puncor(.05) Method: krieger Uncorrected overall critical P-value: .05 Number of P-values: 27 Corrected overall critical P-value: .05102041 Number of rejected P-values: 15 * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

-- Roger B Newson BSc MSc DPhil Lecturer in Medical Statistics Respiratory Epidemiology and Public Health Group National Heart and Lung Institute Imperial College London Royal Brompton Campus Room 33, Emmanuel Kaye Building 1B Manresa Road London SW3 6LR UNITED KINGDOM Tel: +44 (0)20 7352 8121 ext 3381 Fax: +44 (0)20 7351 8322 Email: r.newson@imperial.ac.uk Web page: http://www.imperial.ac.uk/nhli/r.newson/ Departmental Web page: http://www1.imperial.ac.uk/medicine/about/divisions/nhli/respiration/popgenetics/reph/ Opinions expressed are those of the author, not of the institution. * * For searches and help try: * http://www.stata.com/help.cgi?search * http://www.stata.com/support/statalist/faq * http://www.ats.ucla.edu/stat/stata/

**Follow-Ups**:**st: tabulate, summarize() for svy data***From:*Ricardo Ovaldia <ovaldia@yahoo.com>

**RE: st: strange -multproc- results***From:*"Feiveson, Alan H. (JSC-SK311)" <alan.h.feiveson@nasa.gov>

**References**:**st: strange -multproc- results***From:*"Feiveson, Alan H. (JSC-SK311)" <alan.h.feiveson@nasa.gov>

- Prev by Date:
**Re: st: RE: Moving legends in combined graphs** - Next by Date:
**Re: st: is it possible to write explicit equation, GMM estimation with constraints ??** - Previous by thread:
**st: strange -multproc- results** - Next by thread:
**RE: st: strange -multproc- results** - Index(es):