# RE: st: Numerous t-tests

 From Roger Newson To statalist@hsphsun2.harvard.edu Subject RE: st: Numerous t-tests Date Wed, 26 Oct 2005 13:01:54 +0100

More about multiple-testing procedures in Stata can be found in Newson (2003), which you can download in preprint .pdf form from within Stata by typing

net describe multproc, from(http://www.kcl-phs.org.uk/rogernewson/papers)

and downloading the ancillary file -multproc.pdf-. This reference contains an example of the use of multiple-test procedures with -somersd-.

I hope this helps.

Roger

References

Newson, R. and the ALSPAC Study Team. 2003. Multiple-test procedures and smile plots.
The Stata Journal 3(2): 109-132. Also downloadable from Roger Newson's website at
http://www.kcl-phs.org.uk/rogernewson.

At 12:07 26/10/2005, Maarten wrote:

The probable reason that Stata did not include a canned solution to your problem is that multiple testing like this has the danger of capitalizing on chance. A significance level of 5% means that you will find a significant result that is not really there - a false positive -- in 5% of all tests you perform. If you run 20 tests you will, on average, find 1 bogus effect. Running many tests and keeping the significant ones will thus invalidate your inference. The probability of a false positive is no longer 5%, but much higher. So, you should consider doing a bonferroni correction of your significance levels. The bonferroni correction means that the significance level should be divided by the number of tests. So if you choose a significance level of 5% and do 10 tests, the corrected significance level should be 0.5%. This is a somewhat conservative correction and less conservative alternatives exist like holm's or sidak's method.

Hope this helps,
Maarten

-----Original Message-----
From: owner-statalist@hsphsun2.harvard.edu [mailto:owner-statalist@hsphsun2.harvard.edu]On Behalf Of Fred Wolfe
Sent: woensdag 26 oktober 2005 12:42
To: statalist@hsphsun2.harvard.edu
Subject: Re: st: Numerous t-tests

You can do it with the -foreach- command, as in ...

foreach var of varlist age duration fatig mood {
ttest `var',by(nodmardbio)
}

Fred

At 05:32 AM 10/26/2005, you wrote:
>I have what should be a very simple question, but I haven't been able to
>figure out the answer. I want to do group mean comparison t-tests and
>rank sum tests on a number of different variables.. But, it appears that
>both -ttest- and -ranksum- will only do 1 variable at a time. So, is
>there a single command that will let me put in any number of variables
>(along with the 2-level grouping variable, of course) and will then do ALL
>the tests at once?
>
>Thanks.
>
>Allan
>
>
>*
>* For searches and help try:
>* http://www.stata.com/support/faqs/res/findit.html
>* http://www.stata.com/support/statalist/faq
>* http://www.ats.ucla.edu/stat/stata/

Fred Wolfe
National Data Bank for Rheumatic Diseases
Wichita, Kansas
Tel (316) 263-2125 Fax (316) 263-0761
fwolfe@arthritis-research.org

*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/

*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
```
--
Roger Newson
Lecturer in Medical Statistics
Department of Public Health Sciences
Division of Asthma, Allergy and Lung Biology
King's College London

5th Floor, Capital House
42 Weston Street
London SE1 3QD
United Kingdom

Tel: 020 7848 6648 International +44 20 7848 6648
Fax: 020 7848 6620 International +44 20 7848 6620
or 020 7848 6605 International +44 20 7848 6605
Email: roger.newson@kcl.ac.uk
Website: http://phs.kcl.ac.uk/rogernewson/

Opinions expressed are those of the author, not the institution.

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
```