Stata The Stata listserver
[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

Re: st: newbie question: nonsig posthoc after sig anova

From   Roger Newson <>
Subject   Re: st: newbie question: nonsig posthoc after sig anova
Date   Mon, 18 Aug 2003 18:44:11 +0100

At 08:29 18/08/03 -0700, Jim Seward wrote:
I'm a new member to the list.  My background is in neuropsychology rather
than statistics so please excuse this naive question.  I did a simple ANOVA
comparing a mean score across five groups.  The ANOVA was barely
significanct (0.04) but in post hoc tests using Bonferroni, Scheffe and
Sida none of the group mean differences were near significant (the largest
group mean difference had a p of 0.14).

Any explanation is appreciated.
An F-test comparing 5 groups is implicitly defining a 95% confidence region, in 4-dimensional "hyperspace", for 4 mean differences between the 4 non-reference groups and the reference group. Jim's "barely significant" P-value implies that this confidence region does not contain the vector of 4 zero mean differences, which would be the true population mean differences if all 5 groups had the same mean. However, the post-hoc tests seem to imply that the confidence region for the 4 mean differences might contain zero values for any of the 4 mean differences individually. Therefore, although it seems that at least one difference is non-zero, Jim has insufficient data to incriminate one of these differences as being "the culprit".

Jim doesn't say what the ANOVA is about. However, most statisticians nowadays, most of the time, prefer confidence intervals to P-values alone, because P-values only measure the compatibility of the data with zero population differences, and do not give a range of positive and/or negative and/or zero population differences with which the data ARE compatible. A good introduction to confidence intervals, commonly used in the medical sector, is Altman et al. (2000).

I hope this helps.



Douglas Altman, David Machin, Trevor Bryant, Martin Gardner. Statistics with Confidence. London: British Medical Journal Books; 2000. ISBN: 0727913751

Roger Newson
Lecturer in Medical Statistics
Department of Public Health Sciences
King's College London
5th Floor, Capital House
42 Weston Street
London SE1 3QD
United Kingdom

Tel: 020 7848 6648 International +44 20 7848 6648
Fax: 020 7848 6620 International +44 20 7848 6620
or 020 7848 6605 International +44 20 7848 6605

Opinions expressed are those of the author, not the institution.

* For searches and help try:

© Copyright 1996–2017 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index