Statalist The Stata Listserver


[Date Prev][Date Next][Thread Prev][Thread Next][Date index][Thread index]

RE: st: Bonferroni and Sidak adjusted p-values


From   "Steichen, Thomas J." <SteichT@rjrt.com>
To   <statalist@hsphsun2.harvard.edu>
Subject   RE: st: Bonferroni and Sidak adjusted p-values
Date   Thu, 9 Nov 2006 13:14:28 -0500

Thanks Ken,

Getting that issued straightened out let me complete a program 
to "line" multiple comparison results in the classical manner. 

The sample below shows the output for the example data shown in 
my previous message. Here, the lines under "Groups" indicate 
groups (ordered by mean value) of treatments that do not differ 
significantly from each other. Thus there are two groups, AE-AD 
and AD-AC, wherein the treatments do not differ significantly 
from each other based on the Sidak-adjusted p. By definition, 
treatment AF differs from all other treatments, as it is not 
included in a group.

.. mc_lines vol, by(id) sidak sd

Multiple comparison t-tests at p = .05, adjustment method: Sidak

vol: Volume
----------+----------+-----+-----------
     Mean :       SD :  ID : Groups
----------+----------+-----+-----------
    856.4 :   183.51 :  AF :     
    931.3 :   193.51 :  AE : |   
    934.4 :   207.72 :  AA : |   
    953.3 :   201.67 :  AB : |   
   1003.5 :   212.35 :  AD : | | 
   1049.2 :   232.39 :  AC :   | 
----------+----------+-----+-----------

As others have before me, I believe this is a much more useful
and interpretable presentation format for such data. 

An initial version is available to any Statalisters who would be 
willing to test it and provide feedback; just drop me a note. It will
do lining for Sidak-, Bonferroni- and Scheffe-adjusted, as well as 
unadjusted multiple t-tests from a oneway design.

Tom Steichen

Ken Higbee <khigbee@stata.com> replied in part:
> > I have noted on a number of occasions that the Bonferroni and Sidak 
> > adjusted p-values from multiple comparisons by the -oneway- command 
> > differ slightly (but more than simple rounding error) from what my 
> > hand calculations suggest. 
> >
> >                           Comparison of Volume by ID
> >                                    (Sidak)
> > Row Mean-|
> > Col Mean |         AA         AB         AC         AD         AE
> > ---------+-------------------------------------------------------
> >       AB |    18.9015
> >          |      1.000
> >          |
> >       AC |    114.777    95.8757
> >          |      0.000      0.002
> >          |
> >       AD |    69.0773    50.1758      -45.7
> >          |      0.092      0.494      0.655
> >          |
> >       AE |   -3.07991   -21.9814   -117.857   -72.1572
> >          |      1.000      0.999      0.000      0.057
> >          |
> >       AF |   -77.9796   -96.8811   -192.757   -147.057   -74.8997
> >          |      0.023      0.001      0.000      0.000      0.030
> >
> > If someone can confirm this problem -- or point out the
> > error in my calculations -- I would appreciate it.
> 
> You are using 132+132-2 as the degrees of freedom.  However,
> s=sqrt(within MS) is computed from the pooled groups (all 6
> groups in your case) and the degrees of freedom for that pooled s
> is 823.  So for your example, to get the Sidak adjusted p-value,
> you would
> 
>     . di min(1, 1-(1-ttail(823,
>            69.0773/(sqrt(42178.2188)*sqrt(1/132+1/132)))*2)^15)
>     .09208965
> 
> which agrees with what -oneway- produced for the Sidak adjusted
> p-value for the comparison of AA and AD.  Likewise for the
> Bonferroni adjusted p-value, you need to use 823 instead of
> 132+132-2 for the degrees of freedom in your call to -ttail()-.
> For Scheffe you were already using the 823 degrees of freedom in
> the call to -Ftail()-.  That is why it was in agreement with
> -oneway-s output.
 

-----------------------------------------
CONFIDENTIALITY NOTE: This e-mail message, including any
attachment(s), contains information that may be confidential,
protected by the attorney-client or other legal privileges, and/or
proprietary non-public information. If you are not an intended
recipient of this message or an authorized assistant to an intended
recipient, please notify the sender by replying to this message and
then delete it from your system. Use, dissemination, distribution,
or reproduction of this message and/or any of its attachments (if
any) by unintended recipients is not authorized and may be
unlawful.


*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/



© Copyright 1996–2014 StataCorp LP   |   Terms of use   |   Privacy   |   Contact us   |   What's new   |   Site index