Statalisters,
I have noted on a number of occasions that the Bonferroni and Sidak
adjusted p-values from multiple comparisons by the -oneway- command
differ slightly (but more than simple rounding error) from what my
hand calculations suggest. Has anyone else noted this?
I'll give some example output below (using Sidak adjustment) and
also show my hand calculations:
.. oneway vol id, sidak t
| Summary of Volume
ID | Mean Std. Dev. Freq.
------------+------------------------------------
AA | 934.42349 207.71981 132
AB | 953.325 201.66785 140
AC | 1049.2007 232.39271 137
AD | 1003.5008 212.34701 132
AE | 931.34357 193.50937 140
AF | 856.44392 183.51134 148
------------+------------------------------------
Total | 953.14089 213.68692 829
Analysis of Variance
Source SS df MS F Prob > F
--------------------------------------------------------------------
Between groups 3095545.81 5 619109.162 14.68 0.0000
Within groups 34712674.1 823 42178.2188
--------------------------------------------------------------------
Total 37808219.9 828 45662.1013
Bartlett's test for equal variances: chi2(5) = 9.3237 Prob>chi2 = 0.097
Comparison of Volume by ID
(Sidak)
Row Mean-|
Col Mean | AA AB AC AD AE
---------+-------------------------------------------------------
AB | 18.9015
| 1.000
|
AC | 114.777 95.8757
| 0.000 0.002
|
AD | 69.0773 50.1758 -45.7
| 0.092 0.494 0.655
|
AE | -3.07991 -21.9814 -117.857 -72.1572
| 1.000 0.999 0.000 0.057
|
AF | -77.9796 -96.8811 -192.757 -147.057 -74.8997
| 0.023 0.001 0.000 0.000 0.030
As an example, calculate p for the AD-AA comparison (shown as
0.092 above).
First, calculate t as t = diff / (s * sqrt( 1/n1 + 1/n4))
where diff is in the above table, s = sqrt(within MS) and
n1 and n4 are the Freq's from the first table above.
.. di 69.0773 / (sqrt(42178.2188) * sqrt(1/132 +1/132))
2.7325191
Now calculate 1-tailed t probability
.. di ttail(132+132-2, 2.7325191)
..00335642
Note: there are n = 15 comparisons to be adjusted for
Calculate Sidak adjustment (using 2-tailed probability)
.. di min(1, 1 - (1 - .00335642 * 2)^15)
..09609597
Note: this differs from the reported 0.092
Calculate Bonferroni adjustment (using 2-tailed probability)
.. di min(1, .00335642*2*15)
..1006926
Note: the Bonferroni adjusted p is reported as 0.096
Calculate Scheffe adjustment (using t value)
.. di Ftail(5,823,(2.7325191^2)/5)
..18949275
Note: the Scheffe adjusted p is reported as 0.189
(this one is usually OK within round-off)
If someone can confirm this problem -- or point out the
error in my calculations -- I would appreciate it.
Tom
-----------------------------------------
Thomas J. Steichen
[email protected]
-----------------------------------------
-----------------------------------------
CONFIDENTIALITY NOTE: This e-mail message, including any
attachment(s), contains information that may be confidential,
protected by the attorney-client or other legal privileges, and/or
proprietary non-public information. If you are not an intended
recipient of this message or an authorized assistant to an intended
recipient, please notify the sender by replying to this message and
then delete it from your system. Use, dissemination, distribution,
or reproduction of this message and/or any of its attachments (if
any) by unintended recipients is not authorized and may be
unlawful.
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/
