Statalisters,
I have noted on a number of occasions that the Bonferroni and Sidak
adjusted pvalues from multiple comparisons by the oneway command
differ slightly (but more than simple rounding error) from what my
hand calculations suggest. Has anyone else noted this?
I'll give some example output below (using Sidak adjustment) and
also show my hand calculations:
.. oneway vol id, sidak t
 Summary of Volume
ID  Mean Std. Dev. Freq.
+
AA  934.42349 207.71981 132
AB  953.325 201.66785 140
AC  1049.2007 232.39271 137
AD  1003.5008 212.34701 132
AE  931.34357 193.50937 140
AF  856.44392 183.51134 148
+
Total  953.14089 213.68692 829
Analysis of Variance
Source SS df MS F Prob > F

Between groups 3095545.81 5 619109.162 14.68 0.0000
Within groups 34712674.1 823 42178.2188

Total 37808219.9 828 45662.1013
Bartlett's test for equal variances: chi2(5) = 9.3237 Prob>chi2 = 0.097
Comparison of Volume by ID
(Sidak)
Row Mean
Col Mean  AA AB AC AD AE
+
AB  18.9015
 1.000

AC  114.777 95.8757
 0.000 0.002

AD  69.0773 50.1758 45.7
 0.092 0.494 0.655

AE  3.07991 21.9814 117.857 72.1572
 1.000 0.999 0.000 0.057

AF  77.9796 96.8811 192.757 147.057 74.8997
 0.023 0.001 0.000 0.000 0.030
As an example, calculate p for the ADAA comparison (shown as
0.092 above).
First, calculate t as t = diff / (s * sqrt( 1/n1 + 1/n4))
where diff is in the above table, s = sqrt(within MS) and
n1 and n4 are the Freq's from the first table above.
.. di 69.0773 / (sqrt(42178.2188) * sqrt(1/132 +1/132))
2.7325191
Now calculate 1tailed t probability
.. di ttail(132+1322, 2.7325191)
..00335642
Note: there are n = 15 comparisons to be adjusted for
Calculate Sidak adjustment (using 2tailed probability)
.. di min(1, 1  (1  .00335642 * 2)^15)
..09609597
Note: this differs from the reported 0.092
Calculate Bonferroni adjustment (using 2tailed probability)
.. di min(1, .00335642*2*15)
..1006926
Note: the Bonferroni adjusted p is reported as 0.096
Calculate Scheffe adjustment (using t value)
.. di Ftail(5,823,(2.7325191^2)/5)
..18949275
Note: the Scheffe adjusted p is reported as 0.189
(this one is usually OK within roundoff)
If someone can confirm this problem  or point out the
error in my calculations  I would appreciate it.
Tom

Thomas J. Steichen
[email protected]


CONFIDENTIALITY NOTE: This email message, including any
attachment(s), contains information that may be confidential,
protected by the attorneyclient or other legal privileges, and/or
proprietary nonpublic information. If you are not an intended
recipient of this message or an authorized assistant to an intended
recipient, please notify the sender by replying to this message and
then delete it from your system. Use, dissemination, distribution,
or reproduction of this message and/or any of its attachments (if
any) by unintended recipients is not authorized and may be
unlawful.
*
* For searches and help try:
* http://www.stata.com/support/faqs/res/findit.html
* http://www.stata.com/support/statalist/faq
* http://www.ats.ucla.edu/stat/stata/