st: Scheffe, Bonferroni and Sidak tests

[Richard Williams <Richard.A.Williams.5@nd.edu>]

The -oneway- program gives you the option of doing Scheffe, Bonferroni and 
Sidak tests.  Some other commands offer similar options, e.g. -test-.  Is 
there any consensus as to which is best?  Are there any situations in which 
one is clearly preferable to another?  My experience has been that results 
tend to be similar, and the examples from the Stata reference guide also 
show little difference.

Also, as a practical matter, how often do these tests get done in 
practice?  I see them rarely in the stuff I read and no reviewer has ever 
asked me to provide this info.  I understand the rationale for these tests, 
but if you take that rationale to its logical conclusion it seems like 
every coefficient in every model should have its p-value adjusted to 
reflect the fact that multiple tests are going on.  The tests also seem 
like they can be excessively conservative -- all differences between pairs 
of groups could be significant at the .05 level, but after making one of 
these adjustments none of the differences could be significant.



-------------------------------------------
Richard Williams, Associate Professor
OFFICE: (574)631-6668, (574)631-6463
FAX:    (574)288-4373
HOME:   (574)289-5227
EMAIL:  Richard.A.Williams.5@ND.Edu
WWW (personal):    http://www.nd.edu/~rwilliam
WWW (department):    http://www.nd.edu/~soc

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/