Bookmark and Share

Notice: On April 23, 2014, Statalist moved from an email list to a forum, based at

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: st: RE: RE: two sample test under generalized Behrens-Fisher conditions

From   Steven Samuels <>
Subject   Re: st: RE: RE: two sample test under generalized Behrens-Fisher conditions
Date   Tue, 14 Dec 2010 12:09:34 -0500

I don't think that highly of t-tests. To quote Hampel et al. (Robust Statistics: The Approach Based on Influence Functions, Wiley, NY, 1986) p. 405:

"Many statisticians are proud of the so-called robustness of the t- test and more generally of the test in fixed-effects models in the analysis of variance. But this robustness is only a rather moderate and limited robustness of level ("robustness of validity"); the power ("robustness of efficiencey") and hence also the length of confidence intervals and the size of standard erros is very nonrobust. Consequently, a significant result can be believed, but non- significance may just be due to the inefficiency of least squares."

Perhaps the easiest alternative to teach would be one based on trimmed means, which are not only easy to understand (as opposed to, say, M- Estimators and robust regression), but, unlike the median, have an easy standard error formula.


On Dec 14, 2010, at 10:16 AM, Nick Cox wrote:

I see the problem. I couldn't (wouldn't) fit -glm- in an introductory course either.

In similar circumstances I usually assert that t tests work well even if the assumptions are not well satisfied. This is an idea that goes back at least to G.E.P. Box in Biometrika 1953:

Box, G.E.P. 1953. Non-normality and tests on variances. Biometrika 40: 318-35.


Airey, David C

I was looking for "stark cookbooky" solutions for a (too) short intro course that will not address GLM. But transformations they will be told about, and the last time I taught this course, your help file about transformations was required reading. Thanks for that citation. Looks like a good book.

Nick Cox

In this kind of territory, I would always

1. Check out what is said in Rupert G. Miller, Beyond ANOVA. See on the CRC Press reissue

Your library may hold a copy of the Wiley original.

2. Be wary of the stark cookbooky alternative: data if normal, ranks otherwise. What happened to the idea of transformations or link functions? How do you decide when the data are approximately normal any way?

Here is an example of a different approach. In the auto data, -mpg- given -foreign- is neither normal nor heteroscedastic. But these are secondary issues. Consider this set of results. In each - family(normal)- is implied.

foreach v in "power 1" "power 0.5" "log" "power -0.5" "power -1" {
	qui glm mpg foreign, link(`v')
	mat b = e(b)
	mat V = e(V)
	di "`v'"    "{col 20}" %3.2f   b[1,1] / sqrt(V[1,1])

power 1            3.63
power 0.5          3.70
log                3.75
power -0.5         -3.78
power -1           -3.80

The change of sign of what -glm- calls the z statistic is an expected side-effect of changing to inverse transformations. More importantly, z changes only very slowly and the collective set of results points to the idea that 1/mpg is a more appropriate scale than mpg on which to test for differences. This of course matches basic science.

Generalized linear models are nearly 40 years old as a family. When are they going to receive the recognition they deserve?

Airey, David C

I was reading a little about what to do when you have both unequal variance and non-normality. Neither the equal variance t-test nor the Mann-Whitney U test are best when you want to interpret the difference in means or medians.

I had found the Stata command -fprank-, but it turns out this robust ranks test doesn't escape a symmetry assumption to interpret the location difference.

I found that some recommend using Welch's t-test on the ranked data (Zimmerman and Zumbo (1993) Rank transformations and the power of the Student's t test and the Welch t' test for non-normal populations with unequal variances. Canadian Journal of Experimental Psychology 47:3, 523-539).

This appears easy and satisfying solution to teach with: always use unequal variances t-test and use ranks if the data are also not normal.

*   For searches and help try:

*   For searches and help try:

© Copyright 1996–2017 StataCorp LLC   |   Terms of use   |   Privacy   |   Contact us   |   Site index