Search
   >> Home >> Resources & support >> FAQs >> Repeated-measures ANOVA examples

Why do I get an error message when I try to run a repeated-measures ANOVA?

Title   Repeated-measures ANOVA examples
Authors Kenneth Higbee, StataCorp
Wesley Eddings, StataCorp
Date February 2000; updated August 2009; minor revisions July 2011

Introduction
Examples with one repeated variable Examples with two or more repeated variables Summary
References

Introduction

Repeated-measures ANOVA, obtained with the repeated() option of the anova command, requires more structural information about your model than a regular ANOVA, as mentioned in the technical note on page 34 of [R] anova. When this information cannot be determined from the information provided in your anova command, you end up getting error messages such as

    could not determine between-subject error term; use bse() option
    r(421);

or

    could not determine between-subject basic unit; use bseunit() option
    r(422);

These error messages can almost always be avoided with the proper specification of your ANOVA model.

You can jump ahead to the summary to see a list of common user errors and how to overcome them. The examples presented here demonstrate how to obtain a repeated-measures ANOVA and show ways to overcome common errors.

The command wsanova, written by John Gleason and presented in article sg103 of STB-47 (Gleason 1999), provides a different syntax for specifying certain types of repeated-measures ANOVA designs. Not all repeated-measures ANOVA designs are supported by wsanova, but for some problems you might find the syntax more intuitive. (See below for installation instructions.) In other cases, using Stata’s anova command with the repeated() option may be the more natural, or the only, way to obtain the analysis you seek.

The anova manual entry (see the Repeated-measures ANOVA section in [R] anova) presents three repeated-measures ANOVA examples. The examples range from a simple dataset having five persons with measures on four drugs taken from table 4.3 of Winer, Brown, and Michels (1991), to the more complicated data from table 7.13 of Winer, Brown, and Michels (1991) involving two repeated-measures variables (and their interactions) along with a between-subjects term.

Gleason (1999) demonstrates the wsanova command with data from Cole and Grizzle (1966). With these data he provides three examples that illustrate a repeated-measures ANOVA with none, one, and two between-subjects factors.

Here I demonstrate the anova and wsanova commands to specify various types of repeated-measures ANOVAs. I repeat the examples from the anova manual entry and the wsanova STB article (Gleason 1999). A couple of other examples are also presented. Seven examples involving one repeated variable and three examples involving two repeated variables are shown. Along the way I comment on the common types of user mistakes made in specifying these kinds of models and show how to overcome the difficulty.


Examples with one repeated variable

The following examples illustrate various ways repeated-measures ANOVA models with one repeated measure variable may be specified in Stata. I start with the simplest repeated measures design and progress through more complicated designs. I demonstrate how to use both the anova command and the wsanova command (when possible) and discuss potential problems and possible solutions.


Person repeated on drug example from the anova manual entry

The example starting on page 31 of [R] anova is taken from table 4.3 of Winer, Brown, and Michels (1991). Using tabdisp we can get a tabular view of the data.

. use http://www.stata-press.com/data/r13/t43
(T4.3 -- Winer, Brown, Michels)

. tabdisp person drug, cellvar(score)

drug
person 1 2 3 4
1 30 28 16 34
2 14 18 10 22
3 24 20 18 30
4 38 34 20 44
5 26 28 14 30

The data are in long format.

. list, sep(4)

  person drug score
1. 1 1 30
2. 1 2 28
3. 1 3 16
4. 1 4 34
5. 2 1 14
6. 2 2 18
7. 2 3 10
8. 2 4 22
9. 3 1 24
10. 3 2 20
11. 3 3 18
12. 3 4 30
13. 4 1 38
14. 4 2 34
15. 4 3 20
16. 4 4 44
17. 5 1 26
18. 5 2 28
19. 5 3 14
20. 5 4 30

An error users make is to try to execute the anova (or wsanova) command with the data in wide format. For instance, if my data looked like this

. list

  person drug1 drug2 drug3 drug4
1. 1 30 28 16 34
2. 2 14 18 10 22
3. 3 24 20 18 30
4. 4 38 34 20 44
5. 5 26 28 14 30

I would not be able to run the appropriate anova command. The data can be changed to the long format needed by anova by using the reshape command.

. reshape long drug, i(person) j(dr)
(note: j = 1 2 3 4)

Data                               wide   ->   long
Number of obs. 5 -> 20
Number of variables 5 -> 3
j variable (4 values) -> dr
xij variables:
drug1 drug2 ... drug4 -> drug

I would have to rename the drug variable score and then rename the dr variable drug to have the same variable names shown in my earlier listing of the original long-format dataset.

The repeated-measures anova for this example is

. anova score person drug, repeated(drug) Number of obs = 20 R-squared = 0.9244 Root MSE = 3.06594 Adj R-squared = 0.8803
Source Partial SS df MS F Prob > F
Model 1379 7 197 20.96 0.0000
person 680.8 4 170.2 18.11 0.0001
drug 698.2 3 232.733333 24.76 0.0000
Residual 112.8 12 9.4
Total 1491.8 19 78.5157895
Between-subjects error term: person Levels: 5 (4 df) Lowest b.s.e. variable: person Repeated variable: drug Huynh-Feldt epsilon = 1.0789 *Huynh-Feldt epsilon reset to 1.0000 Greenhouse-Geisser epsilon = 0.6049 Box's conservative epsilon = 0.3333
Prob > F
Source df F Regular H-F G-G Box
drug 3 24.76 0.0000 0.0000 0.0006 0.0076
Residual 12

An explanation of the output is included in the manual.

A common error that might be made when trying to run anova on this simple example is to enter

    . anova score drug, repeated(drug)
    could not determine between-subject error term; use bse() option
    r(421);
You might be tempted, after seeing the above error message, to type
    . anova score drug, repeated(drug) bse(person)
    term not in model
    r(147);

but this approach also fails. The moral of this last error message is that to perform the necessary computations for a repeated-measures ANOVA, the between-subjects error term must be a term in the ANOVA model. Here we need to have person as one of the terms in the model. This leads to the correct specification anova score person drug, repeated(drug) as shown earlier.

The wsanova command presented in STB-47 sg103 (Gleason 1999) can also perform this analysis. To obtain this command, type net STB-47 followed by net describe sg103, and then follow the installation instructions. See help stb for details.

. wsanova score drug, id(person) epsilon

                           Number of obs =      20     R-squared     =  0.9244
                           Root MSE      = 3.06594     Adj R-squared =  0.8803

Source Partial SS df MS F Prob > F
person 680.8 4 170.2
drug 698.2 3 232.733333 24.76 0.0000
Residual 112.8 12 9.4
Total 1491.8 19 78.5157895
Note: Within subjects F-test(s) above assume sphericity of residuals; p-values corrected for lack of sphericity appear below. Greenhouse-Geisser (G-G) epsilon: 0.6049 Huynh-Feldt (H-F) epsilon: 1.0000 Sphericity G-G H-F
Source df F Prob > F Prob > F Prob > F
drug 3 24.76 0.0000 0.0006 0.0000

We get the same information we did with the anova command. Which command to use for this simple case is a matter of personal preference. You can either use

    anova score person drug, repeated(drug)

or download wsanova and use

    wsanova score drug, id(person) epsilon

No between-subjects factors example from wsanova STB article

The examples in Gleason (1999) demonstrating the wsanova command use a dataset obtained from Cole and Grizzle (1966). With the net command (also see help stb), you can obtain the dataset, histamin.dta, as well as the wsanova command. Type net STB-47 followed by net describe sg103, then follow the instructions.

Gleason’s first example, a “single factor within subject (randomized blocks) design” is the same underlying ANOVA design as presented in the previous example. Since this example is similar to the previous one, I simply show how you can obtain the analysis using the anova and wsanova commands without additional comments. The analysis using anova proceeds just as it did with our previous example. This time, we have lhist measurements on dogs over time. Unlike our first example, we restrict the analysis to the first group of dogs with the if group==1 command qualifier.

. use histamin, clear
(Blood histamine levels in dogs)

. anova lhist dog time if group==1, repeated(time)

                           Number of obs =      16     R-squared     =  0.9388
                           Root MSE      = .409681     Adj R-squared =  0.8979

Source Partial SS df MS F Prob > F
Model 23.1592161 6 3.85986934 23.00 0.0001
dog 16.9024081 3 5.63413604 33.57 0.0000
time 6.25680792 3 2.08560264 12.43 0.0015
Residual 1.51054662 9 .167838513
Total 24.6697627 15 1.64465084
Between-subjects error term: dog Levels: 4 (3 df) Lowest b.s.e. variable: dog Repeated variable: time Huynh-Feldt epsilon = 0.5376 Greenhouse-Geisser epsilon = 0.4061 Box's conservative epsilon = 0.3333
Prob > F
Source df F Regular H-F G-G Box
time 3 12.43 0.0015 0.0138 0.0267 0.0388
Residual 9

The same results are also easily obtained with the wsanova command.

. wsanova lhist time if group==1, id(dog) epsilon

                                 Number of obs =      16     R-squared     =  0.9388
                                 Root MSE      = .409681     Adj R-squared =  0.8979

Source Partial SS df MS F Prob > F
dog 16.9024081 3 5.63413604
time 6.25680792 3 2.08560264 12.43 0.0015
Residual 1.51054662 9 .167838513
Total 24.6697627 15 1.64465084
Note: Within subjects F-test(s) above assume sphericity of residuals; p-values corrected for lack of sphericity appear below. Greenhouse-Geisser (G-G) epsilon: 0.4061 Huynh-Feldt (H-F) epsilon: 0.5376 Sphericity G-G H-F
Source df F Prob > F Prob > F Prob > F
time 3 12.43 0.0015 0.0267 0.0138

You may use

    anova lhist dog time if group==1, repeated(time)

or download wsanova and use

    wsanova lhist time if group==1, id(dog) epsilon

Both commands provide the same information.


Dial calibration example from the anova manual entry

The example starting on page 32 of [R] anova is taken from table 7.7 of Winer, Brown, and Michels (1991). By using tabdisp we can get a tabular view of the data.

. use http://www.stata-press.com/data/r13/t77, clear
(T7.7 -- Winer, Brown, Michels)

. tabdisp shape subject calib, cell(score)

2 methods for calibrating dials and
subject nested in calib
4 dial 1   2
shapes 1 2 3 1 2 3
1 0 3 4 4 5 7
2 0 1 3 2 4 5
3 5 5 6 7 6 8
4 3 4 2 8 6 9

I have the data in long form.

. list, sep(4)

 calib subject shape score
1. 1 1 1 0
2. 1 1 2 0
3. 1 1 3 5
4. 1 1 4 3
5. 1 2 1 3
6. 1 2 2 1
7. 1 2 3 5
8. 1 2 4 4
9. 1 3 1 4
10. 1 3 2 3
11. 1 3 3 6
12. 1 3 4 2
13. 2 1 1 4
14. 2 1 2 2
15. 2 1 3 7
16. 2 1 4 8
17. 2 2 1 5
18. 2 2 2 4
19. 2 2 3 6
20. 2 2 4 6
21. 2 3 1 7
22. 2 3 2 5
23. 2 3 3 8
24. 2 3 4 9

If instead you had the data in a wide format, you would need to use the reshape command to get it into long format before using the anova (or wsanova) command. For an example of using reshape, see the first example.

You should understand your model before attempting to use anova. For this dataset, both calib and shape are fixed while subject is random. The full model includes terms for calib, subject nested within calib, shape, shape interacted with calib, and shape interacted with subject nested within calib. As usual, we let this highest order term drop and become the residual error. The shape variable is the repeated variable. This produces an ANOVA with one between-subjects factor (same underlying design as the next example). If you were to examine the expected mean squares for this setup (Winer, Brown, and Michels 1991), you would find the appropriate error term for the test of calib is subject|calib. The appropriate error term for shape and shape#calib is shape#subject|calib (which is the residual error since we do not include the term in the model).

Armed with this information, it becomes easy to specify the correct anova command.

. anova score calib / subject|calib shape calib#shape, repeated(shape)

                             Number of obs =      24     R-squared     =  0.8925
                             Root MSE      = 1.11181     Adj R-squared =  0.7939

Source Partial SS df MS F Prob > F
Model 123.125 11 11.1931818 9.06 0.0003
calib 51.0416667 1 51.0416667 11.89 0.0261
subject|calib 17.1666667 4 4.29166667
shape 47.4583333 3 15.8194444 12.80 0.0005
calib#shape 7.45833333 3 2.48611111 2.01 0.1662
Residual 14.8333333 12 1.23611111
Total 137.958333 23 5.99818841
Between-subjects error term: subject|calib Levels: 6 (4 df) Lowest b.s.e. variable: subject Covariance pooled over: calib (for repeated variable) Repeated variable: shape Huynh-Feldt epsilon = 0.8483 Greenhouse-Geisser epsilon = 0.4751 Box's conservative epsilon = 0.3333
Prob > F
Source df F Regular H-F G-G Box
shape 3 12.80 0.0005 0.0011 0.0099 0.0232
calib#shape 3 2.01 0.1662 0.1791 0.2152 0.2291
Residual 12

A common error when unfamiliar with the underlying model is to just list some variables in the anova command (possibly with some interactions included), and then get the following error message.

    . anova score calib subject shape calib#shape, repeated(shape)
    could not determine between-subject error term; use bse() option
    r(421);

Stata’s anova command needs the between-subject error term (here subject|calib) to be included in the model to obtain the repeated-measures corrections.

The wsanova command (Gleason 1999) seems like a natural alternative to use for this example. It seems you should be able to say

    . wsanova score shape, id(subject) between(calib) epsilon
    epsilon option is invalid with missing data
    r(499);

but something went wrong. This dataset has no missing observations. This is just wsanova’s way of saying it is confused. What could have caused the confusion? Look at the listing of the data near the beginning of this example. In particular, pay attention to how the subject variable is set up. We have subjects going from 1 to 3 for the first level of calib and then going from 1 to 3 again for the second level of calib. anova was able to handle this, but wsanova is confused. We can help wsanova out of its confusion by generating a new variable that gives a unique number to each subject regardless of which level of calib is involved. We use the group() function of the egen command to help us.

. egen z = group(calib subject)

. wsanova score shape, id(z) between(calib) epsilon

                                      Number of obs =      24     R-squared     =  0.8925
                                      Root MSE      = 1.11181     Adj R-squared =  0.7939

Source Partial SS df MS F Prob > F
Between subjects: 51.0416667 1 51.0416667 11.89 0.0261
calib 51.0416667 1 51.0416667 11.89 0.0261
z*calib 17.1666667 4 4.29166667
Within subjects: 54.9166667 6 9.15277778 7.40 0.0017
shape 47.4583333 3 15.8194444 12.80 0.0005
shape*calib 7.45833333 3 2.48611111 2.01 0.1662
Residual 14.8333333 12 1.23611111
Total 137.958333 23 5.99818841
Note: Within subjects F-test(s) above assume sphericity of residuals; p-values corrected for lack of sphericity appear below. Greenhouse-Geisser (G-G) epsilon: 0.4751 Huynh-Feldt (H-F) epsilon: 0.8483 Sphericity G-G H-F
Source df F Prob > F Prob > F Prob > F
shape 3 12.80 0.0005 0.0099 0.0011
shape*calib 3 2.01 0.1662 0.2152 0.1791

We have been able to reproduce the same results we obtained with anova. There is one test provided in the output of wsanova above that is not automatically produced with anova. If you look back at the ANOVA table produced by wsanova, you will see it produces an overall test for “Within subjects”. Here it produces an F of 7.40.

    Within subjects:    |  54.9166667     6  9.15277778       7.40     0.0017

Using the test command we can easily obtain this same test after running anova.

. test shape calib#shape

Source Partial SS df MS F Prob > F
shape calib#shape 54.9166667 6 9.15277778 7.40 0.0017
Residual 14.8333333 12 1.23611111

With this example you can either do

    anova score calib / subject|calib shape calib#shape , repeated(shape)

or download wsanova (see above for installation instructions) and do

    egen z = group(calib subject)
    wsanova score shape, id(z) between(calib) epsilon

Both provide the same information.


One between-subjects factor example from wsanova STB article

The examples in Gleason (1999) demonstrating the wsanova command use a dataset obtained from Cole and Grizzle (1966). With the net command (also see help stb) you can obtain the dataset, histamin.dta, as well as the wsanova command (type net STB-47 followed by net describe sg103, and then follow the instructions). Gleason’s second example, a one between-subjects factor ANOVA design, is the same underlying ANOVA design presented in the previous example.

Since this example is similar to the previous one, I simply show how you can obtain the analysis using the anova and wsanova commands without additional comments. The analysis using anova proceeds just as it did with our previous example. This time we have lhist measurements on dogs nested within groups over time. Following the lead of Gleason (1999) we restrict the data with the if dog != 6 command qualifier.

. use histamin, clear
(Blood histamine levels in dogs)

. anova lhist group / dog|group time time#group if dog!=6, repeated(time)

                            Number of obs =      60     R-squared     =  0.9709
                            Root MSE      =  .27427     Adj R-squared =  0.9479

Source Partial SS df MS F Prob > F
Model 82.6836382 26 3.18013993 42.28 0.0000
group 27.0286268 3 9.00954226 4.07 0.0359
dog|group 24.3468341 11 2.21334855
time 12.0589871 3 4.01966235 53.44 0.0000
time#group 17.5232918 9 1.94703243 25.88 0.0000
Residual 2.48238892 33 .075223907
Total 85.1660271 59 1.44349199
Between-subjects error term: dog|group Levels: 15 (11 df) Lowest b.s.e. variable: dog Covariance pooled over: group (for repeated variable) Repeated variable: time Huynh-Feldt epsilon = 0.8475 Greenhouse-Geisser epsilon = 0.5694 Box's conservative epsilon = 0.3333
Prob > F
Source df F Regular H-F G-G Box
time 3 53.44 0.0000 0.0000 0.0000 0.0000
time#group 9 25.88 0.0000 0.0000 0.0000 0.0000
Residual 33

I can obtain the overall within-subjects test as follows:

. test time time#group

Source Partial SS df MS F Prob > F
time time#group 31.3081774 12 2.60901478 34.68 0.0000
Residual 2.48238892 33 .075223907

This same analysis is also easy with wsanova:

. wsanova lhist time if dog!=6, id(dog) between(group) epsilon

                                       Number of obs =      60     R-squared     =  0.9709
                                       Root MSE      =  .27427     Adj R-squared =  0.9479

Source Partial SS df MS F Prob > F
Between subjects: 27.0286268 3 9.00954226 4.07 0.0359
group 27.0286268 3 9.00954226 4.07 0.0359
dog*group 24.3468341 11 2.21334855
Within subjects: 31.3081774 12 2.60901478 34.68 0.0000
time 12.0589871 3 4.01966235 53.44 0.0000
time*group 17.5232918 9 1.94703243 25.88 0.0000
Residual 2.48238892 33 .075223907
Total 85.1660271 59 1.44349199
Note: Within subjects F-test(s) above assume sphericity of residuals; p-values corrected for lack of sphericity appear below. Greenhouse-Geisser (G-G) epsilon: 0.5694 Huynh-Feldt (H-F) epsilon: 0.8475 Sphericity G-G H-F
Source df F Prob > F Prob > F Prob > F
time 3 53.44 0.0000 0.0000 0.0000
time*group 9 25.88 0.0000 0.0000 0.0000

This example has the dogs numbered from 1 to 16, so (unlike the previous example) there is no need to generate a new id() variable for the wsanova command.

For this example, you can pick between running

    anova lhist group / dog|group time time#group if dog != 6, repeated(time)

and downloading wsanova and running

    wsanova lhist time if dog != 6, id(dog) between(group) epsilon

to obtain the results.


Two between-subjects factors example from wsanova STB article

The third example in Gleason (1999) demonstrating the wsanova command also uses the histamin.dta dataset obtained from Cole and Grizzle (1966). This example expands from the previous example by splitting the group variable, which has four levels, into two variables, depleted and drug, each with two levels corresponding to a 2 × 2 factorial. We end up having two between-subject factors plus their interaction. Again, following the lead of Gleason (1999), we restrict the data with the if dog != 6 command qualifier.

Here is the result of running wsanova on this dataset:

. use histamin, clear
(Blood histamine levels in dogs)

. wsanova lhist time if dog!=6, id(dog) between(drug depl drug*depl) eps
 
                                      Number of obs =      60     R-squared     =  0.9709
                                      Root MSE      =  .27427     Adj R-squared =  0.9479
Source Partial SS df MS F Prob > F
Between subjects: 27.0286268 3 9.00954226 4.07 0.0359
drug 5.99336256 1 5.99336256 2.71 0.1281
depleted 15.4484076 1 15.4484076 6.98 0.0229
drug*depleted 4.69087549 1 4.69087549 2.12 0.1734
dog*drug*depleted 24.3468341 11 2.21334855
Within subjects: 31.3081774 12 2.60901478 34.68 0.0000
time 12.0589871 3 4.01966235 53.44 0.0000
time*drug 1.84429539 3 .614765129 8.17 0.0003
time*depleted 12.0897855 3 4.02992849 53.57 0.0000
time*drug*depleted 2.93077944 3 .976926479 12.99 0.0000
Residual 2.48238892 33 .075223907
Total 85.1660271 59 1.44349199
Note: Within subjects F-test(s) above assume sphericity of residuals; p-values corrected for lack of sphericity appear below. Greenhouse-Geisser (G-G) epsilon: 0.5694 Huynh-Feldt (H-F) epsilon: 0.8475 Sphericity G-G H-F
Source df F Prob > F Prob > F Prob > F
time 3 53.44 0.0000 0.0000 0.0000
time*drug 3 8.17 0.0003 0.0039 0.0008
time*depleted 3 53.57 0.0000 0.0000 0.0000
time*drug*depleted 3 12.99 0.0000 0.0005 0.0000

The anova command with the repeated() option can also be used on this problem:

. anova lhist drug dep drug#dep / dog|drug#dep time time#drug time#dep time#drug#dep 
> if dog!=6, rep(time)

                                   Number of obs =      60     R-squared     =  0.9709
                                   Root MSE      =  .27427     Adj R-squared =  0.9479

Source Partial SS df MS F Prob > F
Model 82.6836382 26 3.18013993 42.28 0.0000
drug 6.1513201 1 6.1513201 2.78 0.1237
depleted 15.712679 1 15.712679 7.10 0.0220
drug#depleted 4.69087549 1 4.69087549 2.12 0.1734
dog|drug#depleted 24.3468341 11 2.21334855
time 12.0589871 3 4.01966235 53.44 0.0000
time#drug 1.84429539 3 .614765129 8.17 0.0003
time#depleted 12.0897855 3 4.02992849 53.57 0.0000
time#drug#depleted 2.93077944 3 .976926479 12.99 0.0000
Residual 2.48238892 33 .075223907
Total 85.1660271 59 1.44349199
Between-subjects error term: dog|drug#depleted Levels: 15 (11 df) Lowest b.s.e. variable: dog Covariance pooled over: drug#depleted (for repeated variable) Repeated variable: time Huynh-Feldt epsilon = 0.8475 Greenhouse-Geisser epsilon = 0.5694 Box’s conservative epsilon = 0.3333
Prob > F
Source df F Regular H-F G-G Box
time 3 53.44 0.0000 0.0000 0.0000 0.0000
time#drug 3 8.17 0.0003 0.0008 0.0039 0.0156
time#depleted 3 53.57 0.0000 0.0000 0.0000 0.0000
time#drug#depleted 3 12.99 0.0000 0.0000 0.0005 0.0041
Residual 33

If you look closely, you will find a difference in the results for the drug and the depleted terms between anova and wsanova. This is due to the imbalance in the data from excluding the observations associated with the sixth dog.

. tabulate drug depleted if dog!=6

Drug Depleted pre-test
administer histamines?
ed No Yes Total
Morphine 16 12 28
TriMeth 16 16 32
Total 32 28 60

The wsanova command actually performs its work with two separate calls to anova instead of getting the whole ANOVA table at one time. The anova command with the repeated() option computes the complete model in one estimation. In the presence of imbalanced data, this method can sometimes make a difference in the results. In these cases, I recommend using the anova command.

Gleason (1999) also shows for this example how to use the wonly() option in conjunction with the between() option of wsanova to control which terms end up in the ANOVA table.

. wsanova lhist time if dog!=6, id(dog) between(drug depl) wonly(time time*depl) epsilon

                                     Number of obs =      60     R-squared     =  0.9103
                                     Root MSE      = .442692     Adj R-squared =  0.8642

Source Partial SS df MS F Prob > F
Between subjects: 22.3377513 2 11.1688756 4.62 0.0326
drug 6.87754936 1 6.87754936 2.84 0.1176
depleted 16.8857304 1 16.8857304 6.98 0.0215
dog*drug*depleted 29.0377096 12 2.41980913
Within subjects: 26.1474934 6 4.35791556 22.24 0.0000
time 12.0454347 3 4.0151449 20.49 0.0000
time*depleted 12.3626079 3 4.12086929 21.03 0.0000
Residual 7.64307289 39 .195976228
Total 85.1660271 59 1.44349199
Note: Within subjects F-test(s) above assume sphericity of residuals; p-values corrected for lack of sphericity appear below. Greenhouse-Geisser (G-G) epsilon: 0.5694 Huynh-Feldt (H-F) epsilon: 0.7651 Sphericity G-G H-F
Source df F Prob > F Prob > F Prob > F
time 3 20.49 0.0000 0.0000 0.0000
time*depleted 3 21.03 0.0000 0.0000 0.0000

We can, of course, obtain the same results directly with anova.

. anova lhist drug depl / dog|drug#depl time time#depl if dog!=6, rep(time)

                                   Number of obs =      60     R-squared     =  0.9103
                                   Root MSE      = .442692     Adj R-squared =  0.8642


Source Partial SS df MS F Prob > F
Model 77.5229542 20 3.87614771 19.78 0.0000
drug 6.87754936 1 6.87754936 2.84 0.1176
depleted 16.8857304 1 16.8857304 6.98 0.0215
dog|drug#depleted 29.0377096 12 2.41980913
time 12.0454347 3 4.0151449 20.49 0.0000
time#depleted 12.3626079 3 4.12086929 21.03 0.0000
Residual 7.64307289 39 .195976228
Total 85.1660271 59 1.44349199
Between-subjects error term: dog|drug#depleted Levels: 15 (12 df) Lowest b.s.e. variable: dog Covariance pooled over: drug#depleted (for repeated variable) Repeated variable: time Huynh-Feldt epsilon = 0.7651 Greenhouse-Geisser epsilon = 0.5694 Box's conservative epsilon = 0.3333
Prob > F
Source df F Regular H-F G-G Box
time 3 20.49 0.0000 0.0000 0.0000 0.0006
time#depleted 3 21.03 0.0000 0.0000 0.0000 0.0005
Residual 39

As with the previous examples, it is important to understand your model and to make sure to include the between-subjects error term in the model. Here it is the term dog|drug#depleted. The wsanova command puts this term (labeled as dog*drug*depleted) into the model automatically based on the options you specify.

This example does point out that for models with imbalance there can sometimes be a difference between wsanova and anova in the reported ANOVA table for some of the terms. In these cases, you should rely on the anova command.


Another two between-subjects factors example

This example is taken from the data of table 7.22 of Winer, Brown, and Michels (1991) and has a similar underlying structure to that of the previous example.

For this example, we have an experiment on a learning task with the variables anxiety and tension, each at two levels in a factorial layout. Nested within this interaction is subject. These are the variables involved in the between-subjects portion of our ANOVA. There are four trials—our repeated variable. We are also interested in examining the interaction of trial with the other terms in the model.

Here is a tabular view of the data:

. use t722, clear
(T7.22 -- Winer, Brown, Michels)

. tabdisp subject trial, by(anxiety tension) c(response) concise stubw(10)

effect of
anxiety --
2 levels,
muscular
tension --
2 levels
and trial
subject 1 2 3 4
1
1
1 18 14 12 6
2 19 12 8 4
3 14 10 6 2
1
2
4 16 12 10 4
5 12 8 6 2
6 18 10 5 1
2
1
7 16 10 8 4
8 18 8 4 1
9 16 12 6 2
2
2
10 19 16 10 8
11 16 14 10 9
12 16 12 8 8

In the following anova command, I take advantage of Stata’s ability to allow abbreviations for the variable names.

. anova response an te an#te / su|an#te tr an#tr te#tr an#te#tr, rep(tr)

                           Number of obs =      48     R-squared     =  0.9585
                           Root MSE      = 1.47432     Adj R-squared =  0.9188

Source Partial SS df MS F Prob > F
Model 1205.83333 23 52.4275362 24.12 0.0000
anxiety 10.0833333 1 10.0833333 0.98 0.3517
tension 8.33333333 1 8.33333333 0.81 0.3949
anxiety#tension 80.0833333 1 80.0833333 7.77 0.0237
subject|anxiety#tension 82.5 8 10.3125
trial 991.5 3 330.5 152.05 0.0000
anxiety#trial 8.41666667 3 2.80555556 1.29 0.3003
tension#trial 12.1666667 3 4.05555556 1.87 0.1624
anxiety#tension#trial 12.75 3 4.25 1.96 0.1477
Residual 52.1666667 24 2.17361111
Total 1258 47 26.7659574
Between-subjects error term: subject|anxiety#tension Levels: 12 (8 df) Lowest b.s.e. variable: subject Covariance pooled over: anxiety#tension (for repeated variable) Repeated variable: trial Huynh-Feldt epsilon = 0.9023 Greenhouse-Geisser epsilon = 0.5361 Box's conservative epsilon = 0.3333
Prob > F
Source df F Regular H-F G-G Box
trial 3 152.05 0.0000 0.0000 0.0000 0.0000
anxiety#trial 3 1.29 0.3003 0.3015 0.3002 0.2888
tension#trial 3 1.87 0.1624 0.1693 0.1967 0.2091
anxiety#tension#trial 3 1.96 0.1477 0.1550 0.1847 0.1996
Residual 24

The wsanova command (Gleason 1999) can also be used for this example.

. wsanova response trial, id(subject) between(anx tens anx*tens) epsilon

                           Number of obs =      48     R-squared     =  0.9585
                           Root MSE      = 1.47432     Adj R-squared =  0.9188

Source Partial SS df MS F Prob > F
Between subjects: 98.5 3 32.8333333 3.18 0.0845
anxiety 10.0833333 1 10.0833333 0.98 0.3517
tension 8.33333333 1 8.33333333 0.81 0.3949
anxiety*tension 80.0833333 1 80.0833333 7.77 0.0237
subject*anxiety*tension 82.5 8 10.3125
Within subjects: 1024.83333 12 85.4027778 39.29 0.0000
trial 991.5 3 330.5 152.05 0.0000
trial*anxiety 8.41666667 3 2.80555556 1.29 0.3003
trial*tension 12.1666667 3 4.05555556 1.87 0.1624
trial*anxiety*tension 12.75 3 4.25 1.96 0.1477
Residual 52.1666667 24 2.17361111
Total 1258 47 26.7659574
Note: Within subjects F-test(s) above assume sphericity of residuals; p-values corrected for lack of sphericity appear below. Greenhouse-Geisser (G-G) epsilon: 0.5361 Huynh-Feldt (H-F) epsilon: 0.9023 Sphericity G-G H-F
Source df F Prob > F Prob > F Prob > F
trial 3 152.05 0.0000 0.0000 0.0000
trial*anxiety 3 1.29 0.3003 0.3002 0.3015
trial*tension 3 1.87 0.1624 0.1967 0.1693
trial*anxiety*tension 3 1.96 0.1477 0.1847 0.1550

You can choose between

    anova response an te an#te / su|an#te tr an#tr te#tr an#te#tr , rep(tr)

or download wsanova (see above for installation instructions) and then type

    wsanova response trial, id(subject) between(anx tens anx*tens) epsilon

A complicated design with one repeated variable

Table 9–11 of Myers (1966) presents an interesting dataset with factor A having two levels, G (representing groups) nested within A (a total of four groups), factor B with two levels that is crossed with A and G|A, S (representing subjects) nested with all of this (S|B#G|A) for a total of 16 subjects, then factor C, the repeated-measures variable with three levels. Each of the 16 subjects has measures for the three levels of C. The interaction of C with the other terms is also included in the model.

Here is a look at the data:

. use tm911, clear
(Table 9-11 in Myers)

. list, sep(12)

  A G B S C res
1. 1 1 1 1 1 4
2. 1 1 1 1 2 5
3. 1 1 1 1 3 8
4. 1 1 1 2 1 3
5. 1 1 1 2 2 6
6. 1 1 1 2 3 10
7. 1 1 2 3 1 3
8. 1 1 2 3 2 6
9. 1 1 2 3 3 10
10. 1 1 2 4 1 4
11. 1 1 2 4 2 5
12. 1 1 2 4 3 9
13. 1 2 1 5 1 4
14. 1 2 1 5 2 7
15. 1 2 1 5 3 8
16. 1 2 1 6 1 3
17. 1 2 1 6 2 6
18. 1 2 1 6 3 9
19. 1 2 2 7 1 1
20. 1 2 2 7 2 6
21. 1 2 2 7 3 8
22. 1 2 2 8 1 4
23. 1 2 2 8 2 2
24. 1 2 2 8 3 12
25. 2 3 1 9 1 7
26. 2 3 1 9 2 7
27. 2 3 1 9 3 11
28. 2 3 1 10 1 4
29. 2 3 1 10 2 8
30. 2 3 1 10 3 14
31. 2 3 2 11 1 9
32. 2 3 2 11 2 8
33. 2 3 2 11 3 16
34. 2 3 2 12 1 7
35. 2 3 2 12 2 10
36. 2 3 2 12 3 19
37. 2 4 1 13 1 3
38. 2 4 1 13 2 5
39. 2 4 1 13 3 9
40. 2 4 1 14 1 2
41. 2 4 1 14 2 7
42. 2 4 1 14 3 8
43. 2 4 2 15 1 10
44. 2 4 2 15 2 12
45. 2 4 2 15 3 13
46. 2 4 2 16 1 9
47. 2 4 2 16 2 11
48. 2 4 2 16 3 15

Myers (1966) indicates that for this example the ANOVA table should have the following structure:

 
Model Term F-Test
Between S
Between G
A MS(A) / MS(G|A)
G|A
Within G
B MS(B) / MS(B#G|A)
B#A MS(B#A) / MS(B#G|A)
B#G|A MS(B#G|A) / MS(S|B#G|A)
S|B#G|A
Within S
C MS(C) / MS(C#G|A)
C#A MS(C#A) / MS(C#G|A)
C#G|A MS(C#G|A) / MS(C#B#G|A)
C#B MS(C#B) / MS(C#B#G|A)
C#B#A MS(C#B#A) / MS(C#B#G|A)
C#B#G|A MS(C#B#G|A) / MS(C#S|B#G|A)
C#S|B#G|A

How did Myers (1966) determine the appropriate mean square to use in the denominator of each of the F tests listed above? He first determined which factors were fixed and which were random and which factors were nested and which were crossed. Then, from that, he figured the expected mean squares for each term. From these he could see which terms were the appropriate error terms for other terms in the model. See Winer, Brown, and Michels (1991) or some other good book on ANOVA modeling to understand “fixed factors”, “random factors”, “nesting”, “crossing”, “expected mean squares”, etc.

The anova command allows the “/” notation that indicates the terms to the left of the slash are to be tested using the term to the right of the slash as the error term. This method makes it easy to get all but one of the F tests from the complicated ANOVA table above with one call to anova. The remaining F test (the test for the C#G|A term) is easily obtained with a call to the test command after running anova. Again, I drop the largest possible interaction term (C#S|B#G|A) so that the residual (which would have had zero degrees of freedom if the term were left in the model) becomes that interaction term.

. anova res A / G|A B B#A / B#G|A / S|B#G|A C C#A / C#G|A C#B C#B#A / C#B#G|A / , rep(C)

                             Number of obs =      48     R-squared     =  0.9346
                             Root MSE      = 1.70171     Adj R-squared =  0.8080

Source Partial SS df MS F Prob > F
Model 662.645833 31 21.375672 7.38 0.0001
A 136.6875 1 136.6875 24.76 0.0381
G|A 11.0416667 2 5.52083333
B 54.1875 1 54.1875 7.45 0.1121
B#A 67.6875 1 67.6875 9.31 0.0927
B#G|A 14.5416667 2 7.27083333
B#G|A 14.5416667 2 7.27083333 13.96 0.0025
S|B#G|A 4.16666667 8 .520833333
C 337.166667 2 168.583333 34.88 0.0029
C#A 1.5 2 .75 0.16 0.8612
C#G|A 19.3333333 4 4.83333333
C#B 8 2 4 2.04 0.2448
C#B#A .5 2 .25 0.13 0.8836
C#B#G|A 7.83333333 4 1.95833333
C#B#G|A 7.83333333 4 1.95833333 0.68 0.6182
Residual 46.3333333 16 2.89583333
Total 708.979167 47 15.0846631
Between-subjects error term: S|B#G|A Levels: 16 (8 df) Lowest b.s.e. variable: S Covariance pooled over: B#G|A (for repeated variable) Repeated variable: C Huynh-Feldt epsilon = 2.4863 *Huynh-Feldt epsilon reset to 1.0000 Greenhouse-Geisser epsilon = 0.9961 Box's conservative epsilon = 0.5000
Prob > F
Source df F Regular H-F G-G Box
C 2 34.88 0.0029 0.0029 0.0030 0.0275
C#A 2 0.16 0.8612 0.8612 0.8605 0.7317
C#G|A 4
C#B 2 2.04 0.2448 0.2448 0.2451 0.2892
C#B#A 2 0.13 0.8836 0.8836 0.8830 0.7551
C#B#G|A 4
C#B#G|A 4 0.68 0.6182 0.6182 0.6177 0.5354
Residual 16
. test C#G|A / C#B#G|A
Source Partial SS df MS F Prob > F
C#G|A 19.3333333 4 4.83333333 2.47 0.2015
C#B#G|A 7.83333333 4 1.95833333

The wsanova command (Gleason 1999) can produce the appropriate mean squares for the terms in the model but will not be able to automatically create the correct F tests for most of the terms. It does not understand all of the structure of this complicated model. Here is what you can obtain from wsanova:

. wsanova res C, id(S) between(A G*A B B*A B*G*A) epsilon

                                        Number of obs =      48     R-squared     =  0.9346
                                        Root MSE      = 1.70171     Adj R-squared =  0.8080

Source Partial SS df MS F Prob > F
Between subjects: 284.145833 7 40.5922619 77.94 0.0000
A 136.6875 1 136.6875 262.44 0.0000
G*A 11.0416667 2 5.52083333 10.60 0.0056
B 54.1875 1 54.1875 104.04 0.0000
B*A 67.6875 1 67.6875 129.96 0.0000
B*G*A 14.5416667 2 7.27083333 13.96 0.0025
S*A*G*B 4.16666667 8 .520833333
Within subjects: 374.333333 16 23.3958333 8.08 0.0001
C 337.166667 2 168.583333 58.22 0.0000
C*A 1.5 2 .75 0.26 0.7750
C*G*A 19.3333333 4 4.83333333 1.67 0.2061
C*B 8 2 4 1.38 0.2797
C*B*A .5 2 .25 0.09 0.9177
C*B*G*A 7.83333333 4 1.95833333 0.68 0.6182
Residual 46.3333333 16 2.89583333
Total 708.979167 47 15.0846631
Note: Within subjects F-test(s) above assume sphericity of residuals; p-values corrected for lack of sphericity appear below. Greenhouse-Geisser (G-G) epsilon: 0.9961 Huynh-Feldt (H-F) epsilon: 1.0000 Sphericity G-G H-F
Source df F Prob > F Prob > F Prob > F
C 2 58.22 0.0000 0.0000 0.0000
C*A 2 0.26 0.7750 0.7742 0.7750
C*G*A 4 1.67 0.2061 0.2064 0.2061
C*B 2 1.38 0.2797 0.2797 0.2797
C*B*A 2 0.09 0.9177 0.9171 0.9177
C*B*G*A 4 0.68 0.6182 0.6177 0.6182

Remember that for this complicated ANOVA you should ignore most of the F tests produced in the output from the wsanova command. Instead, you need to produce the correct F tests from the mean-squares in the ANOVA table after running wsanova. Using the anova command and taking advantage of the “/” notation gives you the appropriate F tests directly in the ANOVA table.

If you did not understand the underlying model for this example and just tried entering variable names into the anova command hoping something good would come out, you would most likely be disappointed. While understanding the underlying model is helpful with simple problems, it becomes crucial with more complicated designs.


Examples with two or more repeated variables

Shown below are three examples of repeated-measures ANOVAs where the subjects have repeated observations over more than one variable. Unlike the previous section of this document where I outlined the use of both anova and wsanova (Gleason 1999), with more than one repeated-measures variable, the anova command is the only choice.


No between-subjects factors with two repeated variables

This example is obtained by restricting our attention of the data from the next example to only one level of the between-subjects variable. This choice produces an example with no between-subjects factors and two repeated variables. The data come from table 7.13 of Winer, Brown, and Michels (1991). After keeping only those observations of interest to this example, we have three subjects, each with nine accuracy scores on all combinations of the three different dials and three different periods. With subject a random factor and both dial and period fixed factors, the appropriate error term for the test of dial is the dial#subject interaction. Likewise, period#subject is the correct error term for period, and period#dial#subject (which we will drop so that it becomes residual error) is the appropriate error term for period#dial.

Here are the data:

. use http://www.stata-press.com/data/r13/t713, clear
(T7.13 -- Winer, Brown, Michels)

. keep if noise==1
(27 observations deleted)

. drop noise

. label var subject ""

. tabdisp subject dial period, cell(score)

10 minute time periods and dial
1   2   3
subject 1 2 3 1 2 3 1 2 3
1 45 53 60 40 52 57 28 37 46
2 35 41 50 30 37 47 25 32 41
3 60 65 75 58 54 70 40 47 50

By specifying both the period and dial variables in the repeated() option of anova along with appropriate use of the “/” notation for specifying the proper error terms in the model, we can easily obtain the desired ANOVA table.

. anova score subject period / subject#period dial / subject#dial period#dial, 
> repeated(period dial)

                           Number of obs =      27     R-squared     =  0.9871
                           Root MSE      = 2.60342     Adj R-squared =  0.9580

Source Partial SS df MS F Prob > F
Model 4146.44444 18 230.358025 33.99 0.0000
subject 1828.22222 2 914.111111 29.54 0.0040
period 1124.66667 2 562.333333 18.17 0.0098
subject#period 123.777778 4 30.9444444
dial 1020.66667 2 510.333333 51.32 0.0014
subject#dial 39.7777778 4 9.94444444
period#dial 9.33333333 4 2.33333333 0.34 0.8410
Residual 54.2222222 8 6.77777778
Total 4200.66667 26 161.564103
Between-subjects error term: subject Levels: 3 (2 df) Lowest b.s.e. variable: subject Repeated variable: period Huynh-Feldt epsilon = 0.6829 Greenhouse-Geisser epsilon = 0.5419 Box's conservative epsilon = 0.5000
Prob > F
Source df F Regular H-F G-G Box
period 2 18.17 0.0098 0.0275 0.0441 0.0509
subject#period 4
Repeated variable: dial Huynh-Feldt epsilon = 0.7129 Greenhouse-Geisser epsilon = 0.5481 Box's conservative epsilon = 0.5000
Prob > F
Source df F Regular H-F G-G Box
dial 2 51.32 0.0014 0.0062 0.0147 0.0189
subject#dial 4
Repeated variables: period#dial Huynh-Feldt epsilon = 0.2631 Greenhouse-Geisser epsilon = 0.2532 Box's conservative epsilon = 0.2500
Prob > F
Source df F Regular H-F G-G Box
period#dial 4 0.34 0.8410 0.6246 0.6187 0.6168
Residual 8

The test on subject in the main ANOVA table should be ignored.

With multiple repeated variables we obtain the various epsilon corrections (Greenhouse–Geisser, Huynh–Feldt, Box’s conservative epsilon) to the p-values for each repeated variable and each interaction of those repeated variables.


One between-subjects factor with two repeated-variables example from the anova manual entry

This example can be found starting on page 35 of [R] anova. The data are from table 7.13 of Winer, Brown, and Michels (1991). There is one between-subject factor, noise, with two levels. There are three subjects nested within each level of noise. As with the previous example, there are two repeated variables, period and dial, each with three levels, so that each subject has nine values recorded. Details of this dataset and the underlying model can be found in [R] anova and in Winer, Brown, and Michels (1991).

Here are the data:

. use http://www.stata-press.com/data/r13/t713, clear
(T7.13 -- Winer, Brown, Michels)
 
. tabdisp subject dial period, by(noise) cell(score) stubwidth(11)
 
noise
background
and subject 10 minute time periods and dial
nested in 1   2   3
noise 1 2 3 1 2 3 1 2 3
1
1 45 53 60 40 52 57 28 37 46
2 35 41 50 30 37 47 25 32 41
3 60 65 75 58 54 70 40 47 50
2
1 50 48 61 25 34 51 16 23 35
2 42 45 55 30 37 43 22 27 37
3 56 60 77 40 39 57 31 29 46

Here are the ANOVA results for these data:

. anova score noise / subject|noise period noise#period / period#subject|noise dial 
> noise#dial / dial#subject|noise period#dial noise#period#dial, repeated(period dial)

                                        Number of obs =      54     R-squared     =  0.9872
                                        Root MSE      = 2.81859     Adj R-squared =  0.9576

Source Partial SS df MS F Prob > F
Model 9797.72222 37 264.803303 33.33 0.0000
noise 468.166667 1 468.166667 0.75 0.4348
subject|noise 2491.11111 4 622.777778
period 3722.33333 2 1861.16667 63.39 0.0000
noise#period 333 2 166.5 5.67 0.0293
period#subject|noise 234.888889 8 29.3611111
dial 2370.33333 2 1185.16667 89.82 0.0000
noise#dial 50.3333333 2 25.1666667 1.91 0.2102
dial#subject|noise 105.555556 8 13.1944444
period#dial 10.6666667 4 2.66666667 0.34 0.8499
noise#period#dial 11.3333333 4 2.83333333 0.36 0.8357
Residual 127.111111 16 7.94444444
Total 9924.83333 53 187.261006
Between-subjects error term: subject|noise Levels: 6 (4 df) Lowest b.s.e. variable: subject Covariance pooled over: noise (for repeated variables) Repeated variable: period Huynh-Feldt epsilon = 1.0668 *Huynh-Feldt epsilon reset to 1.0000 Greenhouse-Geisser epsilon = 0.6476 Box's conservative epsilon = 0.5000
Prob > F
Source df F Regular H-F G-G Box
period 2 63.39 0.0000 0.0000 0.0003 0.0013
noise#period 2 5.67 0.0293 0.0293 0.0569 0.0759
period#subject|noise 8
Repeated variable: dial Huynh-Feldt epsilon = 2.0788 *Huynh-Feldt epsilon reset to 1.0000 Greenhouse-Geisser epsilon = 0.9171 Box's conservative epsilon = 0.5000
Prob > F
Source df F Regular H-F G-G Box
dial 2 89.82 0.0000 0.0000 0.0000 0.0007
noise#dial 2 1.91 0.2102 0.2102 0.2152 0.2394
dial#subject|noise 8
Repeated variables: period#dial Huynh-Feldt epsilon = 1.3258 *Huynh-Feldt epsilon reset to 1.0000 Greenhouse-Geisser epsilon = 0.5134 Box's conservative epsilon = 0.2500
Prob > F
Source df F Regular H-F G-G Box
period#dial 4 0.34 0.8499 0.8499 0.7295 0.5934
noise#period#dial 4 0.36 0.8357 0.8357 0.7156 0.5825
Residual 16

Again we see that in addition to the main ANOVA table we obtain an adjusted table for each repeated variable (and their interaction). This result gives the epsilon adjustments to the p-values for those terms in the model involving the repeated measures variable(s).


A complicated design with two repeated variables

This example is an expanded version of the last example in the single repeated-variable section of this document (a complicated design with one repeated variable). The original data and example were taken from table 9–11 of Myers (1966). I added another repeated-measures variable, D, with three levels (thus expanding the data by a factor of three). I created a fake res variable to replace the one provided in table 9–11 of Myers (1966). The new model is much larger than the original since D is interacted with all of the other terms in the model.

Here is part of the data:

 . list, sep(12)
 
  A G B S C D res
1. 1 1 1 1 1 1 22
2. 1 1 1 1 1 2 23
3. 1 1 1 1 1 3 29
4. 1 1 1 1 2 1 28
5. 1 1 1 1 2 2 30
6. 1 1 1 1 2 3 34
7. 1 1 1 1 3 1 41
8. 1 1 1 1 3 2 42
9. 1 1 1 1 3 3 45
10. 1 1 1 2 1 1 15
11. 1 1 1 2 1 2 19
12. 1 1 1 2 1 3 15
13. 1 1 1 2 2 1 31
14. 1 1 1 2 2 2 31
15. 1 1 1 2 2 3 30
 
...
 
133. 2 4 2 15 3 1 67
134. 2 4 2 15 3 2 67
135. 2 4 2 15 3 3 71
136. 2 4 2 16 1 1 48
137. 2 4 2 16 1 2 51
138. 2 4 2 16 1 3 48
139. 2 4 2 16 2 1 56
140. 2 4 2 16 2 2 61
141. 2 4 2 16 2 3 60
142. 2 4 2 16 3 1 76
143. 2 4 2 16 3 2 75
144. 2 4 2 16 3 3 78

Following the lead of Myers (1966), I want to create an ANOVA table with the following information:

Model Term F-Test
Between S
Between G
A MS(A) / MS(G|A)
G|A
Within G
B MS(B) / MS(B#G|A)
B#A MS(B#A) / MS(B#G|A)
B#G|A MS(B#G|A) / MS(S|B#G|A)
S|B#G|A
Within S
C MS(C) / MS(C#G|A)
C#A MS(C#A) / MS(C#G|A)
C#G|A MS(C#G|A) / MS(C#B#G|A)
C#B MS(C#B) / MS(C#B#G|A)
C#B#A MS(C#B#A) / MS(C#B#G|A)
C#B#G|A MS(C#B#G|A) / MS(C#S|B#G|A)
C#S|B#G|A
D MS(D) / MS(D#G|A)
D#A MS(D#A) / MS(D#G|A)
D#G|A MS(D#G|A) / MS(D#B#G|A)
D#B MS(D#B) / MS(D#B#G|A)
D#B#A MS(D#B#A) / MS(D#B#G|A)
D#B#G|A MS(D#B#G|A) / MS(D#S|B#G|A)
D#S|B#G|A
D#C MS(D#C) / MS(D#C#G|A)
D#C#A MS(D#C#A) / MS(D#C#G|A)
D#C#G|A MS(D#C#G|A) / MS(D#C#B#G|A)
D#C#B MS(D#C#B) / MS(D#C#B#G|A)
D#C#B#A MS(D#C#B#A) / MS(D#C#B#G|A)
D#C#B#G|A MS(D#C#B#G|A) / MS(D#C#S|B#G|A)
D#C#S|B#G|A

By writing the anova model in natural order (see above) and using the “/” notation, I can get all but three of the tests outlined above with one call to anova. The other three tests (on C#G|A, D#B|A, and D#C#G|A) can be obtained using the test command.

As more terms are added to the model, the matsize must be set higher to accommodate the larger model. Here I had to set the matsize to 2322. Also realize that with large designs it may take a while to run. Depending on the speed of your computer, you will probably see Stata pausing for a while then printing out a few lines of output and then pausing again. This is normal behavior.

Here is the anova run:

. set matsize 2322    

Current memory allocation

                    current                                 memory usage
    settable          value     description                 (1M = 1024k)
set maxvar 5000 max. variables allowed 1.909M
set memory 50M max. data space 50.000M
set matsize 2322 max. RHS vars in models 41.330M
93.239M
. anova res A / G|A B B#A / B#G|A / S|B#G|A C C#A / C#G|A C#B C#B#A / C#B#G|A / C#S|B#G|A D > D#A / D#G|A D#B D#B#A / D#B#G|A / D#S|B#G|A D#C D#C#A / D#C#G|A D#C#B D#C#B#A / D#C#B#G|A > / , repeated(C D) Number of obs = 144 R-squared = 0.9966 Root MSE = 2.40875 Adj R-squared = 0.9848
Source Partial SS df MS F Prob > F
Model 54466.9722 111 490.693443 84.57 0.0000
A 10201 1 10201 23.46 0.0401
G|A 869.805556 2 434.902778
B 3948.02778 1 3948.02778 6.30 0.1288
B#A 5184 1 5184 8.27 0.1026
B#G|A 1253.80556 2 626.902778
B#G|A 1253.80556 2 626.902778 17.95 0.0011
S|B#G|A 279.333333 8 34.9166667
C 25644.4306 2 12822.2153 36.24 0.0027
C#A 75.875 2 37.9375 0.11 0.9008
C#G|A 1415.19444 4 353.798611
C#B 574.013889 2 287.006944 1.99 0.2515
C#B#A 98.2916667 2 49.1458333 0.34 0.7303
C#B#G|A 577.527778 4 144.381944
C#B#G|A 577.527778 4 144.381944 0.57 0.6872
C#S|B#G|A 4042 16 252.625
D 110.722222 2 55.3611111 11.01 0.0236
D#A 1.5 2 .75 0.15 0.8660
D#G|A 20.1111111 4 5.02777778
D#B 1.72222222 2 .861111111 0.08 0.9268
D#B#A 24.5 2 12.25 1.10 0.4156
D#B#G|A 44.4444444 4 11.1111111
D#B#G|A 44.4444444 4 11.1111111 3.78 0.0238
D#S|B#G|A 47 16 2.9375
D#C 2.36111111 4 .590277778 0.25 0.8997
D#C#A 8.5 4 2.125 0.91 0.5012
D#C#G|A 18.6388889 8 2.32986111
D#C#B 2.11111111 4 .527777778 0.42 0.7881
D#C#B#A 12.0833333 4 3.02083333 2.42 0.1334
D#C#B#G|A 9.97222222 8 1.24652778
D#C#B#G|A 9.97222222 8 1.24652778 0.21 0.9859
Residual 185.666667 32 5.80208333
Total 54652.6389 143 382.186286
Between-subjects error term: S|B#G|A Levels: 16 (8 df) Lowest b.s.e. variable: S Covariance pooled over: B#G|A (for repeated variables) Repeated variable: C Huynh-Feldt epsilon = 2.4621 *Huynh-Feldt epsilon reset to 1.0000 Greenhouse-Geisser epsilon = 0.9891 Box's conservative epsilon = 0.5000
Prob > F
Source df F Regular H-F G-G Box
C 2 36.24 0.0027 0.0027 0.0029 0.0265
C#A 2 0.11 0.9008 0.9008 0.8991 0.7744
C#G|A 4
C#B 2 1.99 0.2515 0.2515 0.2524 0.2940
C#B#A 2 0.34 0.7303 0.7303 0.7285 0.6186
C#B#G|A 4
C#B#G|A 4 0.57 0.6872 0.6872 0.6855 0.5861
C#S|B#G|A 16
Repeated variable: D Huynh-Feldt epsilon = 1.5569 *Huynh-Feldt epsilon reset to 1.0000 Greenhouse-Geisser epsilon = 0.7039 Box's conservative epsilon = 0.5000
Prob > F
Source df F Regular H-F G-G Box
D 2 11.01 0.0236 0.0236 0.0481 0.0801
D#A 2 0.15 0.8660 0.8660 0.8028 0.7365
D#G|A 4
D#B 2 0.08 0.9268 0.9268 0.8719 0.8069
D#B#A 2 1.10 0.4156 0.4156 0.4107 0.4039
D#B#G|A 4
D#B#G|A 4 3.78 0.0238 0.0238 0.0446 0.0698
D#S|B#G|A 16
Repeated variables: D#C Huynh-Feldt epsilon = 1.5707 *Huynh-Feldt epsilon reset to 1.0000 Greenhouse-Geisser epsilon = 0.5864 Box's conservative epsilon = 0.2500
Prob > F
Source df F Regular H-F G-G Box
D#C 4 0.25 0.8997 0.8997 0.8155 0.6647
D#C#A 4 0.91 0.5012 0.5012 0.4786 0.4404
D#C#G|A 8
D#C#B 4 0.42 0.7881 0.7881 0.7053 0.5820
D#C#B#A 4 2.42 0.1334 0.1334 0.1891 0.2598
D#C#B#G|A 8
D#C#B#G|A 8 0.21 0.9859 0.9859 0.9454 0.8112
Residual 32
. test C#G|A / C#B#G|A
Source Partial SS df MS F Prob > F
C#G|A 1415.19444 4 353.798611 2.45 0.2033
C#B#G|A 577.527778 4 144.381944
. test D#G|A / D#B#G|A
Source Partial SS df MS F Prob > F
D#G|A 20.1111111 4 5.02777778 0.45 0.7693
D#B#G|A 44.4444444 4 11.1111111
. test D#C#G|A / D#C#B#G|A
Source Partial SS df MS F Prob > F
D#C#G|A 18.6388889 8 2.32986111 1.87 0.1975
D#C#B#G|A 9.97222222 8 1.24652778

With complicated designs, you might need a larger matrix than Stata allows. If you get a “matsize too small” error, you can use the dropemptycells option to eliminate empty cells from the design matrix.

Stata will allow up to four repeated-measures variables in the repeated() option and can handle even more complicated designs than presented here. The most limiting thing you will find with complicated designs is the maximum matrix size allowed by Stata.


Summary

I have presented seven examples involving one repeated-measurement variable. These examples range from the simplest design to a complicated design. With all of these examples, I discussed the use of both anova with the repeated() option and wsanova (Gleason 1999).

For simple designs involving only one repeated-measures variable, the wsanova command syntax might be most natural, depending on how you think about ANOVA models. With more complicated designs, I advise that you first understand the underlying model you are trying to estimate and then use the anova command to get what you need.

I presented three examples involving two repeated-measures variables (Stata allows up to four repeated-measures variables). These examples also ranged from simple to complex. With these examples I demonstrated only the anova command because the wsanova command is not designed to handle multiple repeated measures.

In the course of showing these examples, I also outlined the errors users sometimes make and the solutions to those errors. Here is a summary of common mistakes and solutions:

Many problems can be avoided by first understanding your underlying model. As the design becomes more complicated, this understanding becomes more crucial. Books that cover ANOVA in detail such as Winer, Brown, and Michels (1991) can help you understand “fixed effects”, “random effects”, “nesting”, “crossing”, “expected mean squares”, and determining the appropriate error terms to use in your ANOVA F tests.


References

Cole, J. W. L., and J. E. Grizzle. 1966.
Applications of multivariate analysis of variance to repeated measures experiments. Biometrics 22: 810–828.
Gleason, J. R. 1999.
Within subjects (repeated measures) ANOVA, including between subjects factors. Stata Technical Bulletin 47: 40-45. Reprinted in Stata Technical Bulletin Reprints, vol. 8, pp. 236–243.
Myers, J. L. 1966.
Fundamentals of Experimental Design. Boston: Allyn and Bacon.
Winer, B. J., D. R. Brown, and K. M. Michels. 1991.
Statistical Principles in Experimental Design. 3rd Edition. New York: McGraw–Hill.
The Stata Blog: Not Elsewhere Classified Find us on Facebook Follow us on Twitter LinkedIn Google+ Watch us on YouTube