Re: st: GLM and ANOVA complaints

[David Airey <david.airey@vanderbilt.edu>]

Since my ANOVA was not finishing anytime soon, and some interesting  
posts were being added to the thread, I broke out and have posted the  
do file and partial results below. You can see that the timing of this  
model is astonishingly slow, which makes me suspicious. The way I wrote  
the model did not leave anything to the residual, but this can be  
modified by moving the term D*T*P*I*A|S*F to the residual. In terms of  
timing to complete computations I don't think this matters. Joseph's  
model is not what I had in mind. By 4 within-subject factors I mean  
repeated measures. In his model there is no repeated() option  
indicating this.

My actual data set has instead of 2 levels of strain, 20 levels of  
strain and 38,400 observations. Such a data set does not compute in  
Stata 8/SE using ANOVA (Previously, Ken Higbee point out that this  
might be another story if use MANOVA, because it leaves the within  
factors and between factors on different sides of the model and a  
smaller X'X requirement.). Monday I meet with a statistician who helps  
run Vanderbilt's VAMPIRE parallel system. We'll determine where SAS's  
desktop limits lie with this data set, first. I've posted here because  
it could be that something is wrong with my computer or do-file that is  
causing this molasses-like performance. I was hoping by posting this  
do-file that some users with fast PC's could run my do file and tell me  
if the problem is my computer or the OS X version of Stata.

I apologize if what gets posted is not in monotype font. It always is  
when I send.

-Dave Airey

------------------------------------------------------------------------ 
--------
       log:  /Users/dairey/Desktop/mice/design/null_small.smcl
  log type:  smcl
 opened on:  26 Sep 2003, 20:46:22

.
. /* Behavior design:
>
> between subjects effects
>    strain (2) = S
>    sex (2) = F
>
> animals = A
>    2 per sex and strain combination (8 total)
>
> within subjects effects
>    drug (3) = D
>    period (4) = T
>    intensity (2) = P
>    interval (2) = I
>
> total observations = (2 * 2 * 4 * 3 * 2 * 2 * 2)  = 384
>
> */
.
. clear

. set obs 384
obs was 0, now 384

.
. egen I = seq(), from(1) to(2) block(1)

. egen P = seq(), from(1) to(2) block(2)

. egen T = seq(), from(1) to(4) block (4)

. egen D = seq(), from(1) to(3) block(16)

. egen A = seq(), from(1) to(8) block(48)

. egen F = seq(), from(1) to(2) block(96)

. egen S = seq(), from(1) to(2) block(192)

. set seed 1234

. gen y = invnorm(uniform())

. outsheet using null_small.txt, replace

.
. set more off

.
. anova y S F S*F / A|S*F ///
>         D S*D F*D S*F*D / D*A|S*F ///
>         T S*T F*T S*F*T / T*A|S*F ///
>         P S*P F*P S*F*P / P*A|S*F ///
>         I S*I F*I S*F*I / I*A|S*F ///
>         D*T S*D*T F*D*T S*F*D*T / D*T*A|S*F ///
>         D*P S*D*P F*D*P S*F*D*P / D*P*A|S*F ///
>         D*I S*D*I F*D*I S*F*D*I / D*I*A|S*F ///
>         T*P S*T*P F*T*P S*F*T*P / T*P*A|S*F ///
>         T*I S*T*I F*T*I S*F*T*I / T*I*A|S*F ///
>         P*I S*P*I F*P*I S*F*P*I / P*I*A|S*F ///
>         D*T*P S*D*T*P F*D*T*P S*F*D*T*P / D*T*P*A|S*F ///
>         D*T*I S*D*T*I F*D*T*I S*F*D*T*I / D*T*I*A|S*F ///
>         D*P*I S*D*P*I F*D*P*I S*D*F*P*I / D*P*I*A|S*F ///
>         T*P*I S*T*P*I F*T*P*I S*F*T*P*I / T*P*I*A|S*F ///
>         D*T*P*I S*D*T*P*I F*D*T*P*I / D*T*P*I*A|S*F ///
>         S*F*D*T*P*I /, ///
>         repeated(D T P I)

                           Number of obs =     384     R-squared     =   
1.0000
                           Root MSE      =       0     Adj R-squared =

                  Source |  Partial SS    df       MS           F      
Prob > F
            
--------------+----------------------------------------------------
                   Model |   405.31568   383  1.05826548
                         |
                       S |  2.88419294     1  2.88419294       3.34      
0.1416
                       F |  .228571433     1  .228571433       0.26      
0.6340
                     S*F |  .071187734     1  .071187734       0.08      
0.7883
                   A|S*F |  3.45426619     4  .863566547
            
--------------+----------------------------------------------------
                       D |  .492576154     2  .246288077       0.22      
0.8052
                     S*D |   .43288709     2  .216443545       0.20      
0.8261
                     F*D |  3.33579223     2  1.66789611       1.51      
0.2782
                   S*F*D |  2.30196723     2  1.15098362       1.04      
0.3966
                 D*A|S*F |  8.84971212     8  1.10621402
            
--------------+----------------------------------------------------
                       T |  1.46316172     3  .487720573       0.84      
0.4963
                     S*T |  9.44018353     3  3.14672784       5.44      
0.0135
                     F*T |  1.20527617     3  .401758722       0.69      
0.5730
                   S*F*T |  1.40904977     3  .469683256       0.81      
0.5115
                 T*A|S*F |   6.9424209    12  .578535075
            
--------------+----------------------------------------------------
                       P |  1.13630778     1  1.13630778       2.35      
0.2000
                     S*P |  .000906918     1  .000906918       0.00      
0.9675
                     F*P |   .30154612     1   .30154612       0.62      
0.4738
                   S*F*P |  .117438218     1  .117438218       0.24      
0.6479
                 P*A|S*F |  1.93316363     4  .483290908
            
--------------+----------------------------------------------------
                       I |  .001341356     1  .001341356       0.00      
0.9686
                     S*I |  .535697234     1  .535697234       0.70      
0.4500
                     F*I |  .867663226     1  .867663226       1.13      
0.3471
                   S*F*I |  1.81116971     1  1.81116971       2.37      
0.1989
                 I*A|S*F |  3.06276128     4  .765690319
            
--------------+----------------------------------------------------
                     D*T |  4.80299014     6  .800498357       0.65      
0.6926
                   S*D*T |  6.28382994     6  1.04730499       0.85      
0.5477
                   F*D*T |   1.5040146     6    .2506691       0.20      
0.9727
                 S*F*D*T |  13.7968781     6  2.29947968       1.86      
0.1303
               D*T*A|S*F |  29.7289332    24  1.23870555
            
--------------+----------------------------------------------------
                     D*P |  3.64908046     2  1.82454023       2.17      
0.1765
                   S*D*P |  .201221515     2  .100610758       0.12      
0.8887
                   F*D*P |  4.09528487     2  2.04764244       2.44      
0.1491
                 S*F*D*P |   .30983134     2   .15491567       0.18      
0.8351
               D*P*A|S*F |  6.72284922     8  .840356152
            
--------------+----------------------------------------------------
                     D*I |   .92948765     2  .464743825       1.00      
0.4090
                   S*D*I |  5.48050997     2  2.74025499       5.91      
0.0266
                   F*D*I |   7.4009702     2   3.7004851       7.98      
0.0124
                 S*F*D*I |  5.33961937     2  2.66980969       5.76      
0.0283
               D*I*A|S*F |  3.71079174     8  .463848968
            
--------------+----------------------------------------------------
                     T*P |  6.97281454     3  2.32427151       3.41      
0.0531
                   S*T*P |  4.96822244     3  1.65607415       2.43      
0.1158
                   F*T*P |  1.12573865     3  .375246215       0.55      
0.6573
                 S*F*T*P |  17.9134982     3  5.97116608       8.76      
0.0024
               T*P*A|S*F |  8.17922148    12   .68160179
            
--------------+----------------------------------------------------
                     T*I |  .557579681     3  .185859894       0.09      
0.9623
                   S*T*I |  .833987016     3  .277995672       0.14      
0.9345
                   F*T*I |  .080230349     3   .02674345       0.01      
0.9978
                 S*F*T*I |  3.79065925     3  1.26355308       0.63      
0.6073
               T*I*A|S*F |  23.9241218    12  1.99367682
            
--------------+----------------------------------------------------
                     P*I |  .000169583     1  .000169583       0.00      
0.9931
                   S*P*I |     .055236     1     .055236       0.03      
0.8756
                   F*P*I |  1.26872301     1  1.26872301       0.64      
0.4689
                 S*F*P*I |   .30558143     1   .30558143       0.15      
0.7149
               P*I*A|S*F |   7.9444391     4  1.98610977
            
--------------+----------------------------------------------------
                   D*T*P |  6.54535065     6  1.09089177       1.50      
0.2215
                 S*D*T*P |  12.5280613     6  2.08801021       2.87      
0.0300
                 F*D*T*P |  7.63558042     6  1.27259674       1.75      
0.1534
               S*F*D*T*P |  1.48628511     6  .247714185       0.34      
0.9088
             D*T*P*A|S*F |  17.4882934    24  .728678892
            
--------------+----------------------------------------------------
                   D*T*I |  6.62821026     6  1.10470171       1.15      
0.3647
                 S*D*T*I |    7.854499     6  1.30908317       1.36      
0.2693
                 F*D*T*I |  1.95479897     6  .325799828       0.34      
0.9092
               S*F*D*T*I |    4.874086     6  .812347667       0.85      
0.5474
             D*T*I*A|S*F |  23.0486542    24  .960360593
            
--------------+----------------------------------------------------
                   D*P*I |   4.5983398     2   2.2991699       3.32      
0.0893
                 S*D*P*I |  1.35994576     2  .679972879       0.98      
0.4158
                 F*D*P*I |  5.45282793     2  2.72641396       3.93      
0.0646
               S*D*F*P*I |  2.90376777     2  1.45188388       2.09      
0.1855
             D*P*I*A|S*F |  5.54424817     8  .693031021
            
--------------+----------------------------------------------------
                   T*P*I |  2.65651896     3  .885506321       0.96      
0.4421
                 S*T*P*I |  .415394971     3   .13846499       0.15      
0.9274
                 F*T*P*I |  3.17494885     3  1.05831628       1.15      
0.3688
               S*F*T*P*I |  7.11547503     3  2.37182501       2.58      
0.1024
             T*P*I*A|S*F |  11.0439513    12  .920329278
            
--------------+----------------------------------------------------
                 D*T*P*I |  2.69605069     6  .449341782       0.53      
0.7818
               S*D*T*P*I |  7.57132999     6  1.26188833       1.48      
0.2266
               F*D*T*P*I |  7.17993817     6  1.19665636       1.40      
0.2534
           D*T*P*I*A|S*F |  20.4420825    24  .851753437
            
--------------+----------------------------------------------------
             S*F*D*T*P*I |  8.96128538     6  1.49354756
                         |
                Residual |           0     0
            
--------------+----------------------------------------------------
                   Total |   405.31568   383  1.05826548


Between-subjects error term:  A|S*F
                     Levels:  8         (4 df)
     Lowest b.s.e. variable:  A
     Covariance pooled over:  S*F       (for repeated variables)


--Break--
r(1);

end of do-file
--Break--
r(1);

. log close
       log:  /Users/dairey/Desktop/mice/design/null_small.smcl
  log type:  smcl
 closed on:  27 Sep 2003, 19:37:33
------------------------------------------------------------------------ 
--------

*
*   For searches and help try:
*   http://www.stata.com/support/faqs/res/findit.html
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/