Search
   >> Home >> Resources & support >> FAQs >> Chow tests
The following FAQ is based on an exchange that started on Statalist.

Can you explain Chow tests?

Title   Chow tests
Author William Gould, StataCorp
Date January 2002; updated July 2011

Privately I was asked yet another question on Chow tests. The question started out “Is a Chow test the correct test to determine whether data can be pooled together?” and went on from there. Like many on the list, I am tired of seeing and answering questions on Chow tests. I do not blame the questioner for asking; I blame their teachers for confusing them with what is, these days, unnecessary jargon.

In the past, I have always given in and cast my answer in Chow-test terms. In this reply, I try a different approach and, I think, the result is more useful.

This reply concerns linear regression (though the technique is really more general than that), and I gloss over the detail of pooling the residuals and whether the residual variances are really the same. For the last, I think I can be forgiven.

Here is what I wrote:

Is a Chow test the correct test to determine whether data can be pooled together?

A Chow test is simply a test of whether the coefficients estimated over one group of the data are equal to the coefficients estimated over another, and you would be better off to forget the word Chow and remember that definition.

History: In the days when statistical packages were not as sophisticated as they are now, testing whether coefficients were equal was not so easy. You had to write your own program, typically in FORTRAN. Chow showed a way you could perform a Wald test based on statistics that were commonly reported, and that would produce the same result as if you performed the Wald test.

What does it mean “whether data can be pooled together”? Do you often meet nonprofessionals who say to you, “I was wondering whether the data could be pooled?” Forget that phrase, too: it is another piece of jargon for testing whether the behavior is the same, as measured by whether the coefficients are the same.

Let’s pretend that you have some model and two or more groups of data. Your model predicts something about the behavior within the group based on certain characteristics that vary within the group. Under the assumption that each group's behavior is unique, you have

        y_1 = X_1*b_1 + u_1                   (equation for group 1)
        y_2 = X_2*b_2 + u_2                   (equation for group 2)

and so on. Now you want to test whether the behavior for one group is the same as for another, which means you want to test

        b_1 = b_2 = ...

How do you do that? Testing coefficients across separately fitted models is difficult to impossible, depending on things we need not go into right now. A trick is to “pool” the data to convert the multiple equations into one giant equation

        y   = d1*(X_1*b1 + u1) + d2*(X_2*b2 + u2) + ...

where y is the set of all outcomes (y_1, y_2, ...), and d1 is a variable that is 1 when the data are for group 1 and 0 otherwise, d2 is 1 when the data are for group 2 and 0 otherwise, ....

Notice that from the above I can retrieve the original equations. Setting d1=1 and d2=d3=...=0, I get the equation for group 1; setting d1=0 and d2=1 and d3=...=0, I get the equation for group 2; and so on.

Now let’s start with

        y   = d1*(X_1*b1 + u1) + d2*(X_2*b2 + u2) + ...

and rewrite it by a little algebraic manipulation:

        y   = d1*(X_1*b1 + u1) + d2*(X_2*b2 + u2) + ...
            = d1*X_1*b1 + d1*u2 + d2*X_2*b2 + d2*u2 + ...
            = d1*X_1*b1 + d2*X_2*b2 + ... + d1*u1 + d2*u2 + ...
            = X_1*d1*b1 + X_2*d2*b2 + ... + d1*u1 + d2*u2 + ...
            = (X_1*d1)*b1 + (X_2*d2)*b2 + ... + d1*u1 + d2*u2 + ...

By stacking the data, I can obtain estimates of b1, b2, ...

I include not X_1 in my model, but X_1*d1 (a set of variables equal to X_1 when group is 1 and 0 otherwise); I include not X_2 in my model, but X_2*d2 (a set of variables equal to X_2 when group is 2 and 0 otherwise); and so on.

Let’s use auto.dta and pretend that I have two groups.

. sysuse auto
. generate group1=rep78==3
. generate group2=group1==0

I could fit the models separately:

. regress price mpg weight if group1==1

Source SS df MS Number of obs = 30
F( 2, 27) = 16.20
Model 196545318 2 98272658.8 Prob > F = 0.0000
Residual 163826398 27 6067644.36 R-squared = 0.5454
Adj R-squared = 0.5117
Total 360371715 29 12426610.9 Root MSE = 2463.3
price Coef. Std. Err. t P>|t| [95% Conf. Interval]
mpg 13.14912 184.5661 0.07 0.944 -365.5492 391.8474
weight 3.517687 1.015855 3.46 0.002 1.433324 5.60205
_cons -5431.147 6599.898 -0.82 0.418 -18973.02 8110.725
. regress price mpg weight if group2==1
Source SS df MS Number of obs = 44
F( 2, 41) = 5.16
Model 54562909.6 2 27281454.8 Prob > F = 0.0100
Residual 216614915 41 5283290.61 R-squared = 0.2012
Adj R-squared = 0.1622
Total 271177825 43 6306461.04 Root MSE = 2298.5
price Coef. Std. Err. t P>|t| [95% Conf. Interval]
mpg -170.5474 93.3656 -1.83 0.075 -359.103 18.0083
weight .0527381 .8064713 0.07 0.948 -1.575964 1.68144
_cons 9685.028 4190.693 2.31 0.026 1221.752 18148.3

I could fit the combined model:

. generate mpg1=mpg*group1
. generate weight1=weight*group1

. generate mpg2=mpg*group2
. generate weight2=weight*group2

. regress price group1 mpg1 weight1 group2 mpg2 weight2, noconstant 

Source SS df MS Number of obs = 74
F( 6, 68) = 91.38
Model 3.0674e+09 6 511232168 Prob > F = 0.0000
Residual 380441313 68 5594725.19 R-squared = 0.8897
Adj R-squared = 0.8799
Total 3.4478e+09 74 46592355.7 Root MSE = 2365.3
price Coef. Std. Err. t P>|t| [95% Conf. Interval]
group1 -5431.147 6337.479 -0.86 0.394 -18077.39 7215.096
mpg1 13.14912 177.2275 0.07 0.941 -340.5029 366.8012
weight1 3.517687 .9754638 3.61 0.001 1.571179 5.464194
group2 9685.028 4312.439 2.25 0.028 1079.69 18290.37
mpg2 -170.5474 96.07802 -1.78 0.080 -362.2681 21.17334
weight2 .0527381 .8299005 0.06 0.950 -1.603303 1.708779

What is this noconstant option? We must remember that when we fit the separate models, each has its own intercept. There was an intercept in X_1, X_2, and so on. What I have done above is literally translate

        y   = (X_1*d1)*b1 + (X_2*d2)*b2 + d1*u1 + d2*u2

and included the variables group1 and group2 (variables equal to 1 for their respective groups) and told Stata to omit the overall intercept.

I do not recommend you fit the model the way I have just illustrated because of numerical concerns—we will get to that later. Fit the models separately or jointly, and you will get the same estimates for b_1 and b_2.

Now we can test whether the coefficients are the same for the two groups:

 . test _b[mpg1]=_b[mpg2], notest

  ( 1)  mpg1 - mpg2 = 0

 . test _b[weight1]=_b[weight2], accum

  ( 1)  mpg1 - mpg2 = 0
  ( 2)  weight1 - weight2 = 0

        F(  2,    68) =    5.61
             Prob > F =    0.0056

That is the Chow test. Something was omitted: the intercept. If we really wanted to test whether the two groups were the same, we would would test

 . test _b[mpg1]=_b[mpg2]

  ( 1)  mpg1 - mpg2 = 0

 . test _b[weight1]=_b[weight2], accum

  ( 1)  mpg1 - mpg2 = 0
  ( 2)  weight1 - weight2 = 0

 . test _b[group1]=_b[group2], accum

  ( 1)  mpg1 - mpg2 = 0
  ( 2)  weight1 - weight2 = 0
  ( 3)  group1 - group2 = 0

        F(  3,    68) =    4.07
             Prob > F =    0.0102

Using this approach, however, we are not tied down by what the “Chow test” can test. We can formulate any hypothesis we want. We might think that weight works the same way in both groups but that mpg works differently, and each group has its own intercept. Then we could test

 . test _b[mpg1]=_b[mpg2]

  ( 1)  mpg1 - mpg2 = 0

        F(  1,    68) =    0.83
             Prob > F =    0.3654

by itself. If we had more variables, we could test any subset of variables.

Is “pooling the data” justified? Of course it is: we just established that pooling the data is just another way of fitting separate models and that fitting separate models is certainly justified—we got the same coefficients. That’s why I told you to forget the phrase about whether pooling the data is justified. People who ask that do not really mean to ask what they are saying; they mean to ask whether the coefficients are the same. In that case, they should say that. Pooling is always justified, and it corresponds to nothing more than the mathematical trick of writing separate equations,

        y_1 = X_1*b_1 + u_1                   (equation for group 1)
        y_2 = X_2*b_2 + u_2                   (equation for group 2)

as one equation

        y   = (X_1*d1)*b1 + (X_2*d2)*b2 + d1*u1 + d2*u2

There are many ways I can write the above equation, and I want to write it a little differently because of numerical concerns. Starting with

        y   = (X_1*d1)*b1 + (X_2*d2)*b2 + d1*u1 + d2*u2

let’s do some algebra to obtain

        y   = X*b1 + d2*X_2*(b2-b1)     + d1*u1 + d2*u2

where X = (X_1, X_2). In this formulation, I measure not b1 and b2, but b1 and (b2−b1). This is numerically more stable, and I can still test that b2==b1 by testing whether (b2−b1)=0.

Let’s fit this model

. regress price mpg weight mpg2 weight2 group2

Source SS df MS Number of obs = 74
F( 5, 68) = 9.10
Model 254624083 5 50924816.7 Prob > F = 0.0000
Residual 380441313 68 5594725.19 R-squared = 0.4009
Adj R-squared = 0.3569
Total 635065396 73 8699525.97 Root MSE = 2365.3
price Coef. Std. Err. t P>|t| [95% Conf. Interval]
mpg 13.14912 177.2275 0.07 0.941 -340.5029 366.8012
weight 3.517687 .9754638 3.61 0.001 1.571179 5.464194
mpg2 -183.6965 201.5951 -0.91 0.365 -585.9733 218.5803
weight2 -3.464949 1.280728 -2.71 0.009 -6.020602 -.9092956
group2 15116.17 7665.557 1.97 0.053 -180.2075 30412.56
_cons -5431.147 6337.479 -0.86 0.394 -18077.39 7215.096

and, if I want to test whether the coefficients are the same, I can do

 . test _b[mpg2]=0, notest

  ( 1)  mpg2 = 0

 . test _b[weight2]=0, accum
  
  ( 1)  mpg2 = 0
  ( 2)  weight2 = 0

        F(  2,    68) =    5.61
             Prob > F =    0.0056

and that gives the same answer yet again. If I want to test whether *ALL* the coefficients are the same (including the intercept), I can type

 . test _b[mpg2]=0, notest

  ( 1)  mpg2 = 0

 . test _b[weight2]=0, accum notest

  ( 1)  mpg2 = 0
  ( 2)  weight2 = 0

 . test _b[group2]=0, accum

  ( 1)  mpg2 = 0
  ( 2)  weight2 = 0
  ( 3)  group2 = 0

        F(  3,    68) =    4.07
             Prob > F =    0.0102

Just as before, I can test any subset.

Using this difference formulation, if I had three groups, starting with

        y   = (X_1*d1)*b1 + (X_2*d2)*b2 + (X_3*d3)*b3 + d1*u1 + d2*u2 + d3*u3

as

        y   = X*b1 + (X_2*d2)*(b2-b1) + (X_3*d3)*(b3-b1) + d1*u1 + d2*u2 + d3*u3

Let’s create the group variables and fit this model:

. sysuse auto, clear

. generate group1=rep78==3
. generate group2=rep78==4
. generate group3=(group1+group2)==0
 
. generate mpg1=mpg*group1
. generate weight1=weight*group1

. generate mpg2=mpg*group2
. generate weight2=weight*group2

. generate mpg3=mpg*group3
. generate weight3=weight*group3

. regress price mpg weight mpg2 weight2 group2 mpg3 weight3 group3

Source SS df MS Number of obs = 74
F( 8, 65) = 5.80
Model 264415585 8 33051948.1 Prob > F = 0.0000
Residual 370649811 65 5702304.78 R-squared = 0.4164
Adj R-squared = 0.3445
Total 635065396 73 8699525.97 Root MSE = 2387.9
price Coef. Std. Err. t P>|t| [95% Conf. Interval]
mpg 13.14912 178.9234 0.07 0.942 -344.1855 370.4837
weight 3.517687 .9847976 3.57 0.001 1.55091 5.484463
mpg2 130.5261 336.6547 0.39 0.699 -541.8198 802.872
weight2 -2.18337 1.837314 -1.19 0.239 -5.85274 1.486
group2 4560.193 12222.22 0.37 0.710 -19849.27 28969.66
mpg3 -194.1974 216.3459 -0.90 0.373 -626.27 237.8752
weight3 -3.160952 1.73308 -1.82 0.073 -6.622152 .3002481
group3 14556.66 9167.998 1.59 0.117 -3753.101 32866.41
_cons -5431.147 6398.12 -0.85 0.399 -18209.07 7346.781

If I want to test whether the three groups were the same in the Wald-test sense, I can type

. test (_b[mpg2]=0) (_b[weight2]=0) (_b[group2]=0) /*
>   */ (_b[mpg3]=0) (_b[weight3]=0) (_b[group3]=0)

( 1)  mpg2 = 0
( 2)  weight2 = 0
( 3)  group2 = 0
( 4)  mpg3 = 0
( 5)  weight3 = 0
( 6)  group3 = 0

        F(  6,    65) =    2.28
             Prob > F =    0.0463

I could more easily type the above command as

. testparm mpg2 weight2 group2 mpg3 weight3 group3

 ( 1)  mpg2 = 0
 ( 2)  weight2 = 0
 ( 3)  group2 = 0
 ( 4)  mpg3 = 0
 ( 5)  weight3 = 0
 ( 6)  group3 = 0

       F(  6,    65) =    2.28
            Prob > F =    0.0463

Alternatively, we can use factor variables and contrast to perform the same test:

. generate group=cond(rep78==3,1,cond(rep78==4,2,3))

. regress price c.mpg##i.group c.weight##i.group

Source SS df MS Number of obs = 74
F( 8, 65) = 5.80
Model 264415585 8 33051948.1 Prob > F = 0.0000
Residual 370649811 65 5702304.78 R-squared = 0.4164
Adj R-squared = 0.3445
Total 635065396 73 8699525.97 Root MSE = 2387.9
price Coef. Std. Err. t P>|t| [95% Conf. Interval]
mpg 13.14912 178.9234 0.07 0.942 -344.1855 370.4837
group
2 4560.193 12222.22 0.37 0.710 -19849.27 28969.66
3 14556.66 9167.998 1.59 0.117 -3753.101 32866.41
group#c.mpg
2 130.5261 336.6547 0.39 0.699 -541.8198 802.872
3 -194.1974 216.3459 -0.90 0.373 -626.27 237.8752
weight 3.517687 .9847976 3.57 0.001 1.55091 5.484463
group#c.weight
2 -2.18337 1.837314 -1.19 0.239 -5.85274 1.486
3 -3.160952 1.73308 -1.82 0.073 -6.622152 .3002481
_cons -5431.147 6398.12 -0.85 0.399 -18209.07 7346.781
. contrast group group#c.mpg group#c.weight, overall Contrasts of marginal linear predictions Margins : asbalanced
df F P>F
group 2 1.29 0.2835
group#c.mpg 2 0.78 0.4617
group#c.weight 2 1.88 0.1602
Overall 6 2.28 0.0463
Residual 65
The Stata Blog: Not Elsewhere Classified Find us on Facebook Follow us on Twitter LinkedIn Google+ Watch us on YouTube