Multivariate meta-analysis

Let's see it work

• Multivariate meta-analysis

• Example dataset: Treatment of moderate periodontal disease

• Constant-only model: Multivariate meta-analysis

• Incorporating moderators: Multivariate meta-regression

• Random-effects estimation methods and between-study covariance structure

• estat heterogeneity

• Postestimation: Predicting random effects

• Additional resources

Multivariate meta-analysis

Suppose we are interested in investigating the effect of multiple dietary regimens on weight loss. Multiple effect sizes that compare each of these diets with a control group (not following any diet) can be computed. These effect sizes are usually correlated because they share a common control group. Or perhaps we want to explore the impact of a new teaching technique on math (outcome 1), physics (outcome 2), and chemistry (outcome 3) testing scores. Three effect sizes that compare the three testing scores across two groups of students (those who were taught using the new technique and those who were not) are computed. These three effect sizes are dependent because they were calculated using the same set of students.

Example dataset: Treatment of moderate periodontal disease

Consider a dataset from Antczak-Bouckoms et al. (1993) of five randomized controlled trials that explored the impact of two (surgical and nonsurgical) procedures on treating periodontal disease. Two outcomes of interest are improvements from baseline (pretreatment) in probing depth (y1) and attachment level (y2) around the teeth. The main objectives of the periodontal treatment were to reduce probing depths and increase attachment levels (Berkey et al. 1998). Because the two outcomes y1 and y2 are measured on the same subject, they should not be treated as independent.

. webuse periodontal
(Treatment of moderate periodontal disease)

. describe

Contains data from https://www.stata-press.com/data/r17/periodontal.dta
 Observations:             5                  Treatment of moderate periodontal disease
    Variables:             9                  13 Jan 2021 18:11
                                              (_dta has notes)


Variable      Storage   Display    Value
    name         type    format    label      Variable label

trial           str23   %23s                  Trial label
pubyear         byte    %9.0g                 Publication year centered at 1983
y1              float   %6.2f                 Mean improvement in probing depth (mm)
y2              float   %6.2f                 Mean improvement in attachment level (mm)
v11             float   %6.4f                 Variance of y1
v12             float   %6.4f                 Covariance of y1 and y2
v22             float   %6.4f                 Variance of y2
s1              double  %10.0g                Standard error of y1
s2              double  %10.0g                Standard error of y2

 Sorted by:

Variables v11, v12, and v22 define the within-study covariance matrix for each study.

Constant-only model: Multivariate meta-analysis

If we were to perform two separate univariate meta-analyses for outcomes y1 and y2, we would be ignoring the dependence among the two outcomes, which may lead to incorrect inference. We use the command meta mvregress to conduct a bivariate meta-analysis as follows:

. meta mvregress y1 y2, wcovvariables(v11 v12 v22)

Performing EM optimization ...

Performing gradient-based optimization:
Iteration 0:   log restricted-likelihood =  2.0594015
Iteration 1:   log restricted-likelihood =  2.0822925
Iteration 2:   log restricted-likelihood =  2.0823276
Iteration 3:   log restricted-likelihood =  2.0823276

Multivariate random-effects meta-analysis       Number of obs     =         10
Method: REML                                    Number of studies =          5
                                                Obs per study:
                                                              min =          2
                                                              avg =        2.0
                                                              max =          2
                                                Wald chi2(0)      =          .
Log restricted-likelihood = 2.0823276           Prob > chi2       =          .



             
 
 Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
 
 
 
y1           
 
       _cons 
 
   .3534282   .0588486     6.01   0.000      .238087    .4687694
 
 
 
y2           
 
       _cons 
 
  -.3392152   .0879051    -3.86   0.000    -.5115061   -.1669243
 
 
 
Test of homogeneity: Q_M = chi2(8) = 128.23                Prob > Q_M = 0.0000



   Random-effects parameters 
 
   Estimate

Unstructured:                
 
                      sd(y1) 
 
   .1083191
                      sd(y2) 
 
   .1806968
                 corr(y1,y2) 
 
   .6087987

The order in which you specify variables within wcovvariables() is important (see wcovvariables() in [META] meta mvregress for details).

The first table displays the regression (fixed-effects) coefficient estimates from the bivariate meta-analysis. These estimates correspond to the overall bivariate effect sizes. The overall improvement in probing depth is roughly 0.35 mm, and the overall attachment level was reduced by 0.34 mm.

The multivariate homogeneity test, which tests whether the bivariate effect sizes (\( \theta_{1j}, \theta_{2j} \)) are constant across studies, is rejected (p < 0.0001).

The second table displays the standard deviations of the random effects corresponding to outcomes y1 and y2, as well as their correlation.

We could have performed a fixed-effects multivariate meta-analysis by specifying option fixed:

. meta mvregress y1 y2, wcovvariables(v11 v12 v22) fixed
(output omitted)

By performing a fixed-effects multivariate meta-analysis, we assume that study-specific bivariate effect sizes are the same across the studies and that the observed variability is due to sampling error. This assumption is often not satisfied in practice.

Incorporating moderators: Multivariate meta-regression

Berkey et al. (1998) argued that as the surgical experience accumulates, the surgical procedure will become more efficient, so the most recent studies may show greater surgical benefits. We will include variable pubyear, a surrogate for the time when the trial was performed, as a moderator to explain a portion of the heterogeneity highlighted in the previous section.

. meta mvregress y1 y2 = pubyear, wcovvariables(v*)

Performing EM optimization ...

Performing gradient-based optimization:
Iteration 0:   log restricted-likelihood = -3.5544446
Iteration 1:   log restricted-likelihood = -3.5402141
Iteration 2:   log restricted-likelihood = -3.5399568
Iteration 3:   log restricted-likelihood = -3.5399567

Multivariate random-effects meta-regression     Number of obs     =         10
Method: REML                                    Number of studies =          5
                                                Obs per study:
                                                              min =          2
                                                              avg =        2.0
                                                              max =          2
                                                Wald chi2(2)      =       0.40
Log restricted-likelihood = -3.5399567          Prob > chi2       =     0.8197



             
 
 Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
 
 
 
y1           
 
     pubyear 
 
   .0048615   .0218511     0.22   0.824    -.0379658    .0476888
       _cons 
 
   .3587569     .07345     4.88   0.000     .2147975    .5027163
 
 
 
y2           
 
     pubyear 
 
  -.0115367   .0299635    -0.39   0.700     -.070264    .0471907
       _cons 
 
  -.3357368   .0979979    -3.43   0.001    -.5278091   -.1436645
 
 
 
Test of homogeneity: Q_M = chi2(6) = 125.76                Prob > Q_M = 0.0000



   Random-effects parameters 
 
   Estimate

Unstructured:                
 
                      sd(y1) 
 
   .1429917
                      sd(y2) 
 
   .2021314
                 corr(y1,y2) 
 
    .561385

In the wcovvariables() option, we used the stub notation v* to refer to variables v11, v12, and v22.

The estimates of the regression coefficients of variable pubyear are 0.0049 with a 95% CI of [\(−\)0.0380, 0.0477] for outcome y1 and \(-\)0.0115 with a 95% CI of [\(−\)0.0703, 0.0472] for outcome y2. The coefficients are not significant according to the z tests, with the respective p-values, p = 0.824 and p = 0.7. It appears that pubyear did not explain the heterogeneity among the effect sizes.

Random-effects estimation methods and between-study covariance structure

Here we modify the default REML estimation method and use the ML estimation instead. We also use an independent covariance structure for the random effects. This may be done by specifying random(mle, covariance(independent)).

. meta mvregress y*, wcovvariables(v*) random(mle, covariance(independent))

Performing EM optimization ...

Performing gradient-based optimization:
Iteration 0:   log likelihood =  5.1641932
Iteration 1:   log likelihood =  5.1654142
Iteration 2:   log likelihood =  5.1654153
Iteration 3:   log likelihood =  5.1654153

Multivariate random-effects meta-analysis       Number of obs     =         10
Method: ML                                      Number of studies =          5
                                                Obs per study:
                                                              min =          2
                                                              avg =        2.0
                                                              max =          2
                                                Wald chi2(0)      =          .
Log likelihood = 5.1654153                      Prob > chi2       =          .



             
 
 Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
 
 
 
y1           
 
       _cons 
 
   .3572553   .0499616     7.15   0.000     .2593323    .4551782
 
 
 
y2           
 
       _cons 
 
  -.3538886   .0788344    -4.49   0.000    -.5084013    -.199376
 
 
 
Test of homogeneity: Q_M = chi2(8) = 128.23                Prob > Q_M = 0.0000



   Random-effects parameters 
 
   Estimate

Independent:                 
 
                      sd(y1) 
 
   .0845313
                      sd(y2) 
 
   .1596039

The random-effects parameter table now reports two terms, sd(y1) and sd(y2), because the correlation is assumed to be 0 under the independent covariance structure assumption.

Also see [META] meta mvregress.

estat heterogeneity

After you fit your multivariate meta-analysis model, you should quantify the amount of heterogeneity among the studies that was not accounted for by the model. You can use estat heterogeneity to do that.

. estat heterogeneity

Method: Cochran
Joint:
  I2 (%) = 93.76
      H2 = 16.03

Method: Jackson–White–Riley
y1:
  I2 (%) = 67.29
       R =  1.75

y2:
  I2 (%) = 94.40
       R =  4.23

Joint:
  I2 (%) = 87.49
       R =  2.83

This command produces heterogeneity statistics that extend the concept of univariate heterogeneity statistics, such as \( Q \) and \( I^2 \), to the multivariate setting.

For instance, Cochran's \( I^2 = \) 93.76% means that 93.76% of the heterogeneity is due to true heterogeneity between the studies as opposed to the sampling variability.

One potential shortcoming of the Cochran statistics is that they quantify the amount of heterogeneity jointly for all outcomes. The Jackson–White–Riley statistics provide ways to assess the contribution of each outcome to the total heterogeneity, in addition to their joint contribution.

For example, we can see that there is more heterogeneity among the effect sizes of outcome y2 (\( I^2 = \) 94.40%) than among the effect sizes of y1 (\( I^2 = \) 67.29%)

Also see [META] estat heterogeneity.

Postestimation: Predicting random effects

. predict double u*, reffects reses(se_u*)

. list trial u* se_u*



 
                   trial           u1           u2       se_u1       se_u2


  1. 
 Philstrom et al. (1983)    .05452276    .00844496   .05419716   .12705759

  2. 
    Lindhe et al. (1982)    -.0829853   -.22848358   .05668313   .13754507

  3. 
   Knowles et al. (1979)    .02838328    .21906828   .06359432   .13645361

  4. 
  Ramfjord et al. (1987)   -.07043129    .04982566   .06192739   .13633583

  5. 
    Becker et al. (1988)    .07051055   -.04885532   .04624526   .10346863

We listed the random-effects variables u1 and u2 with their corresponding standard-error variables se_u1 and se_u2. The random effects are study-specific deviations from the overall mean effect size. For example, for study 1 and outcome y1, the predicted mean improvement in probing depth is about 0.05 mm higher than the overall mean improvement in probing depth, \( \hat{\theta}_1 = \) 0.357.

For more detail about other postestimation tools, see [META] meta mvregress postestimation.

References

Antczak‐Bouckoms, A., K. Joshipura, E. Burdick, and J. F. Camilla Tulloch. 1993. Meta‐analysis of surgical versus non‐surgical methods of treatment for periodontal disease. Journal of Clinical Periodontology 20: 259–268.

Berkey, C. S., D. C. Hoaglin, A. Antczak‐Bouckoms, F. Mosteller, and G. A. Colditz. 1998. Meta‐analysis of multiple outcomes by regression with random effects. Statistics in Medicine 17: 2537–2550.