»  Home »  Resources & support »  FAQs »  How do I meta set my data when the effect sizes are risk ratios (RRs), odds ratios (ORs), incidence-rate ratios (IRRs), or hazard ratios (HRs) with their confidence intervals?

How do I meta set my data when the effect sizes are risk ratios (RRs), odds ratios (ORs), incidence-rate ratios (IRRs), or hazard ratios (HRs) with their confidence intervals?

Title   How do I meta set my data when the effect sizes are risk ratios (RRs), odds ratios (ORs), incidence-rate ratios (IRRs), or hazard ratios (HRs) with their confidence intervals?
Author Dan (Mia) Lv, StataCorp

The answer is similar, regardless of whether the effect sizes are risk ratios (RRs), odds ratios (ORs), incidence-rate ratios (IRRs), or hazard ratios (HRs).

Let's assume that we want to perform meta-analysis using the following dataset, where the effect sizes are risk ratios (RRs). The confidence intervals for the risk ratios are also included in this dataset. Let’s have a look at the data.

. webuse bcgrratio2, clear
(BCG data with computed risk ratios)

. list rr rr_ll rr_ul

rr rr_ll rr_ul
1. .411 .134 1.257
2. .205 .086 .486
3. .26 .073 .919
4. .237 .179 .312
5. .804 .516 1.254
6. .456 .387 .536
7. .198 .078 .499
8. 1.012 .895 1.145
9. .625 .393 .996
10. .254 .149 .431
11. .712 .573 .886
12. 1.562 .374 6.528
13. .983 .582 1.659

In order to perform meta-analysis, we need to use the meta set command to declare the meta-analysis data using the effect size variable and confidence interval variables.

Because meta set is designed to work with normally distributed effect sizes with symmetric confidence intervals, we cannot specify our rr, rr_ll, and rr_ul variables directly in meta set. As we can see here, the confidence intervals of RRs are not symmetric around RRs, because RRs are not normally distributed. The same happens with ORs, IRRs, and HRs. In order to get normalized effect sizes and symmetric confidence intervals, we need to apply the natural logarithm transformation to rr as well as the limits of the confidence intervals, rr_ll and rr_ul.

. generate double log_rr = log(rr)

. generate double log_rr_ll = log(rr_ll)

. generate double log_rr_ul = log(rr_ul)

. list log_rr log_rr_ll log_rr_ul

log_rr log_rr_ll log_rr_ul
1. -.88916206 -2.0099155 .22872793
2. -1.5847453 -2.453408 -.72154666
3. -1.3470736 -2.6172958 -.08446916
4. -1.4396951 -1.7203695 -1.1647521
5. -.21815601 -.66164851 .22633844
6. -.78526247 -.94933059 -.62362112
7. -1.6194882 -2.5510465 -.69514918
8. .01192857 -.11093156 .13540464
9. -.47000363 -.93394567 -.00400802
10. -1.370421 -1.903809 -.84164719
11. -.33967737 -.55686956 -.12103833
12. .44596705 -.98349948 1.8761006
13. -.01714616 -.54128483 .50621501

In the above code, we specify the storage type double when we generate the transformed variables because we want them to have a high precision. double has a higher precision than the default float type (please see [D] Data types for more detailed information). Next we specify meta set with the transformed data. However, we get the following error message:

. meta set log_rr log_rr_ll log_rr_ul
confidence intervals not symmetric
  CIs defined by variables log_rr_ll and log_rr_ul must be
  symmetric and based on a normal distribution. If you are working with
  effect sizes in the original metric, such as odds ratios or hazard
  ratios, with meta set, you should specify the effect sizes and
  CIs in a normalizing metric, such as the log metric.

  The default tolerance to determine the CI asymmetry is 1e-6. Effect
  sizes and their CIs are often reported with limited precision that,
  after the normalizing transformation, may lead to asymmetric CIs. In
  that case, the default of 1e-6 may be too stringent. You may loosen
  the tolerance by specifying option civartolerance().
r(459);

This error message indicates that meta set determined that the confidence intervals are not symmetric. Why are the transformed confidence intervals still not symmetric? The original variables rr, rr_ll, and rr_ul are usually stored with only a few decimal digits (three digits in this case), and therefore the transformed confidence intervals are slightly asymmetric around transformed effect sizes. You should not see this error message if your data have adequate precision.

By default, the symmetry check is performed by meta set with a tolerance level of 1e-6, which means the largest relative difference between log_rr_ul-log_rr and log_rr-log_rr_ll cannot be greater than 1e-6. (The relative difference between \(x\) and \(y\) is defined as |\(x\)–\(y\)|/(|\(y\)|+1).) Let’s compute the relative difference for our transformed dataset.

. generate double rel_diff = reldif((log_rr_ul-log_rr), (log_rr-log_rr_ll))

. summarize rel_diff

Variable Obs Mean Std. dev. Min Max
rel_diff 13 .0019658 .0013932 .0002746 .0044752

We can see the maximum relative difference is 0.0044752, which is greater than 1e-6. That is why the symmetry check failed. We also notice the values of relative difference are all reasonably small, which confirms that the natural logarithm transformation is appropriate here (to make RRs normal). If the relative difference has any large values, that means the transformed confidence intervals are not close to symmetric. In this case, you need to check how the effect sizes and their confidence intervals are originally computed and decide if a different type of transformation is needed.

In the current situation, where the transformed confidence intervals are roughly symmetric, we can relax the criterion for the symmetry check by specifying the option civartolerance() with meta set. The default tolerance level for the symmetry check is 1e-6. Now we can change it to 1e-2 to avoid this error. The bigger the tolerance level we specify, the less strict the performed check will be. If you are interested, please see Why am I getting the error message “confidence intervals not symmetric” after I run meta set? for more detailed information about this error message and the option civartolerance().

. meta set log_rr log_rr_ll log_rr_ul, civartolerance(1e-2)

This time, the symmetry check is passed and we have declared the meta-analysis data using the transformed variables. We are ready for some analysis.

If we run meta summarize or meta forestplot, we will see that, by default, log RRs are displayed as the effect size instead of RRs because this is the data metric specified with meta set.

. meta summarize

  Effect-size label: Effect size
        Effect size: log_rr
          Std. err.: _meta_se

Meta-analysis summary                     Number of studies =     13
Random-effects model                      Heterogeneity:
Method: REML                                          tau2 =  0.3121
                                                    I2 (%) =   92.20
                                                        H2 =   12.82

Study Effect size [95% conf. interval] % weight
Study 1 -0.889 -2.008 0.230 5.05
Study 2 -1.585 -2.451 -0.719 6.36
Study 3 -1.347 -2.613 -0.081 4.42
Study 4 -1.440 -1.718 -1.162 9.71
Study 5 -0.218 -0.662 0.226 8.87
Study 6 -0.785 -0.948 -0.622 10.11
Study 7 -0.785 -0.948 -0.622 10.11
Study 8 0.012 -0.111 0.135 10.20
Study 9 -0.470 -0.935 -0.005 8.75
Study 10 -1.370 -1.902 -0.839 8.36
Study 11 -0.340 -0.558 -0.122 9.94
Study 12 0.446 -0.984 1.876 3.82
Study 13 -0.017 -0.541 0.507 8.41
theta -0.714 -1.066 -0.362
Test of theta = 0: z = -3.97 Prob > |z| = 0.0001 Test of homogeneity: Q = chi2(12) = 151.72 Prob > Q = 0.0000

However, you might prefer to display the effect sizes in the original metric (RRs). In that case, you can specify the option eform(Risk ratios) with meta summarize or meta forestplot.

. meta summarize, eform(Risk ratios)

  Effect-size label: Effect size
        Effect size: log_rr
          Std. err.: _meta_se

Meta-analysis summary                     Number of studies =     13
Random-effects model                      Heterogeneity:
Method: REML                                          tau2 =  0.3121
                                                    I2 (%) =   92.20
                                                        H2 =   12.82

Study Risk ratios [95% conf. interval] % weight
Study 1 0.411 0.134 1.259 5.05
Study 2 0.205 0.086 0.487 6.36
Study 3 0.260 0.073 0.923 4.42
Study 4 0.237 0.180 0.313 9.71
Study 5 0.804 0.516 1.253 8.87
Study 6 0.456 0.387 0.537 10.11
Study 7 0.198 0.078 0.501 6.01
Study 8 1.012 0.895 1.145 10.20
Study 9 0.625 0.393 0.995 8.75
Study 10 0.254 0.149 0.432 8.36
Study 11 0.712 0.573 0.885 9.94
Study 12 1.562 0.374 6.526 3.82
Study 13 0.983 0.582 1.660 8.41
exp(theta) 0.490 0.345 0.696
Test of theta = 0: z = -3.97 Prob > |z| = 0.0001 Test of homogeneity: Q = chi2(12) = 151.72 Prob > Q = 0.0000
. meta forestplot, eform(Risk ratios)

Graph 1

The option eform() displays the exponentiated effect sizes and the exponentiated confidence intervals using the label we specify, therefore showing our data in the original (RRs) metric. Please note that when eform() is specified, the x axis is automatically shown on an exponentiated scale that makes confidence intervals look symmetric, which is the standard in the literature.

Stata

Shop

Support

Company


The Stata Blog: Not Elsewhere Classified Find us on Facebook Follow us on Twitter LinkedIn YouTube Instagram
© Copyright 1996–2023 StataCorp LLC   •   Terms of use   •   Privacy   •   Contact us