Home  /  Resources & support  /  FAQs  /  Running multiple Markov chains in parallel

## How can I run multiple Markov chains in parallel?

 Title Running multiple Markov chains in parallel Authors Nikolay Balov and Yulia Marchenko, StataCorp

The nchains() option produces multiple chains with the bayes: prefix and the bayesmh command. The chains are simulated sequentially. But you may want to simulate the chains simultaneously to save time. You can do this by using the unofficial bayesparallel: prefix.

You can install bayesparallel: by typing

. net install bayesparallel, from("http://www.stata.com/users/nbalov")

After installation, type

. help bayesparallel

Below, we demonstrate a quick example of how to use this command. Let's fit a simple Bayesian linear regression of mpg on weight using our auto dataset.

First, we will simulate multiple chains sequentially using the standard syntax. To demonstrate, we will generate only two chains:

. webuse auto
(1978 automobile data)

. bayes, nchains(2) rseed(16): regress mpg weight

Chain 1
Burn-in ...
Simulation ...

Chain 2
Burn-in ...
Simulation ...

Model summary

Likelihood:
mpg ~ regress(xb_mpg,{sigma2})

Priors:
{mpg:weight _cons} ~ normal(0,10000)                                     (1)
{sigma2} ~ igamma(.01,.01)

(1) Parameters are elements of the linear form xb_mpg.

Bayesian linear regression                    Number of chains    =          2
Random-walk Metropolisâ€“Hastings sampling      Per MCMC chain:
Iterations      =     12,500
Burn-in         =      2,500
Sample size     =     10,000
Number of obs       =         74
Avg acceptance rate =      .3448
Avg efficiency: min =      .1256
avg =      .1469
max =      .1864
Avg log marginal-likelihood = -218.62073      Max Gelmanâ€“Rubin Rc =      1.003
 Equal-tailed Mean Std. dev. MCSE Median [95% cred. interval] mpg weight -.0060058 .0005249 .00001 -.0060001 -.0070166 -.0049596 _cons 39.43578 1.637784 .032675 39.42489 36.08263 42.60404 sigma2 12.16568 2.067553 .033858 11.94594 8.770868 16.81322
Note: Default priors are used for model parameters. Note: Default initial values are used for multiple chains.

We now use bayesparallel: to simulate these same chains simultaneously:

. bayesparallel, nproc(2): bayes, nchains(2) rseed(16): regress mpg weight

Simulating multiple chains ...

Done.

We simply prefixed our initial bayes: specification with the bayesparallel: prefix, also providing the number of processors to use simultaneously in the nproc() option.

We can now use any of the Bayesian features that are available after bayes, nchains():. The bayesparallel prefix does not produce any output, but we can display the results by typing

. bayes

Model summary

Likelihood:
mpg ~ regress(xb_mpg,{sigma2})

Priors:
{mpg:weight _cons} ~ normal(0,10000)                                     (1)
{sigma2} ~ igamma(.01,.01)

(1) Parameters are elements of the linear form xb_mpg.

Bayesian linear regression                    Number of chains    =          2
Random-walk Metropolis-Hastings sampling      Per MCMC chain:
Iterations      =     12,250
Burn-in         =      2,500
Sample size     =     10,000
Number of obs       =         74
Avg acceptance rate =      .3448
Avg efficiency: min =      .1256
avg =      .1469
max =      .1864
Avg log marginal-likelihood = -218.62073      Max Gelman-Rubin Rc =      1.003

 Equal-tailed Mean Std. dev. MCSE Median [95% cred. interval] mpg weight -.0060058 .0005249 .00001 -.0060001 -.0070166 -.0049596 _cons 39.43578 1.637784 .032675 39.42489 36.08263 42.60404 sigma2 12.16568 2.067553 .033858 11.94594 8.770868 16.81322
Note: Default priors are used for model parameters.

We obtain the same parameter estimates but faster.

We can now use any standard Bayesian postestimation command, just as we do after the bayes prefix (or bayesmh). For instance, we can use bayesgraph diagnostics to check MCMC convergence visually:

. bayesgraph diagnostics {mpg:weight}

Or we can use bayestest interval to perform interval hypothesis testing:

. bayestest interval {mpg:weight}, lower(-.005)

Interval tests     Number of chains =         2
MCMC sample size =    20,000

prob1 : {mpg:weight} > -.005

 Mean Std. dev. MCSE prob1 .028 0.16510 .0028899

We used bayes, nchains(): in our demonstration, but you can just as easily use bayesmh, nchains() with the bayesparallel prefix.

See Bayesian analysis and, in particular, Multiple chains and Convergence diagnostics using multiple chains in [BAYES] bayesmh. Read more about Bayesian analysis in the Stata Bayesian Analysis Reference Manual.