 »  Home »  Resources & support »  FAQs »  Running multiple Markov chains in parallel

## How can I run multiple Markov chains in parallel?

 Title Running multiple Markov chains in parallel Authors Nikolay Balov and Yulia Marchenko, StataCorp

Stata 16 has a new option, nchains(), that produces multiple chains with the bayes: prefix and the bayesmh command. The chains are simulated sequentially. But you may want to simulate the chains simultaneously to save time. You can do this by using the unofficial bayesparallel: prefix.

You can install bayesparallel: by typing

. net install bayesparallel, from("http://www.stata.com/users/nbalov")


After installation, type

. help bayesparallel


Below, we demonstrate a quick example of how to use this command. Let's fit a simple Bayesian linear regression of mpg on weight using our auto dataset.

First, we will simulate multiple chains sequentially using the standard syntax. To demonstrate, we will generate only two chains:

. webuse auto
(1978 Automobile Data)

. bayes, nchains(2) rseed(16): regress mpg weight
Chain 1
Burn-in ...
Simulation ...

Chain 2
Burn-in ...
Simulation ...

Model summary

Likelihood:
mpg ~ regress(xb_mpg,{sigma2})

Priors:
{mpg:weight _cons} ~ normal(0,10000)                                     (1)
{sigma2} ~ igamma(.01,.01)

(1) Parameters are elements of the linear form xb_mpg.

Bayesian linear regression                    Number of chains    =          2
Random-walk Metropolis-Hastings sampling      Per MCMC chain:
Iterations      =     12,500
Burn-in         =      2,500
Sample size     =     10,000
Number of obs       =         74
Avg acceptance rate =      .3448
Avg efficiency: min =      .1256
avg =      .1469
max =      .1864
Avg log marginal-likelihood = -218.62073      Max Gelman-Rubin Rc =      1.003

Equal-tailed
Mean   Std. Dev.     MCSE     Median  [95% Cred. Interval]

mpg
weight   -.0060058   .0005249    .00001  -.0060001  -.0070166  -.0049596
_cons    39.43578   1.637784   .032675   39.42489   36.08263   42.60404

sigma2    12.16568   2.067553   .033858   11.94594   8.770868   16.81322

Note: Default priors are used for model parameters.
Note: Default initial values are used for multiple chains.


We now use bayesparallel: to simulate these same chains simultaneously:

. bayesparallel, nproc(2): bayes, nchains(2) rseed(16): regress mpg weight

Simulating multiple chains ...

Done.



We simply prefixed our initial bayes: specification with the bayesparallel: prefix, also providing the number of processors to use simultaneously in the nproc() option.

We can now use any of the Bayesian features that are available after bayes, nchains():. The bayesparallel prefix does not produce any output, but we can display the results by typing

. bayes

Model summary

Likelihood:
mpg ~ regress(xb_mpg,{sigma2})

Priors:
{mpg:weight _cons} ~ normal(0,10000)                                     (1)
{sigma2} ~ igamma(.01,.01)

(1) Parameters are elements of the linear form xb_mpg.

Bayesian linear regression                    Number of chains    =          2
Random-walk Metropolis-Hastings sampling      Per MCMC chain:
Iterations      =     12,250
Burn-in         =      2,500
Sample size     =     10,000
Number of obs       =         74
Avg acceptance rate =      .3448
Avg efficiency: min =      .1256
avg =      .1469
max =      .1864
Avg log marginal-likelihood = -218.62073      Max Gelman-Rubin Rc =      1.003

Equal-tailed
Mean   Std. Dev.     MCSE     Median  [95% Cred. Interval]

mpg
weight   -.0060058   .0005249    .00001  -.0060001  -.0070166  -.0049596
_cons    39.43578   1.637784   .032675   39.42489   36.08263   42.60404

sigma2    12.16568   2.067553   .033858   11.94594   8.770868   16.81322

Note: Default priors are used for model parameters.


We obtain the same parameter estimates but faster.

We can now use any standard Bayesian postestimation command, just as we do after the bayes prefix (or bayesmh). For instance, we can use bayesgraph diagnostics to check MCMC convergence visually:

. bayesgraph diagnostics {mpg:weight}


Or we can use bayestest interval to perform interval hypothesis testing:

. bayestest interval {mpg:weight}, lower(-.005)

Interval tests     Number of chains =         2
MCMC sample size =    20,000

prob1 : {mpg:weight} > -.005

Mean    Std. Dev.      MCSE

prob1        .028     0.16510   .0028899



We used bayes, nchains(): in our demonstration, but you can just as easily use bayesmh, nchains() with the bayesparallel prefix.

Also see New in Bayesian analysis and, in particular, Multiple chains and Convergence diagnostics using multiple chains in [BAYES] bayesmh. Read more about Bayesian analysis in the Stata Bayesian Analysis Reference Manual.