Bayesian analysis is a statistical paradigm that answers research questions
about unknown parameters using probability statements.
For example, what is the probability that the average male height is between
70 and 80 inches or that the average female height is between 60 and 70
inches? What is the probability that people in a particular state vote
Republican or vote Democratic? What is the probability that a person accused of
a crime is guilty? What is the probability that treatment A is more cost
effective than treatment B for a specific health care provider? What is the
probability that a patient's blood pressure decreases if he or she is prescribed
drug A? What is the probability that the odds ratio is between 0.3 and 0.5?
What is the probability that three out of five quiz questions will be answered
correctly by students? What is the probability that children
with ADHD underperform relative to other children on a standardized test?
What is the
probability that there is a positive effect of schooling on wage? What is the
probability that excess returns on an asset are positive? And many more.
Such probabilistic statements are natural to Bayesian analysis because of the
underlying assumption that all parameters are random quantities. In Bayesian
analysis, a parameter is summarized by an entire distribution of values
instead of one fixed value as in classical frequentist analysis.
Estimating this distribution, a posterior distribution of a parameter of
interest, is at the heart of Bayesian analysis.
A posterior distribution comprises a prior distribution about a
parameter and a likelihood model providing information about the
parameter based on observed data. Depending on the chosen prior
distribution and likelihood model, the posterior distribution is either
available analytically or approximated by, for example, one of the
Markov chain Monte Carlo (MCMC) methods.
Bayesian inference uses the posterior distribution to form various summaries
for the model parameters, including point estimates such as posterior means,
medians, percentiles, and interval estimates known as credible intervals.
Moreover, all statistical tests about model parameters can be expressed as
probability statements based on the estimated posterior distribution.
Unique features of Bayesian analysis
include an ability to incorporate prior information in the analysis, an
intuitive interpretation of credible intervals as fixed ranges to which a
parameter is known to belong with a prespecified probability, and an ability
to assign an actual probability to any hypothesis of interest.