Bayesian Analysis

Bayesian analysis is a statistical procedure which endeavors to estimate parameters of an underlying distribution based on the observed distribution. Begin with a "prior distribution" which may be based on anything, including an assessment of the relative likelihoods of parameters or the results of non-Bayesian observations. In practice, it is common to assume a uniform distribution over the appropriate range of values for the prior distribution.

Given the prior distribution, collect data to obtain the observed distribution. Then calculate the likelihood of the observed distribution as a function of parameter values, multiply this likelihood function by the prior distribution, and normalize to obtain a unit probability over all possible values. This is called the posterior distribution. The mode of the distribution is then the parameter estimate, and "probability intervals" (the Bayesian analog of confidence intervals) can be calculated using the standard procedure. Bayesian analysis is somewhat controversial because the validity of the result depends on how valid the prior distribution is, and this cannot be assessed statistically.

REFERENCES:

Gelman, A.; Carlin, J.; Stern, H.; and Rubin, D. Bayesian Data Analysis. Boca Raton, FL: Chapman & Hall, 1995.

Graham, P. "A Plan for Spam." http://www.paulgraham.com/spam.html.

Hoel, P. G.; Port, S. C.; and Stone, C. J. Introduction to Statistical Theory. New York: Houghton Mifflin, pp. 36-42, 1971.

Iversen, G. R. Bayesian Statistical Inference. Thousand Oaks, CA: Sage Pub., 1984.

Press, W. H.; Flannery, B. P.; Teukolsky, S. A.; and Vetterling, W. T. Numerical Recipes in FORTRAN: The Art of Scientific Computing, 2nd ed. Cambridge, England: Cambridge University Press, pp. 799-806, 1992.

Sivia, D. S. Data Analysis: A Bayesian Tutorial. New York: Oxford University Press, 1996.