Gibbs-Sampling

Gibbs sampling, an algorithm to generate a series of samples of the joint probability distribution of two or more random variables. The aim is to approximate the unknown joint distribution. The algorithm is named due to the similarity of the sampling method with methods of statistical physics after physicist Josiah Willard Gibbs. It was developed by S. Geman and D. Geman ( see reference ). Gibbs sampling is a special case of the Metropolis -Hastings algorithm.

Gibbs sampling is particularly suitable when the joint distribution of a random vector, but it is known unknown the conditional distribution of each random variable. The basic principle is to select a variable in a repeating fashion, and to generate in accordance with a value of its conditional distribution function of the values ​​of the other variables. The values ​​of the other variables remain unchanged in this iteration. From the resulting sequence of sample vectors, a Markov chain can be derived. It can be shown that the stationary distribution of this Markov chain is just the desired joint distribution of the random vector.

A particularly favorable application arises in the context of Bayesian networks, especially in estimating the a posteriori distribution, since the usual representation of a Bayesnetzes is a set of conditional distributions. Software BUGS ( ) is an application of the Gibbs - sampling on Bayesian networks.