Bayes estimator
A Bayesian estimator ( named after Thomas Bayes ) is in mathematical statistics an estimator that takes into account, in addition to the observed data, if present, prior knowledge of a parameters to be estimated. According to the procedure of Bayesian statistics knowledge this is modeled by a distribution of the parameter, of the a priori distribution. By Bayes' Theorem gives the conditional distribution of the parameter under the observation data, the a posteriori distribution. To obtain a clear estimate therefrom are capable extent the posterior distribution as the expected value, mode or median is used as a so-called Bayesian estimator. Since the a posteriori expected value of the principal and in practice most commonly used estimator is, some authors refer to this as the Bayes estimator. In general we define a Bayesian estimator as the value which minimizes the expected value of a loss function under the a-posteriori distribution. For a quadratic loss function, then the a posteriori expectation value is just used as an estimate.
Definition
It indicate the parameters to be estimated and the likelihood, that the distribution of observation as a function of. The a priori distribution of the parameter to be identified by. Then
The a posteriori distribution. It is further a function, called loss function, given the values of model the loss that you suffer if estimated from through. It means a value that the expected value of the
Minimizing the loss of the posterior distribution, a Bayesian estimator of. In the case of a discrete distribution of the integrals are to be understood about as summation over.
Special cases
Posterior expectation value
An important and frequently used loss function is the quadratic deviation
With this loss function is obtained as the Bayes estimator of the expected value of the posterior distribution, short of a posteriori expected value
This can be seen in the following way: Differentiating by, resulting
Zeroing this derivative and solving for yields above formula.
A posteriori median
Another important Bayesian estimator is the median of the a posteriori distribution. It results when using the linear loss function
The magnitude of the absolute error. Case of a continuous posterior distribution, the corresponding Bayes estimator obtained as a solution of the equation
So as the median of the distribution with density.
A posteriori mode
For discrete distributed parameter there is the zero-one loss function
At which assigns a constant loss all wrong estimates and not "punished" only an exact estimate. As expected value of this loss function gives the a posteriori probability of the event, ie. This is minimized at the locations at which is at its maximum, that is, the modal values of the a posteriori distribution.
With constantly has distributed the event for all probability zero. In this case can be given instead of a (small ), the loss function
Consider. In the limit then the a posteriori mode is also apparent as a Bayesian estimator.
In the case of a uniform distribution as the a priori distribution, there is the maximum likelihood estimator, which thus represents a special case of a Bayesian estimator.
Example
An urn containing red and black balls in an unknown composition, that is, the probability of drawing a red ball is unknown. In order to appreciate, balls are drawn successively with replacement: Only one draw delivers a red ball, so it will be observed. The number of red balls drawn with a binomial distribution and, therefore applies
Da on the parameters to be estimated is present no information, the DC distribution is used as a priori distribution, that is to. As an a posteriori distribution therefore is the
This is the density of a beta distribution with parameters and. So, as a posterior expectation and as a posteriori mode. The posterior median must be determined numerically and yields about. General results from the red balls in contractions as a posteriori expected value and so the classical maximum likelihood estimator, as a posteriori mode. For small values of not a good approximation of the posterior median.
Practical calculation
One obstacle in the application of Bayesian estimators can be their numerical calculation. A classical approach is the use of so-called conjugated a-priori distributions in which a posterior distribution from a known distribution class is determined, the position parameters can then be easily looked up in a table. If, for example, in the above urn experiment any beta distribution as prior, then there is also a beta distribution as a posteriori distribution.
For general a priori distributions shows the above formula the a posteriori expectation that its calculation of two integrals must be determined over the parameter space. A classical method of approximation is the Laplace approximation in which the integrand to be written as an exponential function, and then the exponent be approximated by a quadratic Taylor approximation.
With the advent of more powerful computers, numerical methods have to calculate the integrals applicable (see Numerical integration). A problem is especially high-dimensional sets of parameters represent, ie the case that a lot of parameters to be estimated from the data. This often Monte Carlo methods are approximate methods are used.