Efficient estimator

Efficiency is a term used in statistics, with an aspect of the quality of an estimator for an unknown parameter can be measured. It is one next to consistency, sufficiency and ( asymptotic ) unbiasedness of the four commonly used criteria for the quality of estimators.

• 2.1 Expectation spreader case
• 2.2 Non expectation spreader case

Idea

The efficiency refers to the variance of an estimator function. The smaller the variance of an estimator function, the closer an estimate is (on average), calculated from a sample, are to the true parameters. A distinction between relative and absolute efficiency.

Once you two are unbiased estimators for the same unknown parameter, ie, the estimator with the smaller variance ( relatively ) efficient or effective. To solve the estimation problem, one would prefer the more efficient estimator. The Cramer -Rao inequality states that there is a lower bound for the variance of the unbiased estimator for many estimation problems. If you have such an estimator is found, there is no other unbiased estimate which has a smaller variance. One can show, therefore, that for an estimation problem an estimator has the minimum variance, so the name of this estimator absolutely efficient.

Example

If variables are independent samples with and and you look for the unknown parameters, the two estimators (for 3 observation values):

Both estimators are expected faithful. For the variance is apparent, however

This applies, i.e., is more efficient than.

Mathematical definition

Expectation spreader case

Formal is an unbiased estimator for the unknown parameter in a family of probability densities and the corresponding density for Fisher information. Then, the efficiency is defined as follows:

.

If you want two unbiased estimators and compare with each other, it means one more efficient estimator, which has the higher value, and so the smaller variance.

One consequence of the Cramer -Rao inequality is that under regularity conditions is limited upward by 1 and therefore such estimators are called efficient, and therefore applies to. This is below the necessary for the Cramer -Rao inequality conditions on the stochastic model the best possible variance of an estimator.

Not expectation spreader case

If the estimator is unbiased, their efficiency can be as

Define. Obviously, the above definition gives as a special case.

Asymptotic efficiency

In general, it is sufficient that estimators are asymptotically efficient, ie when they converge in distribution to a normally distributed random variable whose variance is the inverse of the Fisher information. Formally, to the convergence statement

Can be proved, the Fisher information of the density and is referred. For asymptotically efficient estimator is obviously true

Typical examples of asymptotically efficient estimators are those which are obtained with the aid of the maximum likelihood method.