Estimator

An estimating function is used in mathematical statistics to the basis of available empirical data from a sample to determine an estimated value and thereby obtain information about unknown parameters of a population. Estimators are the basis for the calculation of point estimates and confidence intervals and are used as test statistics in hypothesis testing. They are specific sampling functions and can estimation method, such as the least squares method, the maximum likelihood method or the method of moments, can be determined.

• 2.1 Formal Definition of the estimator

• 3.1 estimators and the estimated value for the average value
• 3.2 estimators and the estimated value of the variance
• 3.3 estimators and estimate of the value of units

• 5.1 unbiasedness
• 5.2 consistency
• 5.3 Minimum variance efficiency
• 5.4 Mean square error

Basic concepts: sampling variables and functions

In general, the experimenter is in the situation that he based on a finite number of observations ( a sample ) wants to make statements about the underlying distribution or its parameters in the population.

Only in rare cases can the population fully charge ( total or full survey), so that it then delivers exactly the desired information. An example of a full survey, the unemployment statistics of the official statistics.

In most cases, however, can not be fully charged, the population, for example because it is too large. If one is interested about the average size of the 18 -year-olds in the EU should be measured every 18 -year-olds, which is practically impossible. Instead, only one sample, a random selection of elements, is levied ( partial survey ).

Variable from the survey

At this point is where the statistical modeling. The variable from the survey, a random variable that describes its distribution, the probability that a particular feature expression occurs from the population in the -th draw. Each observation value is the realization of a variable from the survey.

Sampling function

The definition of the sampling variables allows the definition of sampling functions analogous eg to characteristic values ​​from the descriptive statistics:

Since each sample due to the randomness would be different are these sampling functions random variables whose distribution of

• The nature of the extraction of the sample from the population and
• The distribution of the characteristic in the population

Depends.

Sampling distribution

Under sampling distribution refers to the distribution of a sample function over all possible samples from the population. The sampling function is usually an estimator of an unknown parameter of the population or a test statistic for a hypothesis about an unknown parameter of the population. Therefore, one simply speaks instead of sampling distribution also on the distribution of an estimator or test statistic function. The distribution of the sampling function is used to obtain statements of unknown parameters in the population due to a sample.

The sampling distribution is a frequentistisches concept that Bayesian counterpart is the a posteriori distribution.

Calculation of the sampling distribution

The sampling distribution for a sample function with a certain sample size from a finite universe can be always calculated (see the following examples ) - but in general, people are more interested in general formulas, for example, an indeterminate sample size. Important tools are the following statements:

• Reproductivity of the normal distribution: If the sampling variables mutually independent and normally distributed (), then is also normally distributed ().
• Central Limit Theorem: If the sampling variables independent of each other and exist for them, the expected values ​​and is great for approximate a normal distribution ().

Bootstrap sampling distributions

If a sufficiently large sample is representative of the population, the sampling distribution for any Stichprobenfunkion can be nonparametrically estimated using the bootstrap method, without the distribution must be known. However, in general must be mathematically shown that the bootstrap sampling distributions converges with increasing number of bootstrap samples from the true sampling distribution.

Examples

Example 1

Consider an urn with seven balls with the markings 10, 11, 11, 12, 12, 12 and 16 If you draw two balls with replacement, the following table shows all possible samples from the population:

Each of the possible samples occurs with the probability of. If we now calculate the sample mean of the two balls, we obtain

Summing up the results of proportion to the probability of occurrence of the sample together, we obtain the sampling distribution of:

If you change the type of contraction, from a drawing with replacement in a drawing without replacement, this results in a different distribution for. In the above tables then falls off the main diagonal, so that there is only possible sampling. Therefore, then the following distribution for:

Example 2

In an urn are five red and four blue balls. There are three balls drawn without replacement from this urn. If we define the sampling function: number of red balls drawn among the three, is the hypergeometric distribution with the number of red balls in the urn, as the total number of balls in the urn, and as the number of attempts. Here all the information about the distribution of can be obtained because both the stochastic model ( drawing from an urn ) and the associated parameters (number of red and blue balls ) are known.

Example 3

A wholesale food market gets a delivery of 2,000 glasses with plum compote. Problematic are remaining seeds in the fruit. The customer tolerates a proportion of glasses with cores of 5 %. He wants to make this delivery that the quota is not exceeded. However, a complete survey of the population of 2,000 glasses is not feasible, since 2000 to control the glasses is too complicated and also destroyed the opening of a glass of the goods.

However, one might a small number of glasses to choose at random, so take a sample, and count the number of objectionable glasses. Exceeds this number a certain limit, the critical value of the test statistic, it is believed that even in the delivery are too many objectionable glasses.

A possible sampling function is where a random variable indicates that only the values ​​of one ( glass contains plums with core) or zero (glass contains no plums with core) accepts.

If the random variables are Bernoulli distributed, then is normally distributed approximate due to the central limit theorem.

Estimators

Estimators are special sampling functions to determine parameters or distributions of the population. Are influenced by, among other estimators

• The nature of the drawing of the sample (eg, dragging with or without replacement ) and
• The type of estimation method (eg least-squares method, maximum likelihood method or the method of moments ).

One would ultimately try to specify some intervals based only on the knowledge of the underlying model and the observed sample, the most likely contain the true parameter. Alternatively, one might also test for a given probability of error, if a particular assumption about the parameters ( for example, that contain too many glasses cores) can be confirmed. Estimators form in this sense, the basis of any reasoned decision on the characteristics of the population, the best possible choice of such functions is the result of mathematical investigation.

If you encounter this basis a decision, such as the supply is declining, there is the possibility that the decision is wrong. The following sources of error:

However, in practice there is usually no alternative to statistical methods of this kind the aforementioned problems we disputes in different ways:

Formal Definition of the estimator

Basis of each estimator are the observations of a statistical characteristic. Model theory, this feature is idealized: It is assumed that it is in the observations in truth realizations of random variables whose "true" distribution and "true" distribution parameters are unknown.

To obtain information on the actual properties of the feature, rises to a sample of items. Using these sample elements is estimated then the searched parameters and the desired distribution (see kernel density estimate).

Thus, in order to estimate, for example, a parameter of an unknown distribution, one has formally to do it with a random sample from the periphery, there are thus realizations () the observed random variables. The random variables are then combined by means of a method of estimation in an appropriate estimator. Formally, it is assumed here that a measurable function.

To simplify the calculation of the estimation function is often assumed that the random variables are independent and identically distributed, thus the same distribution and have the same distribution parameters.

Selected estimators

In statistical practice, the population is often sought by the following parameters:

• The mean value and
• The variance of a metric trait and
• The unit value of a dichotomous population.

Estimators and the estimated value for the average value

The mean value is usually estimated by the arithmetic mean of the sample:

The distribution is symmetrical, and the median of the sample can be used as an estimate of the expected value:

Wherein the lower floor function called. The median is the value that a random variable, the " in the middle" is after sorting the data. So there are numerically just as many values ​​above as below the median.

Which estimation function in the case of symmetrical distribution is preferable depends on the considered distribution family.

Estimators and the estimated value of the variance

For the population variance is mostly used as estimator of the corrected sample variance:

Typical other prefactors are and. All of these estimators are indeed asymptotically equivalent, but depending on the type of sample used differently (see also sample variance ).

Estimators and estimate of the value of units

One considers here the urn model with two types of balls. It is the unit value of the balls of first grade in the population can be estimated. As estimator we used the proportion of balls of first grade in the sample,

With: Number of balls first grade in the sample and a binary random variable: ball of the first kind considered in the -th draw () or not drawn ().

The distribution of a binomial distribution in the model with replacement and a hypergeometric distribution in the model without replacement.

The distribution estimators

The distribution of the estimators of course depends on the distribution of the trait in the population.

If the data are normally distributed with mean and variance, has the estimator as a linear transformation of the distribution of Xi

The variance estimator includes a sum of squares with respect to centered normal random variable. Therefore, the expression

Centrally -distributed with n-1 degrees of freedom.

If the distribution of the feature is unknown, in the presence of the condition of the central limit theorem, the distribution of the estimator can be approximated with the specified normal distribution or one of its derived distributions.

Properties of estimators

Unbiasedness

An unbiased estimate to be the true parameters in the average equal to:

However, deviates systematically from, the estimator is distorted (English biased ). The distortion is calculated to be doing

For only asymptotically unbiased estimate, however, only needs to apply:

Consistency

An estimator is called consistent if for each:

With. This is called stochastic convergence.

The chart at right illustrates the process: For each the filled areas have become smaller with increasing sample size.

In simple words, a consistent estimator approaches with growing more and more the true parameters ( estimates the true parameter always accurate).

Consistent estimators must therefore be at least asymptotically unbiased (see above).

This property is fundamental to the entire inductive statistics; it guarantees that increasing the sample size more accurate estimates, confidence intervals smaller and smaller areas of adoption allows in hypothesis testing!

Minimum variance efficiency

The estimator is said to have the smallest possible variance. The estimator of all the unbiased estimators, which has the smallest variance is referred to here as an efficient, the best or most efficient estimation function:

Under certain conditions can be specified for the Cramér -Rao inequality also a lower limit. That is, for an estimation function can be shown that there can be no more efficient estimators; most still just as efficient estimators.

Mean square error

The accuracy of an estimator or an estimator is often expressed by its mean square error ( mean squared error english ). A ( not necessarily an unbiased ) estimator should always have the smallest possible mean square error, the calculation can be determined from the true parameters as the expected value of the squared deviation of the estimator:

As can be seen, the mean square error of the unbiased estimator is not the sum of its variance and the square of the bias (distortion ); for unbiased estimators, however, variance and MSE are equal.