Independent component analysis

Independence analysis or independent component analysis (ICA) is a technique of multivariate statistics. It is used for calculating independent components in a mixture of statistically independent random variables. It is closely related to the blind source separation problem ( BSS).

Problem

It is assumed that the vector is statistically independent random variables. Thus ICA can be applied up to one of the random variables may be Gaussian distributed. The random variables are multiplied with a mixing matrix. For simplicity, it is assumed that this mixing matrix is square. The result is mixed random variables in the vector, which has the same dimension as.

The goal of ICA is to reconstruct the independent random variables in the vector as faithfully as possible. For this is only the result of mixing available and the knowledge that the random variables were originally stochastically independent. It is a suitable matrix sought so that

Since neither the mixing matrix nor the independent random variables are known, these can be reconstructed only with swabs. The variance and thus the energy of the independent random variables can not be determined because the independent random variables and the corresponding column vector of the mixing matrix with an arbitrary constant can be weighted so that the scalings cancel each other:

In addition, the order of the column vectors of the mixing matrix can not be reconstructed.

Troubleshooting

In general, it is assumed that the mixed random variables are zero-mean. This is not the case, this can be achieved by subtraction of the mean.

Pre- Whitening

The pre- whitening is a linear transformation, which is used for pre-processing. To a principal component analysis (PCA) is performed. The results are the eigenvalues ​​and eigenvectors of the covariance matrix of the mixed random variables. The eigenvectors are the lines of the rotation matrix that is multiplied with the vector. The eigenvalues ​​correspond to the variance of each principal component. The reciprocal values ​​of their square roots is used in the formation of the diagonal matrix, so that

, with

By multiplying the diagonal matrix, the variance of the principal components is normalized to 1.

Determination of the independent components

Through the pre- whitening the random variables are not statistically independent, but the problem was reduced to finding an orthogonal rotation matrix:

For those looking for recourse to the central limit theorem. This means that the mix of standardized, centered random numbers is similar with an increasing number of a normal distribution. Since the random variables fulfill this requirement, there must be a rotation matrix type, the possible non- normally distributed random numbers generated. The actual implementation of this search there are different approaches.

Kurtosis

The kurtosis is a measure of the deviation from a normal distribution. It is defined by

Since the random variables are normalized in their variance is equal to one. The kurtosis is zero when the distribution is Gaussian -like. If the kurtosis is negative, it resembles increasingly a uniform distribution. When it is positive, the distribution is more a Laplacian distribution. The kurtosis must therefore be maximized or minimized in order to move away from a normal distribution. This gradient can be used, for example, based on the learning rule of Oja.

Negentropy

Another approach is to maximize the negentropy.

The entropy and call that normal distribution is, the expected value and variance correspond to those of.

However, since it is difficult to determine, one usually uses approximate formulas for the negentropy.

An example of this is the calculation of the - often empirically determined - skewness and kurtosis of the distribution by virtue of:

Fast ICA

Fast ICA is a fixed point algorithm that solves the problem via a Newton method.

411383
de