Factor analysis

Factor analysis, factor analysis, often, is a multivariate statistical technique. It is designed from empirical observations of many different manifest variables ( observables, statistical variables) to a few underlying latent variables to include ( " Factors"). The discovery of these independent variables or characteristics is the meaning of the data-reducing (also dimensionally reducing ) process.

Differences, the factor analysis in exploratory and confirmatory factor analysis. The confirmatory factor analysis is a statistical inference method and can be considered as a special case of a structural equation model.

• 2.1 Geometrical Meaning
• 2.2 Linear factor model
• 2.3 Main theorem
• 2.4 Example

• 3.1 Factors extraction 3.1.1 main axis method
• 3.1.2 maximum likelihood estimate

Background

History

Factor analysis was developed by psychologist Charles Spearman for the evaluation of intelligence tests. In 1904 he showed that the test results to a large extent by a one-dimensional personality trait, the general factor (g- factor), could be explained. The generalization to an analysis of several factors is JC Maxwell Garnett attributed ( Steiger, 1979); It was popularized in the 1940s by Louis Leon Thurstone.

Maximum likelihood estimation methods have been proposed in the 1930s and 40s by Lawley and Victor Barnett; a stable algorithm was developed in the 1960s by Karl Gustav Jöreskog.

To date, however, an iterative variant of the principal component analysis is used for extracting factors, despite the poor properties of convergence. Inaccuracies to total equation of factor analysis and principal component analysis are widespread.

Applications

Factor analysis is a universal tool to the close of the visible manifestations of these phenomena underlying unobservable causes. Thus, for example, constructs such as "intelligence" or " ambition " is not measurable, but are considered to be the cause of many behaviors.

Occasionally, the factor analysis is also used for scientific problems. There are examples of the factor analysis processing of sound signals ( voice recognition ) at which acoustic main factors are extracted. This will voice overlays (airport announcement, conference call recordings) or superimposed music recordings made ​​comprehensible (Blind Source Separation, Independent Component Analysis (ICA ), see also links).

Factor analysis followed by Markus Wirtz and Christof Nightingale generally three objectives:

The exploratory factor analysis is totally dedicated to hidden structures of a sample or of dimension reduction. It is not suitable for Please check your existing theories. The appropriate method for this purpose represents the confirmatory factor analysis

Mathematical framework

Geometrical Meaning

Geometrically, the items included in the calculation are seen as vectors emanating all of the same origin. The length of these p vectors is determined by the commonality of the respective items and are the angles between the vectors by their correlation. The correlation r of two items, and the angle between their vectors hang together as follows

A correlation of 1 thus represents an angle of 0 °, a non-correlation, however, represents a right angle A model of p variables so spans a p -dimensional space. The aim of the factor analysis is to simplify this construct geometrically, so to find a q- dimensional subspace (). It should be " hidden " by the extraction process irrelevant factors. The solution of this process are the so-called " point cloud " in a q -dimensional coordinate system. The coordinates of these points represent the so-called factor loadings is that through the rotation process, the q factors extracted to be shot as close as possible in these point clouds.

Linear factor model

The factor analysis is always based on a linear model:

With

• : Vector of explanatory variables to,
• : Vector with constant values ​​,
• : Matrix of " factor loadings "
• : Vector of factor values ​​,
• : Random vector with mean 0

As a rule will also require that the components are not correlated with each other by. If this requirement is dropped, the model is invariant under orthogonal transformation and.

The empirical data material consists of realizations of the variable vector ( eg questionnaires with questions that have been processed by subjects). To simplify notation, it can be assumed that the raw data were centered in a first step of the evaluation, so that is true.

In a factor analysis to estimate them:

• The number of factors,
• The factor loadings from
• The variances of the residuals,
• The realizations of the factor vector.

The estimation is typically performed in three or more steps of:

• There are possible factors identified ( " extracted ");
• It is decided which number of factors to be considered;
• Factors may be rotated to facilitate their interpretation;
• The last factor vectors for the individual realizations of (eg personal values ​​for individual subjects) are estimated.

Law

From the model assumptions is followed by a short account of the fundamental theorem of factor analysis:

For this rate is simplified to

Here Var represents the variance, Cov the covariance and T for matrix transposition.

The term is that portion of the variance of observables, which is not explained by the factor model. The stated percentage, ie the sum of the squared factor loadings, ie commonality of the variables.

Example

In a garbage sorting system were installed to separate the waste, a magnet with a vertical direction of action and a fan with horizontal direction of action. The geometric coordinates of the waste pieces in the low traps may be part of the data collected. One finds direction correlations in pieces without metal and great sensitivity to wind as well as pieces with metal content and low sensitivity to wind.

Factor analysis can be found then, first, that there are two orthogonal factors that influence the direction of movement.

The application of the method of investigation may then be

• Initially the number of factors to estimate (see below): It is certainly not interesting for every single piece to document the trajectory and adopt its own factor for each piece, but to extract from the correlations of the data essential common factors: most likely form two factors from the data material out

• To determine the strength and orientation of these conditions ( still with no theory about the kinds of influences) or

• From knowledge of the piece properties ( metallic, compact vs. non-metallic, susceptible to wind ) to describe the factors and content for the continuous properties " metal content " and " wind resistance " to describe the " charges " on the factors ( their correlations with the magnetic force and the fan speed ).

It is in this example, the difference between orthogonal and oblique factor analysis clearly: especially in the social sciences is assumed, as a rule of non-orthogonal factors: the social science Analog to fan and magnet in the example are not necessarily at an angle to each other by 90 degrees arranged be and act accordingly.

In an exploratory situation in which one still has no hypotheses about the reasons for the occurrence of correlated impact points, you will be satisfied with the locating and marking of two factors, and try to narrow down on what direction these correlations are due. In a confirmatory situation we will investigate whether the found correlations with two factors (such as might be assumed from a theoretical point ) are actually explain, or whether you have to answer a third factor ( or actually just one factor acts ).

Exploratory factor analysis

The exploratory factor analysis is performed in four steps

Factors extraction

The first step of the factor analysis, the identification of possible factors, the estimation of the factor loadings and residual variances. For such an estimate, you need a quality criterion. This essential theoretical basis is not clearly spelled out in much of the literature.

The "weight" of a factor is determined from the number of measured variables correlate with it, how much do they " load on this factor ." This effect is quantified by the sum of charge squares. (This is true in the orthogonal case with the eigenvalues ​​of the matrix correspond charge ). Here you can sort the factors according to the amount of the charge sum of squares ( LQS ).

If one finds well- separable two sets of factors, one with high and another with low FQ FQ, you will equate the number of factors of the model with the number of LQS high factors. The separability of these groups, you can view on a line plot on the LQS; there is a noticeable kink, this can serve as a separation criterion ( screening test).

Another criterion is that the LQS a common factor should be greater than the variance of a single measurement variables ( otherwise it would be bad as a "common " factor to understand ). This means then i.d.R. LQS > = 1 ( Kaiser criterion ).

Principal axis method

With the main axis of the first method commonalities are estimated either as the regression coefficient of determination of the observed measurement variables to all other measured variables or as the maximum of the magnitudes of the correlations between the measured variables considered with all the other measurement variables. Thereafter, an iterative procedure is carried out:

In the principal axis method so the communalities and variances of the residuals are first estimated and then performed the eigenvalue decomposition. In the principal component analysis, the eigenvalue decomposition is not carried out, and then the commonalities and variance of the residuals can be estimated. For the interpretation of the means that in the principal components analysis of the variance of a measurement variable can be completely explained by the components, while the principal axis method, a proportion of the variance of a measurement variables exist that can not be explained by the factors.

A drawback of the major axis of the method is that in the course of the iteration process, the variance of the residuals can be negative or greater than the variance of the measured variables. The process is then terminated without result.

Maximum likelihood estimate

The parameter estimation is on a secure basis, considering the Γ, and the (not mitnotierten in the previous sections ) μ is determined so as to maximize the likelihood of the observed realizations of x.

However, one must make assumptions about x, the probability distribution of the manifest variables, ie assume a normal distribution is usually in this estimation procedure.

Determining the number of factors

In the extraction result depending on the option and method very many factors. Few of them explain enough variance to justify their continued use can. The choice of factors is primarily the extraction of meaningful, well- interpretable results and is therefore limited objectified. Evidence can provide the following criteria:

• Kaiser criterion
• (Also called elbow criterion) screening test
• Parallel analysis ( a modification of the screening tests)

Basically, several criteria should be used. In particular, in case of doubt, it is advisable, by attributed several factors to check numbers and in terms of charges and interpretability.

Shows the analysis of the underlying theory before a certain number of factors, this can also be used in the factor analysis. Also, the part of the examiner more or less arbitrarily be determined what proportion of the total variance to be explained, the requisite number of factors derived from it from then. However, even with a theory or variance escort determining the factors figure is based on the above criteria to be checked for plausibility.

Factor rotation

The rotation is to make the content better interpretable factors. At your disposal are various methods, including:

• Orthogonal, i.e., the rotated factors are again uncorrelated varimax
• Quartimax
• Equamax

• Oblimin
• Promax

These methods approach the rotation solution iteratively and usually require between 10 and 40 iterative calculations. The basis for the calculation of a correlation matrix.

Factors versus principal component analysis

The factor analysis and principal component analysis have a number of common characteristics:

• Both methods are used for dimension reduction.
• Both methods are linear models between the components / factors and variables.
• Both processes can be applied to both covariance and a correlation matrix.
• Both methods often give similar results ( if applied no rotation in the factor analysis ).

However, there are also a number of differences:

• The principal component analysis begins with the fact that she is looking for a low-dimensional linear subspace that best describes the data. Since the lower space is linear, it can be described by a linear model. It is therefore a descriptive- exploratory process. Factor analysis defines a linear model based and tries to approximate the observed covariance or correlation matrix. It is therefore a model-based method.
• In the principal component analysis, there is a clear rank order of vectors, represented by the descending eigenvalues ​​of the covariance or correlation matrix. In multivariate analysis, the dimension of the factor space is first defined and all vectors are of equal importance.
• In the principal component analysis, a p -dimensional random vector x is represented by a linear combination of the random vectors that are selected so that the first term as large a proportion of the variance of x explained, the second term as much of the residual variance, and so on. If one breaks this sum to q elements, is obtained as a representation of x

• To model only the variances, and not the covariance of x. The total variance, the optimality criterion of principal component analysis, can be written as the accumulated distance between the observations and the mean of the observations. The exact arrangement of the observations in the high-dimensional space in which the linear portion is described by the covariance or correlation matrix, however, does not matter.