Self-organizing map

As a self- organizing maps, or Kohonen Kohonenkarten ( Teuvo Kohonen after, English self-organizing map, SOM or self-organizing feature map, SOFM ) refers to a type of artificial neural networks. They are as unsupervised learning method a powerful tool of data mining. Their operating principle is based on the biological knowledge that many structures in the brain have a linear or planar topology. The signals of the input space, such as visual stimuli, but are multidimensional.

This raises the question of how these multi-dimensional impressions are processed by planar structures. Biological studies indicate that the input signals are mapped so that similar stimuli are close together. The phase space of the applied stimuli is thus mapped.

A signal is introduced to the card, only those areas of the card are energized, which are similar to the signal. The neuron layer acts as a topological feature map, if the situation of the most excited neurons is correlated in a lawful and continuous manner with important signal features.

Application, see self-organizing maps, for example, in computer graphics as a quantization algorithm for color reduction of raster graphics data and bioinformatics cluster analysis.

Lateral inhibition environment

A general principle of operation of the nervous system is that active local groups of neurons inhibit other groups in its environment, and thus their activity to suppress (see lateral inhibition ). The activity of a nerve cell is therefore determined from the superposition of the exciting input signal and the inhibitory input from all layer neurons. Since this lateral inhibition applies everywhere, there is a constant competition for supremacy. The course of the lateral inhibition is excitatory / inhibitory reinforcing and / debilitating for long distances for short distances. It can be shown that this effect is sufficient to cause a localization of the excitation response near the maximum external excitation.

Structure and learning

The structure of a self- organizing map: A n input layer neurons is fully connected with all neurons within the Kohonenkarte, hereinafter referred to as competitive layer. Each to be mapped input stimulus is passed through the connections to each neuron of this Competitive Layers v.

The connection weights between the neurons of the input layer and the neurons in the competitive layer each define a point in the input space of the applied stimuli. All neurons within the competitive layer are interconnected ( retardant) linked inhibitory.

It is customary, but not mandatory to use both for the learning vectors and for the Competitive Layer the Euclidean distance as a distance measure.

If a set of different training data is available, an epoch in the training is complete when all stimuli were applied exactly once in random order to the input layer. The training ends when the network has reached its stable final state.

Learning in a Self-Organizing Map can be formally described as an iterative process. In the initial state, the weight vectors of the neurons are randomly distributed in the network. In each learning step a stimulus is applied to the network. The Self-Organizing Map neural changed the weight vectors of the neurons according to the Hebbian rule, so there is a topographic image over time.

Training a SOM in the example

The following table shows a network, the neurons are arranged in a lattice, and are distributed randomly in space at the beginning. It is trained with input stimuli from the square, which are equally distributed.

Formal description of the training

Given a finite set M of training stimuli mi, which are specified by an n- dimensional vector xi:

Furthermore, a lot of neurons ìN is given to each of which a weight vector wi in X, and a position on a ki Kohonen map is assigned, which is assumed herein as two-dimensional. The maps dimension can be chosen arbitrarily - dimensional, with maps dimensions are less than or equal three used for the visualization of high-dimensional contexts. The positions on the map are discrete, square lattice points correspond ( alternative neighborhood topologies such as hexagonal topologies are also possible ), and each grid point to be occupied by exactly one neuron:

In the learning phase, an element is randomly selected MJT equally distributed from the set of stimuli to the presentation time t. This stimulus sets on the card firmly winner neuron nst, which is referred to as the excitation center. This is exactly the neuron, the weight vector WST to the stimulus vector has the smallest distance in the space X xjt where a metric dX of the input space is given ('. . )

After nst was determined, all the neurons are determined nit that can adjust their weight vectors in addition to the excitation center. It is to the neurons whose distance dA (ks, ki ) on the map is not greater than a time-dependent threshold value is referred to as distance range DELTA.t, where a metric DA (. ,. ) Was given to the card. These neurons are in a subset of N t ⊂ Nt summarized:

In the following the adaptation step, a learning step is applied to all the neurons of N T applied that changes the weight vectors. The learning step can be interpreted as a shift of the weight vectors xjt in the direction of the stimulus vector, in the figure below, the displacement of the weight vector of the winner neuron is shown.

It is according to the model of Ritter et al. (1991 ) exceed the following adaptation rule used:

εt with the time-dependent parameters and equations hsit, which are defined as:

1) The time-dependent learning rate εt:

εstart with the start learning rate and εend as the learning rate at the end of the process, ie after tmax stimulus presentations.

2) The time-dependent distance weighting function hsit:

With delta.t as the neighborhood or adaptation radius around the winning neuron in the map:

With the adaptation radius δstart the beginning of the process, and δend as the adaptation radius at the end of the process.

Thus, a topology - preserving mapping is created, ie that neighboring points are mapped to neighboring points in the input space X on the map, two factors must be considered:

In the illustrated learning tmax presentations are performed, after which the SOM can be transferred to the application phase are presented in the stimuli that did not occur in the learning set. Such a stimulus is associated with the winner neuron, the weight vector has the least distance of the stimulus vector, so that the stimulus via the detour of a neuron weight vector, and a position can be assigned to the neural map. In this way the new stimulus is automatically classified and displayed.

Variants of the SOM

There a number of variations and extensions to the original model of Kohonen have been developed, including:

• Context - SOM (K- SOM)
• Temporary SOM (T- SOM)
• Motorized SOM (M- SOM)
• Neurons gas ( NG- SOM)
• Growing cell structures (GCS - SOM)
• Growing grid structure (GG- SOM)
• Growing Hierarchical SOM (GH - SOM)
• Growing neural gas ( GNG - SOM)
• Parametric SOM (P- SOM)
• Hyperbolic SOM (H- SOM)
• Interpolating SOM (I- SOM)
• Local -weighted regression SOM ( LWR SOM)
• Selective Attention SOM (SA- SOM)
• Learned expectations in GNG SOMs (LE- GNG - SOM)
• Fuzzy - SOM (F- SOM)
• Adaptive - Subspace SOM (AS- SOM)
• Generative Topographic Map (GTM )