Perceptron

• Five input neurons,
• Three hidden neurons and
• One output neuron, and
• Two bias neurons

The perceptron (English perceptron, after engl. Perception, "perception" ) is a simplified artificial neural network, which was first introduced by Frank Rosenblatt in 1958. It consists in the basic version ( simple perceptron ) from a single artificial neuron with adjustable weights and a threshold value. This term today, various combinations of the original model understood, it is between single-layer and multi-layer perceptron (English multi-layer perceptron, MLP) distinguished. The basic operation is to convert an input vector to an output vector, and thus it provides a simple associative memory dar.

History

• Input neurons
• Output neuron

1943 prompted the mathematician Warren McCulloch and Walter Pitts a the " neuron " as a logical threshold element with multiple inputs and a single output in the computer science. It was as a Boolean variable to true, the states and assume false and " fired " ( = true), if the sum of the input signals exceed a threshold value. This corresponded to the neurobiological analogy of an action potential, which emits a nerve cell at a critical change in their membrane potential. McCulloch and Pitts showed that by a suitable combination of several such neurons each simple propositional logic function (AND, OR, NOT) is writable.

In 1949 the psychologist Donald O. Hebb hypothesized learning based on the fact that the activating or inhibiting effect of a synapse 'll calculate the product of the pre-and postsynaptic activity. There is evidence that long- term potentiation and the so-called spike -timing dependent plasticity ( STDP ) the biological correlates of Hebb 's postulate are. Convincing evidence for this hypothesis is still pending.

In 1958 Frank Rosenblatt published the perceptron model that is the basis of artificial neural networks to this day.

Single-layer perceptron

The single-layer perceptron, it is only a single layer of artificial neurons which also represents the output vector. Each neuron is represented by a neuron function and receives the entire input vector as a parameter. The processing is very similar to the so-called Hebbian learning rule for natural neurons. However, the activation factor of this rule will be replaced by a difference between the setpoint and actual value. Since the Hebbian learning rule refers to the weighting of the individual input values ​​, so learning a perceptron is done by adjusting the weight for each neuron. Are the weights once learned, a perceptron is also able to classify input vectors that differ from the originally learned vector easy. Consists precisely in the desired classification ability of the perceptron, which it takes its name.

Perceptron learning rule

There are different versions of the learning rule to respond to the different definitions of the perceptron. For a perceptron with binary input and output values ​​here, the learning rule is specified.

The following considerations are the learning rule of the perceptron basis:

Mathematically, the situation is expressed as follows:

It is

A weight update step then runs as follows:

Rosenblatt was able to demonstrate in Konvergenztheorem that the specified learning process, all solutions can be taught, which may represent a perceptron.

XOR problem

Frank Rosenblatt showed that a simple perceptron with two input values ​​and one output neuron for the representation of simple logical operators AND, OR and NOT can be used. Marvin Minsky and Seymour Papert showed, however, in 1969 after that a single-layer perceptron the XOR operator can not resolve (the problem of linear separability ). This has led to a stop in the research of the artificial neural networks.

The in this context, partly highly polemical debate was ultimately a factional dispute between the representatives of Artificial Intelligence and the " Konnektionisten " for research funds. Frank Rosenblatt had indeed shown that composite logical operators such as XOR ( OR = but NOT AND ) can be described by using a multi-layer perceptron, but he died too early to defend themselves against the attacks of his AI colleagues.

The perceptron as a linear classifier

Beyond all the ( pseudo-) biological analogies a single-layer perceptron is ultimately nothing more than a linear classifier of the form ( linear discriminant function, multiple regression ). In the nomenclature of artificial neural networks are referred to as weights and up as input signals, where the latter can accept only values ​​of 0 or 1 (true or false). If the sum exceeds a threshold value, the assignment of the requested class is set to true or 1, otherwise to false or 0

Multi-layer perceptron

The limitation of the single-layer perceptron could be solved later with the multi-layered perceptron, in which there is beside the output layer and at least one more layer of hidden neurons ( engl. hidden layer ). All neurons of a layer are fully connected to the neurons of the next layer forward ( feed-forward networks). Other topologies have also proven:

• Full Connection: The neurons of a layer are connected to all the neurons of layers.
• Short - Cuts: Some neurons are not only connected to all neurons of the next layer, but also with other neurons of the next layer.

A multilayer perceptron can be trained, among others, with the error return (English backpropagation ). Here, the weights of the connections are changed so that the network can classify the desired pattern by a controlled training phase ( engl. supervised learning ).