Optical flow

As optical flow (English: optical flow ) is in image processing and optical metrology refers to a vector field, which indicates the direction of movement and speed for each pixel of an image sequence. The optical flow can be used as the image projected onto the image plane velocity vectors of visible objects to be understood.

The optical flow is an important representation of motion information; it is used for example in the context of visual navigation of robots and vehicles (visual navigation ) and optical computer mice, and provides a basis for recognition of the three-dimensional structure of scenes for the estimation of movements in space and for the recognition of individual moving objects ( segmentation ). It is believed that the optical flow in animals also play an important role in the visual information processing plays, for example, also in the view navigation.

Methodology and applications

The optical flow is a quantity which can not be calculated directly and unambiguously from the image data, but you have to appreciate with more or less elaborate process. All these estimation methods may be based only on the measurable brightness patterns in the image. All other parameters, which are used in current methods for estimating the optical flow (in particular the derivatives of the gray value function after the two local variables and to time) may in digital (more precisely, discrete ) can only be estimated from the sampled image data as well.

An accurate and unambiguous determination of the direction and length of an optical - flow vector is never possible for a single pixel of principle; all practicable calculation method must necessarily combine information from a domain G to the pixel of each other. Only that portion of the optical flow vector, which runs parallel to the brightness gradient, can be uniquely determined from a local measurement. This fundamental problem is called the aperture problem.

Whether the vector of interest can be determined exactly, so depends on whether the gray value gradients are present within the considered region G in different directions. It is also necessary to have a model idea of ​​what may have basic profile of the optical flow within the considered area; In the simplest case it is assumed that the optical flow within small areas can be regarded as constant. More complicated profiles of the flow field ( eg affine models ) are possible and are used in high-performance method. An interest operator returns those points whose flux vector can be determined particularly safe. In some approaches, the river is only at these selected points is calculated ( feature point tracking).

The optical flow is thus in principle a non- directly measurable quantity; Thus, the term also refers to the ( intended ) result of a calculation method. Often, however, are certain procedures that seek to compute the optical flow, denoted by the same term. Particularly in English, the term optical flow as opposed to methods of motion estimation is used, which originate from the area of ​​video compression. While classical optical flow methods are usually differentially (ie on the basis of derivatives and gradient of the gray level signal ) to work and usually result in a dense motion or vector field, using techniques of image compression, such as the MPEG standard, whole groups of pixels ( blocks) and deliver individual motion vectors for all blocks of pixels.

Important methods for calculating the optical flow are the already mentioned differential method, which usually work pixel by pixel, on the other hand, the block-wise working methods which are used in image coding and photogrammetry: photogrammetric block compensation of image blocks, block correlation (minimized sum of the absolute differences, normalized cross- correlation). A special form of block-wise motion estimation is the building on the Fourier transform phase correlation ( inverse of the normalized cross power spectral density ).

Formulas for the optical flow

The computation of optical flow using differential methods goes back to the developed in 1981 at MIT method of Berthold Horn and Brian Schunck.

It is assumed that the brightness of the corresponding points of the frames is constant in the image sequence. Then follows from the derivative

As a necessary condition the equation for determining the velocities:

(Compare continuity equation)

The solution of this equation is an ill-posed problem in the sense of Jacques Hadamard. Therefore smoothness is also required by the solution.

There are several methods to determine the optical flow including:

• Lucas- Kanade method
• Horn- Schunck method

Some well-known algorithms for the computation of the optical flow are implemented in the C library OpenCV.

Application in the bee and human

Bees and other animals use the optical flow to avoid obstacles and to estimate distances easy. To evaluate clearly the images of both compound eyes and then fly in the direction of the eye with lower optical flow. This results in the simplest way to a trajectory that has the most clearance and the fewest obstacles in front of him.

A similar process is the basis of our everyday experience in pedestrian and road safety: we take the movement of other road users from the corner of his eye true and consider them "unconscious " in their own locomotion. The other hand, shear a road user from this river from our sense of sight is immediately alerted and will sometimes even triggered a protective reflex, such as a jump or a retraction of the head.