Gesture recognition

Gesture recognition is the automatic recognition of human gestures executed by means of a computer. A branch of computer science concerned with the algorithms and mathematical methods for the detection of gestures and the use of gestures for human- computer interaction. Each posture and body movement may represent a gesture in principle. However, the most important is the identification of hand and head gestures. A variant of the gesture recognition is the recognition of so-called mouse gestures.

Definition

With reference to the human-computer interaction Kurtenbach and Hulteen define a gesture as follows: "A gesture is a motion of the body contains information did. Waving goodbye is a gesture. Pressing a key on a keyboard is not a gesture Because The motion of a finger on its way to hitting a key is neither observed- nor significant. All that matters is what Which key pressed ". In contrast, Harling and Edwards waive the requirement for movement and understand by a gesture static hand postures. It is possible to distinguish between systems in which the necessary for the detection sensor is located directly on the user's body, and those in which the user is observed by external sensors.

Device-based gesture recognition

Most based on worn or guided by hand to the body sensor systems use in data gloves integrated acceleration or position sensors. The disadvantage of systems based on data gloves is that the user must put the glove on to use the system.

Hand -guided systems, such as the controller, the Nintendo Wii and the Blue wall produced by the company BeeCon can also be used for gesture input. Both systems can be described by the user take in hand and have accelerometers to determine the movement of the respective device.

On newer devices such as smartphones and tablet computers is touch screens are used, which can be used by " swiping ". In particular, multi- touch screens offer the detection of multiple independent finger pressing simultaneously so that, for example, with two diagonally placed fingertips window can be drawn larger or smaller.

Camera-based gesture recognition

On systems with external sensors are mostly to camera-based systems. The cameras are used to create images of the user. It exist in both systems with a camera and with multiple cameras, the newer systems often work with 3D data, which work either on time-of -flight cameras, or so-called structured light cameras. Camera-based methods rely on techniques of 2D and 3D image analysis to detect the user's posture. Camera -based gesture recognition is used for example in games for video game consoles to be connected EyeToy. A new approach is the gesture control via stereoscopy. The advantage here is that this does not require infrared light and therefore works outdoors.

In the technical image analysis are basically two approaches can be distinguished: Either a database of relevant gestures is created, which have been prepared on the basis of a meridian of over 1,000 video analysis per gesture. Absorbed control gestures are then compared with the database and determined accordingly. This solution is applied, for example by Microsoft with the Xbox in conjunction with the 3D camera Kinect. The disadvantage of this analysis is that it abfordert much processing power with the database. Alternatively, the software works with a real skeleton detection, ie from the camera data body, hand and / or fingers are recognized and assigned by a simplified skeleton model the pre-defined gestures. This solution promises a much greater variety of gesture and precision, but is technically much more demanding.

The aim of the research and development in the coming years is to implement gesture recognition as part of embedded software that is cross-platform and camera independent and very little energy is required, hence, for example, in mobile phones, tablets and navigation systems can be used.

In 2012, a number of commercial vendors have announced that they want to come with devices for gesture recognition on the market, which should be significantly better than the currently available devices (especially the Kinect for the Xbox ). For example, Samsung has introduced at CES 2012 in Las Vegas, the Smart TV. Another company is LeapMotion, the promotional video for The Leap has been criticized in the community, as part manifestly set scenes were played. In Germany gesture control is especially an issue in the field of automotive industry, in which case particularly stable and mobile systems are needed, how they are produced, for example by gestigon who are also working on an embedded solution. Even in the areas of digital signage, media art, media art and performance 3D gesture recognition is often used. An easy way to use gesture recognition in these areas and, for example, to control other software is Kinetic Space. Other manufacturers include Omek, Soft Kinetic and Myestro Interactive.

Gesture types

It can be distinguished by two gesture types. In continuous gestures is a direct link between the observed through the computer, and a movement state in the computer. For example, are controlled by pointing at the screen a pointer. For discrete gestures are, however, for limited amounts of unique gestures with which each action is usually linked. An example of discrete gestures, sign language, in which each gesture is associated with a specific meaning.

Recognition

In the actual detection of gestures the information from the sensors in a flow algorithms that analyze the raw data and recognize gestures. Here algorithms are pattern recognition used. For removing noise in the input data and to reduce data frequently takes place in the first step, a Vorberarbeitung the sensor data. Then features from the input data are extracted. These features serve as input for classification. For this purpose, often hidden Markov models, artificial neural networks and other techniques, which have their origin mostly in the research on artificial intelligence, are used.