Time-of-flight camera

TOF cameras are 3D camera systems, with the transit time method (English: time of flight, TOF) measurement of distances. For this, the scene is illuminated by a light pulse, and the camera meters for each pixel, the time it takes the light to the object and back again. The time required is directly proportional to distance. The camera provides thus for each pixel the distance of the object imaged thereon. The principle is the same laser scanning, with the advantage that an entire scene is captured at one time and need not be sampled.

TOF cameras are in contrast to other methods is a relatively new development. The systems can be used at a range of a few tens of centimeters to about 40 meters. The distance resolution amounts to about 1 cm, the lateral resolutions reach about 200 × 200 pixels. The cameras provide, this is the great advantage of this technique, up to 160 images per second.

Other terms such as PMD camera are still in use for this new technology.

  • Examples of TOF cameras

" PMD 204 × 204 pixels with CamCube and SBI "

"Swiss Ranger 4000 MESA Imaging with 176 × 144 pixels "

" TOF camera from the European ARTTS project with 176 × 144 pixels and power supply via USB"

  • 4.1 Automotive applications
  • 4.2 Human Machine Interfaces / Gaming
  • 4.3 Measurement / Industrial Image Processing
  • 4.4 Robotics
  • 4.5 Medicine
  • 6.1 General
  • 6.2 videos

Construction

A TOF camera comprises at least the following components:

Operation

The simplest form of TOF cameras works with light pulses: The lighting is switched on for a short moment, the light pulse illuminates the scene and is reflected on the objects. The camera lens collects this light and is the scene on the gauge. Depending on the distance learns the incident on the individual pixels of light delay. The duration of the light from the camera to a 2.5 m distant object and back to the camera is: Since the light speed of light ( in the air approximately 299,710 kilometers per second) spreads, these times are very small:

Based on the evaluation principle (see below), the pulse length of the light determines the maximum distance range, the camera can cover. With a pulse length of 50 ns up to distances

Be measured. These short times that the illumination is a critical part of the system. Only with selected LEDs or with complex to be controlled lasers, it is possible to generate such short pulses.

The individual pixels are made of a photosensitive element (e.g., photodiode ), it converts the light into a current. At the photodiode one or several quick closures or switches are mounted, which can be only for a very specific period of time the electrical signal through. A downstream storage element adds to the signal.

In the sample drawing, the pixel operates with two switches ( G1 and G2) and the memory elements ( S1 and S2). The switches are controlled with a pulse signal having the same length as the light pulse, the control signal G2 is shifted by a pulse length. Then the reflected light strikes on the delayed pixels, as only a part of the signal passes into the storage element S1, the other portion is collected in S2. Depending on the distance thus changes the ratio of S1 and S2, as shown in the second graph. Because within 50 ns very little light can be collected, not just a pulse but several thousand with a repetition rate of (tR -1) is emitted and collected, which increases the signal strength.

After recording, the pixels are read out, and the subsequent stage measures the signals S1 and S2. Because the length of the light pulse is known, the distance can be calculated as follows:

In the example, S1 = 0.66 and S2 = 0.33. The distance is thus

Backlit results on the two storage elements an additional signal component. To eliminate this, the recording can be performed again with inactive lighting and these values ​​are subtracted from the signals with lighting. If the objects are further away than the distance range, as result of the above formula wrong distance values ​​. Which can also provided with a second measurement, in which the switching signals are further shifted by T0, can be suppressed. Other systems operate in place of the pulses with a sinusoidal modulation, in which the requirements are less than the slope of the illumination.

Pros and Cons

Benefits

Simple design: Unlike laser scanners, the camera has no moving parts. Since the lighting and the lens are close together, the space requirement compared to stereo and triangulation systems is smaller and shadowing are excluded.

Efficient data processing: With the distance information of the TOF cameras, it is easy to extract only the regions of interest from an image: There is a set distance threshold and only the pixels that provide further distances are observed.

Speed: The TOF cameras form the complete scene from a recording. The frame rate can reach up to 160 frames per second, thus enabling real-time applications.

Pattern independence: Unlike stereo systems, the difficulties with repeating patterns or uniform surfaces can get work TOF cameras to all diffusely reflecting materials.

Disadvantages

Backlight: Although most of the background light is suppressed by the optical filter, the pixel has to deal yet with a very large momentum, the resulting charge must but can also be stored or dissipated. For comparison, the currently economically possible illuminance levels are in the range of one watt. The sun makes in the filtered wavelength range is still 50 watts per square meter. If the scene illuminated a square, so the sun is 50 times stronger than the desired signal.

The various manufacturers have developed different strategies for their sensors in order to suppress this background signal to a large extent can (see for example SBI of the PMD sensor, which operates up to 150 klx ).

Crosstalk: If several systems in operation, it may be that interfere with the various cameras against each other and thus the distance value is falsified. There are several ways to work around this:

  • Time-division multiplex: a higher-level controller starts the measurement of each camera one by one, so that there is always only one lighting in operation.
  • Different frequencies: Working the cameras with slightly different modulation frequencies, as is the light of a camera in the other demodulated only as background portion and does not distort the measurement.

Multiple reflection Because in contrast to the laser scanning systems, a whole scene and not just a single point illuminated, it is possible that multiple reflected light comes back from an object to the sensor. The distance measured may be in this case larger than in reality.

Areas of application

Automotive applications

Time-of- flight cameras can be used as a driver assistance and safety sensors in the automotive sector. These include applications such as active pedestrian protection, emergency brake assist as well as interior applications such as checking for correct riding position.

Human- Machine - Interfaces / Gaming

The real-time capability of TOF cameras the movements of a person can be tracked. Therefore, new interaction possibilities open up with the devices. In addition to the control of devices such as televisions the application of TOF cameras to game consoles is an interesting topic.

Measurement / Industrial Image Processing

The third dimension, measurement tasks such as the determination of liquid levels in silos easily accomplish. In the industrial machine vision benefit, for example, robots that items must record from a conveyor belt from the additional altitude. Door controls can differ using the height easily between animals and humans.

Robotics

Another field of application are mobile robots: With the environmental image in real-time mobile robot can survey their surroundings quickly, avoid obstacles or follow as a person. Since the calculation of distance is simple, only little computing power for creating the environment map is consumed.

Medicine

TOF cameras can also be used as an additional imaging modality in medical technology. Examples of these applications are shown in the following:

  • Breathing Detection: With the help of TOF cameras, it is possible to calculate a multidimensional respiratory signal. It is thus possible to measure self-contained breathing signals without physical markers at several points (eg, thorax, abdomen ). This is important in the irradiation of suspected tumors in the upper body or for the reduction of artefacts, such as computer tomography (CT ) or positron emission tomography (PET).
  • Patient Alignment: TOF cameras also allow the precise positioning of the patient in the clinical setting. For this purpose, the 3-D point cloud of a reference TOF image are registered with another point cloud. Registration is the result of this translation and rotation, which must be applied to the patient that it is exactly the same as during the recording of the reference image. Applications for this are also to be found in the radiation therapy. Here, the irradiation is planned on an initial CT measurement. This plan shall be valid for all following radiation sessions. This can be done by a registry TOF images.
Image sensor Stereo camera Lux Digital Object Identifier
501171
de