Graphics pipeline

The representation of 3-dimensional worlds on the computer is now widespread thanks to appropriate hardware and among other things, part of many computer games and office applications for visualization. The so-called rendering generates, from abstract data (eg, by dots (called vertices ) and triangles) (some realistic looking ) images.

A computer graphics pipeline, also rendering pipeline or simple graphics pipeline, is a model representation in computer graphics that describes what steps must a graphics system for rendering, so the representation of a 3D scene on a screen perform. Since these steps depend on both the software and hardware as well as the desired display properties, there is no generally valid graphics pipeline. For the control of graphics pipelines usually graphics APIs such as Direct3D or OpenGL are used, which abstract the underlying hardware and remove the many tasks the programmer.

The model of the graphics pipeline usually takes place during real-time rendering application. Often, here are most of the steps of the pipeline implemented in hardware, allowing for special optimizations. The term " pipe " is used in a similar sense as the pipeline in processor: Although the individual steps of the pipeline run in parallel, but are blocked until the slowest step was terminated.

Construction

A graphics pipeline can be divided into three major steps: application, geometry and rasterization.

Application

The application step is performed by the software, it can not therefore be divided into pipelined single running steps. However, it is possible to parallelize it on multi-core processors or multiple processor systems. In applying step changes are made to the scene as they are needed, for example, due to the user interaction through input devices or an animation. The new scene with all its primitives - usually triangles, lines and points - is then forwarded to the next step of the pipeline.

Examples of tasks that are typically taken from the application step are collision detection, animation, morphing and acceleration techniques using spatial subdivision schemes.

Geometry

The geometry step, which is responsible for the majority of operations with polygons and their vertices ( vertices ), can be divided into the following five tasks. It depends on the particular implementation, as these objects are organized as actual running parallel pipelined steps.

Definitions

A vertex (plural: vertices ) is a point in the world. These points are used to join the surfaces. In certain cases even directly point clouds are drawn, but this is still the exception.

A triangle (English: Triangle ) is the most common geometric primitives in computer graphics. It is defined by its three corners and a normal vector - the latter is used to indicate the front of the triangle, and is a vector which is perpendicular to the surface. Such a triangle can be provided with a color or texture.

The world coordinate system

The world coordinate system is the coordinate system in the virtual world is created. This should ensure that the following math is easy to use, fulfill some conditions: it must be a rectangular Cartesian coordinate system, in which all axes are scaled equally. As the unit of the coordinate system is defined, however, the developer is left. Whether that is the unit vector of the system is really intended to correspond to a meter or one Angstrom is depends on the application. Whether a right-handed or Left-handed one coordinate system to be used may be determined by the graphics library to use.

The specified objects in the scene (houses, trees, cars ) are given for the convenience of modeling often in their own object coordinate system (also model coordinate system or local coordinate system ). By assigning coordinates in the world coordinate system or a global coordinate system of the whole scene to these objects, the object coordinates by means of translation, rotation or scaling transforms. This is done by multiplication of the corresponding transformation matrices. In addition, can be formed of a plurality of differently transformed object copies, such as a forest from a tree; This technique is called instancing.

Now we can calculate the position of the vertices of the plane in world coordinates by multiplying each point in succession with these four matrices. Since the multiplication of a matrix is ​​quite complex with a vector, it usually goes another way and initially multiplies the four matrices together. The multiplication of two matrices is still more expensive, but must be run for the whole object only once. The multiplications and are equivalent. Thereafter, the resulting matrix may be applied to the points. In practice, the multiplication with the points will, however, still fails to apply, but first the camera matrices - see below - determined.

The order in which the matrices are applied is important, because the matrix multiplication is not commutative. This also applies to the three rotations, such as one can cite an example in mind: the point (1, 0, 0) is on the X-axis, when the first the X and then to about 90 ° the Y-axis is rotated, it ends up on the Z- axis ( having the rotation around the X-axis does not effect to a point which lies on the axis). However it is rotated first about the Y- and the X-axis is the resulting point on the Y -axis. The order itself is arbitrary, as long as you always make the same. The order of x, then y, then z (roll, pitch, heading ) is often most intuitive, because the rotation causes, among other things, that the compass direction with the direction of the "nose" matches.

There are also two conventions to define these matrices, depending on whether you want to work with column vectors or row vectors. Different graphics libraries have different preferences here. OpenGL for example, preferred column vectors, row vectors DirectX. The Decision follows, from which side the point vectors are multiplied to the transformation matrices. For column vectors, the multiplication from right, so, where vout and vin represent 4x1 column vectors. The Konkatennierung of the matrices is from right to left, so for example, if you want to first rotated and then moved. In row vectors, it behaves exactly the opposite. The multiplication is now performed from the left than with 1x4 vectors and the Konkatennierung is also when first rotated and then moved. The matrices presented above are valid for the second case, those for column vectors arise as a transpose it. The rule is that for the multiplication with vectors means that you can swap the order of multiplication by the transpose.

The interesting thing about this matrix concatenation is now that a new coordinate system is defined by any such transformation. This can move on to any size. For example, as the single model of the propeller of the plane, which is then placed by a translation to the aircraft nose. This translation must describe the propeller coordinate system, only the shift from the model coordinate system. For drawing of the entire aircraft so that the transformation matrix for the plane is first determined and then subsequently transformed points multiplies the translation to the propeller of the aircraft model to the matrix and then transforms the propeller points.

The calculated in this kind of matrix is also called the world matrix (English: World Transformation). It must be determined for each object in the world before being displayed. The application can here refer to changes influence, so for example, change the location of our aircraft according to the speed.

Camera transformation

In addition to objects, the scene also defines a virtual camera or a viewer, which indicates the position and direction of view from which the scene is to be rendered. To simplify the subsequent projection and clipping, the scene is transformed so that the camera is at the origin, looking along the Z- axis. The resulting coordinate system is called a camera coordinate system and the camera transformation transform ( English view transformation).

Projection

The projecting step transforms the view volume into a cube with the Eckpuntkoordinaten (-1, -1, -1) and (1, 1, 1); occasionally other target volume can be used. This step is called projection, although he transformed a volume to another volume because the resulting Z- coordinates are not stored in the image, but only when Z -buffering later rasterization step apply. In a perspective illustration of a central projection is used. To limit the number of objects displayed, two additional clipping scheme may be used; the view volume is here so a truncated pyramid ( frustum ). The parallel or orthogonal projection is used, for example, technical illustrations, because it has the advantage that all the parallels in the object space are parallel also in the image space and areas and volumes are the same size regardless of the distance from the viewer. Maps for example, use a orthogonal (so-called orthophoto ), forward pictures of the scenery, however, are not to be used for so - although technically, of course, be represented - they seem so distorted, so that we can do anything.

The reasons why the smallest and the largest distance must be specified here are, firstly, that is divided by this distance to reach the size scaling ( more distant objects in a perspective picture smaller than near objects) and, secondly, that so that the Z- values ​​are scaled to the range 0 to 1, whereby then the Z- buffer is filled. This often has a resolution of only 16 bits, so the near and far values ​​should be chosen wisely. Too great a difference between the near and far value results because of the low resolution of the Z-buffer in so-called Z-fighting. From the formula will also be appreciated that the Nahwert can not be 0, since this point is the focal point of the projection. In this point, there is no image.

For efficiency reasons, the camera and the projection matrix are usually combined in a transformation matrix, so that the camera coordinate system will be omitted. The resulting matrix is usually constant for a single image, while the world matrix for each object looks different. In practice, therefore, View and Projection precomputed, so that during the presentation only needs to be adjusted, the World matrix. However, other, more complex transformations, such as vertex blending are possible. Programmable geometry shader, which change the geometry can also be performed. In the real world rendering step matrix * camera projection matrix * matrix is ​​then calculated and then finally applied to each point. Thus, the points of all objects are transferred directly into the screen coordinate system (at least almost, the value ranges of the axes are for the visible range even -1 .. 1, see " Window- Viewport Transformation" ).

Lighting

Often a scene at different positions comprises placed light sources to make the illumination of the objects appear more realistic. In this case, a gain is computed for the texture for each vertex based on the light sources and the belonging to the corresponding triangle material properties. In subsequent screening step, the vertex values ​​of a triangle are interpolated over its surface. A general lighting ( ambient light) is applied to all surfaces. It is diffuse and thus directional brightness of the scene. The sun is a directional light source, which can be assumed to be infinitely far away. Illumination, which causes the sun on a surface is determined by taking the dot product of the directional vector of the sun and the normal vector of the surface. If the value is negative, the surface faces the sun.

Clipping

Only the primitives that are within the view volume, must actually be rasterized. Primitives that are completely outside of the view volume are discarded; this is called frustum culling. Other culling techniques such as back face culling, reducing the number of primitives to be included can be performed theoretically in any step of the graphics pipeline. Primitives that only partially located inside of the cube must be clipped against the dice. The advantage of the previous projection step is that the clipping always takes place against the same cube. Only the - possibly clipped - primitives that are within the view volume will be forwarded to the next step.

Window- Viewport Transformation

To print the image at an arbitrary target area ( viewport) of the screen, you have a further transformation, the Window- Viewport transformation can be applied. This is a displacement, followed by a scale. The resulting coordinates are device coordinates of the output device. The viewport contains 6 values: width and height of the window in pixels, the upper left corner of the window in window coordinates ( usually 0, 0) and the minimum and maximum values ​​for Z ( usually 0 and 1).

On modern hardware, most of the steps of the geometry calculation in the vertex shader can be performed. This is, in principle, freely programmable, but shall in general at least to transform the points and the lighting calculation. For the programming DirectX to use a custom vertex shader is inevitable from version 10, while older versions have not found a standard shader available.

Scanning

In the screening step, all primitives are rasterized, it is therefore of continuous space discrete fragments created. These can then be colored (if illuminated) are. Furthermore, it is necessary to determine at the respective overlapping polygons visible, so the closer to the viewer. For this so-called masking calculation usually a z-buffer is used. The color of a fragment is dependent on the illumination, texture, and other material properties of the primitive and is often visible interpolated from the triangle vertices. Where available, a fragment shader in the rasterization step is executed for each fragment of the object. If a fragment to be visible, it can now be mixed with already existing color values ​​in the image, if transparencies are simulated or multi-sampling is used. In this step, a pixel from one or more fragments.

So that the user does not see the gradual screening of the primitives, double buffering takes place. The screening takes place in a separate memory area. Once the image has been completely scanned, it is copied at once into the visible area of the image memory.

Inverse

All matrices are used regularly and thus invertible. Since again a regular matrix is formed by the multiplication of two normal matrices, the overall transformation matrix is invertible. The inverse is needed to re- calculate screen coordinates from world coordinates - for example, to close out the cursor position to the clicked object. But since the screen and the mouse have only two dimensions, the third is unknown. It is therefore projected a beam at the cursor position in the world, and then determines the intersection of this ray with the polygons in the world.

Shader

Classy graphics cards were based on the internal structure is still relatively close to the graphics pipeline. With increasing demands on the GPU restrictions were gradually removed to create more flexibility. Modern graphics cards use a programmable shader pipeline controlled, allowing to intervene directly in individual processing steps. To relieve the main processor, additional processing steps have been introduced in the pipeline, which has only running on the CPU.

The main shader units are pixel shader, vertex shader and geometry shader. To optimally exploit all units of the unified shader has been introduced. Thus there is only a single large pool of shader units. As necessary, the pool is divided into different groups of shaders. A strict separation between the shader types is therefore no longer useful. Meanwhile, it is also possible via a so-called compute shader to make any calculations run away from the representation of a graphic on the GPU. The advantage is that these reactions are highly parallelized, there are limitations here, however. These universal calculations are also called GPGPU.