Benchmark (computing)

Benchmarking are standardized measurement and evaluation process that will help you measure the performance of computer systems or system classes and compare them with each other according to certain criteria can. The benchmark tests for the hardware performance in computer systems are known. Benchmarks but are also used to compare the performance of programming languages, compilers and interpreters, therefore, and their run-time systems on the same hardware. Then one speaks of a Software benchmark. A benchmark in general is always a program that a (often derived from mathematics, consuming ) solves problem. Examples are the solution of linear systems of equations ( LINPACK ) or the calculation of Mersenne primes.

History

  • 70s: measurement in MIPS and FLOPS
  • Early 80s: measurement in Dhrystones and Whetstones
  • Since 1989 Application Benchmark tests of the Standard Performance Evaluation Corporation (SPEC). It is a series of applications (eg, neural networks, finite elements, video encoding, etc.) in integer -related ( SpecInt ) and floating point -related ( SPECfp ) programs are divided. Accordingly, the final outcome of the tests is always two numbers, the performance relative to a reference machine (Benchmark) specify. In Spec2000 this is, for example, a Sun UltraSparcIIi/256MB.

Hardware benchmark tests traditionally include only CPU and memory access performance. Graphics card, mass storage, etc. are not tested.

The SPEC update was in August 2006:. SPECcpu 2006 A new version is in the pipeline. For this purpose a " SPEC CPU benchmark Search Program" was launched in 2008.

The most well known benchmarks are:

  • LINPACK
  • Whetstone
  • Dhrystone
  • Standard Performance Evaluation Corporation (SPEC)

Lately, the focus of benchmarks has expanded. In particular, the inclusion of energy consumption as a metric has been established and is used in various SPEC (eg spECpower ) and TPC benchmarks application.

In addition, is dealt with how you can capture current trends in IT, using benchmarks. For example, there are currently several projects that deal with cloud benchmarking or with the benchmarking of peer-to -peer systems.

Example of Hardware Benchmarks

Table of some Dhrystone values ​​as an example of hardware benchmarks:

Software benchmarks

Software benchmarks are used to compare the performance of different programming systems with respect to the execution speed. When creating a software benchmarks the same algorithm implemented in different programming languages ​​and compared the run times of programs. It is necessary to bring together experts in various programming languages ​​to implement the algorithm optimally for the respective programming language. A classic example of such benchmarks is the Ackermann function.

Application Benchmarks

Application Benchmarks are used to evaluate the performance of computer systems in the interaction of all hardware and software components. These realistic application scenarios are created. The term of this automated reproduced application scenarios allows a uniform comparison of different computer systems. Also important quantities such as energy consumption and battery life in mobile devices can be hereby determine comparable.

Examples of application benchmarks are

  • SysMark, MobileMark, EEcoMark
  • 3DMark, PCMark, power cord

Manipulations

Is the benchmark software used known, which provides a scope for manipulation. In practice, device drivers, compilers or sets of commands can be optimized such that common benchmarks run particularly fast.

Interpreter (computing) Whetstone (benchmark) Futuremark Instruction set Stress testing (software)
114627
de