Stress testing (software)

Under a load test ( calque of Load Testing ) is defined as a software test, which generates an expected, even extreme load on the running system and the behavior observed desselbigen and examined. For this purpose, a simulation can be used. The aim here is to

The load test is therefore downstream of the functional test, i.e., the (sub) system must be in a functionally stable state in order to be at all tested for load management can.

  • 4.1 Commercial providers
  • 4.2 Free Software

Characteristics

The load may be that functions are carried out very quickly one after the other, or that parallel activities of virtual users ( multi-user, vUser ) are executed. As a rule, directly at the protocol level ( network protocol ) is working.

Basically, one can distinguish between (1) performance measurements and (2) load tests. Performance measurements repeat selected test cases or individual processes from the system test under a base load: because certain functions are tested for their performance properties, that is, all users run the same process, so the scalability of the individual function (s) will be tested. One speaks in the context of transactions. Load tests in the strict sense test entire process chains, and the Prozessmix on performance, that is, the links between the individual processes; so that they simulate actual operations from the actual commercial operation and provide a not to be underestimated step towards achieving the effective serviceability dar. Again, the scalability is crucial, but now for the whole Prozessmix. A while frequent errors effect are deadlocks during database access, which are difficult to be tested otherwise.

If the system is claimed aware of the defined load limit, one speaks of the stress test. The load ( number of virtual users ) should be gradually increased to above the defined load limit.

Thus, the following questions are examined:

  • How is the response time behavior varies as a function of the load?
  • The system displays undefined behavior (eg crash)?
  • If there is data inconsistency?
  • If the system after reduction of the overload back into the normal range?

In contrast, the low load test, which is intentionally operated at a low intensity is used, the study of the interaction behavior of virtual users and they generate message traffic on the system.

A load test over a longer period (eg, 48-72 hours) is called continuous load test; it serves primarily to detect memory leaks.

The most destructive form of a load test is the fail-over test. This involves the verification of the system behavior under load at failure of system components. Ideally, this emergency scenarios are reviewed, such as the timely switching of additional resources to prevent a total system failure.

Implementation

Generating test data

The test behavior is usually defined using a scripting language with many tools can you look at it - similar to a macro - "Record " via a web browser. This is usually realized through a proxy, which the requests, etc. translated into the scripting language. An important criterion here is the ease in test creation, but also the variability and the supported protocols (HTTP, HTTPS, etc.). Especially in areas where the quantity of data is more important than the precise content, also known as test data generators. These are programs that generate a large amount of data according to a predetermined pattern, the precise size of the data set can be configured in the rule. A common use case is here the velocity measurement of databases.

Test run

In the test run, the recorded behavior (which may be supplemented by random elements or output dependent variables) in arbitrarily high number (Virtual Users) run concurrently and therefore set the application under load by means of the generated script. An important criterion is the maximum load can be generated, and the hardware requirements that go with it.

Another useful feature is the ability to distribute the load generation on many computers that offer some tools. This allows the influence of the network capacity, as well as the hardware limitations of the load-generating computer, be minimized. Recently, some commercial tools integrate the possibility to integrate additional load generators in the cloud.

During the test run, the tool collects as much data. Basically, this is done directly on the side of the load-generating application ( response time, error codes, etc.). Some tools also offer additional options to certain Web-/Datenbank-Server (eg IIS, Apache, MSSQL ) or application server (Tomcat, etc.) to monitor to direct relationships (eg high response time vs. Database access ) to analyze. However, the data recovery can also be modularized take place ( eg utilities on the server of the application under test ). It is crucial that as many ways to collect various data are also provided.

Evaluation

For evaluation usually are certain indicators (eg response time vs. Time, timeouts vs. Number of users, etc. ) in log files and time-dependent graphs available. Good (mostly commercial ) tools also provide opportunities, for example through (auto- ) correlation functions to analyze the dependencies in the behavior (eg high response time vs. Calling a specific page, etc.).

Standardize

As a guide for planning a load and performance tests, the DIN 66273 is an appropriate starting point. This is included in the international standard ISO 14756 and standardized terms and measurement and evaluation method of the power of complex computer systems.

For the instrumentation of applications for performance or response time measurement was adopted within the Open Group Application Response Measurement (ARM) standard. This standard defines a programming interface for the C and Java programming languages.

Software Tools

In order to perform load tests, so-called load test tools offer. In general, a load server is installed, which generates the load on the system under test. The load test tools themselves can be made ​​of either, or you can use standard software, which offers a wealth of features and analysis options.

Commercial providers

  • SOASTA Inc.
  • Simple Works
  • Borland
  • C1 SetCon GmbH - TAPE
  • Compuware
  • Empirix
  • GFB Software Development Company ( Q-up )
  • Hewlett -Packard
  • IBM Rational
  • Intares GmbH
  • Ixia
  • Keynote Systems
  • Microsoft (as part of the Visual Studio suite )
  • Moniforce
  • Neotys SAS ( NeoLoad )
  • OPNET
  • Quotium
  • Proxy Sniffer
  • Scapa Technologies
  • Spirent
  • S_aturn Zott GmbH Co
  • WST
  • Xceptance Inc.

Free Software

  • BadBoy
  • Radio load on PyUnit based performance framework for web applications
  • The Grinder
  • Jamonapi - Java Performance Tuning and Scalability Measuring API
  • JMeter
  • OpenSTA
  • Selenium, test software for web applications
  • SoapUI (Web Services, exists in a free and a commercial version )
499923
de