Dynamic testing

Dynamic software testing process, certain test methods to detect errors in the software with software testing. Error particular should be recognized that arise in dependence on dynamic run-time parameters such as varying input parameters, run-time environment or user interaction.

  • 3.1 Control flow oriented tests 3.1.1 statement testing ( test)
  • 3.1.2 branch coverage test ( edges overlap test) ( test)
  • 3.1.3 Simple condition testing ( test)
  • 3.1.4 Multiple condition coverage test ( test)
  • 3.1.5 Path coverage testing ( test)
  • 4.1 Back-to- Back Test
  • 4.2 mutations test
  • 4.3 Regression Test

Description

While with static method, the software to be tested is not running ( compile ), set dynamic process, the feasibility of the software requires ( runtime). Basic principle of the dynamic process is the execution of the software under test with systematically specified input data (test cases ). And the expected output data are presented to the input data for each test case. The output data generated by the test run are compared with the respective expected data. In case of deviations is an error.

The main task of the individual methods is the determination of appropriate test cases to test the software.

The dynamic method can be categorized as follows:

Function-oriented test methods

Function-oriented test methods are used for the determination of test cases to be checked with those of the extent to which the device under test ( also Ling test, test object or test object called ) met the required specifications (black box). It also speaks of that is tested "against the specifications ." Depending on the test Ling and test type are the specifications here of a different kind when unit test is tested, for example, against the module specification, the interface test against the interface specification and the acceptance test against the technical requirements as laid down as in a specification.

Equivalence class formation

The equivalence classes of the possible values ​​of the inputs (or expenditure ) are divided into classes, of which it can be assumed that errors that occur during the processing of a value from this class also occur with all other members of the class. On the other hand, a representative of the class is processed correctly, then it is assumed that the input of all other members of the class does not lead to errors. In this respect, the values ​​of a class as ( in this respect ) can be equivalently viewed each other.

Based on the equivalence classes of the test cases are formed. To test the valid equivalence classes, the test data from as many valid equivalence classes are formed. To test the invalid equivalence classes, a test date from an invalid equivalence class is combined with exclusively valid test data from the other equivalence classes, respectively.

Example

In the specification of an online banking system is required that only amounts of 0.01 to 500 € must be entered. It can be assumed then that a transfer fee of € 123,45 accepted and executed correctly, if a test has shown that a transfer of € 123.44 will be performed correctly. More generally it can be assumed that all amounts of 0,01 € to 500,00 € to be processed correctly if this is the case for any amount in this field. It is therefore sufficient to test any representatives from the area to get a possible error on the track.

In the same way you can for negative values ​​and values ​​greater than 500 € argue. For tests, it should therefore be sufficient to form three equivalence classes ( a valid and invalid two equivalence classes ):

  • Values ​​of 0.01 up to and including 500.00 € ( valid)
  • Values ​​less than or equal to zero ( invalid)
  • Values ​​greater than 500.00 € ( invalid)

Any payment in online banking system must be authorized by entering a TAN. Analogous to the first equivalence class four equivalence classes can be created here for the TAN:

  • Entering a correct TAN
  • Entering a wrong TAN
  • Input to short TAN

On the basis of the equivalence classes, the test cases are defined below:

  • Entering a valid value ( eg € 123.45 ) and confirmation with correct TAN ( run because everything is correct. )
  • Entering a valid value ( eg € 123.45 ) and confirmation with an incorrect TAN ( failed because wrong TAN. )
  • Entering an invalid value ( eg € 600.00 ) and confirmation with correct TAN ( failed because of invalid value. )
  • Entering a valid value ( eg € 123.45 ) and confirmation with too short a TAN ( failed because TAN is too short. )
  • Entering a valid value ( eg € 123.45 ) and confirmation with too long a TAN ( failed because TAN is too long. )

With the equivalence classes it is not possible to take into account dependencies between different input values. So it is not enough about when considering an input address to check for place names, street names and postcodes, respectively (eg from a database ), whether they belong to the class of valid values ​​. You also need to fit together.

Boundary value analysis

The limit analysis is a special case of the equivalence class analysis. It arose from the observation that errors most frequently occur at the " edges " of the equivalence classes. Therefore here is not arbitrary values ​​to be tested, but so-called boundary values ​​or limits. In the example this would be the values

  • 0,00 € ( invalid entry )
  • € 0.01 ( valid entry )
  • 500.00 € ( valid entry )
  • € 500.01 ( invalid entry )

Pairwise method

Pairwise the method is a method to reduce the number of combinations of a plurality of test input values ​​by not all possible combinations are tested. Instead, only each input value of a field is tested in pairs with each input value of the other input fields. This reduces the number of tests required radical, but also means of course that can not be detected under certain circumstances errors that occur only at very specific combinations of more than two fields.

State -based test methods

State -based test methods are based on finite state machines, which are now often portrayed as a UML diagram.

In the description of the state machine usually no faults are provided. These are additional pieces by every combination " { initial state; Event } " ( also not specified in the machine combinations ), the next state and triggered actions to be specified. These combinations can all be tested then. Can be used are state-based methods except in technical applications to tests of graphical user interfaces and classes that are defined by finite state machines.

The completeness of the thus obtained test cases is given if all of the following criteria are met:

  • If all state transitions through?
  • If all the events that cause state transitions are tested?
  • If all the events that must not cause state transitions, tested?

Cause-and- effect analysis

In this method, causes and effects are identified each part specification. One cause is a single input condition or an equivalence class of input conditions; an effect is an output condition or a system transformation.

The partial specification is converted into a Boolean graph linking causes and effects through logical links. Then the graph is extended to dependencies based on syntactic constraints or environmental conditions. The following types of dependencies between causes are differentiated between:

The resulting graph is converted to a decision table. Certain rules are applied to generate a linkage of n causes n 1 test cases (instead of, as it would be at a complete decision table the case). This corresponds to each test case, a column of the decision table.

Structure-oriented test methods

Structure-oriented test methods to determine test cases based on the software source code ( Whiteboxtest ).

Software modules contain data to be processed, and control structures that control the processing of the data. According to different tests based on the control flow, and tests that have access to data as a basis.

Control flow -oriented tests refer to logical expressions of the implementation. Data flow oriented criteria focus on the data flow of the implementation. In fact, they focus on the way in which respects the values ​​associated with their variables and how these statements affect the process of implementation.

Control flow oriented tests

The control flow based testing methods are based on the control flow graph of the program.

We distinguish between the following types:

  • Statement coverage (C0 )
  • Edges overlap ( C1)
  • Condition coverage (C2, C3)
  • Path coverage (C4 )

Statement testing ( test)

In the statement coverage test all nodes of the control flow graph can be covered by at least one test case, ie, each statement in the source code is executed at least once.

Branch coverage test ( edges overlap test) ( test)

Here are the test cases are specified so that all branches ( edges) are run through the control flow graph at least once.

This method has drawbacks because it is often difficult to go through all program branches, as certain operating system states or difficult to generating data constellations are required. In addition, combinations of branches and complex conditions are not adequately addressed. Finally, the program loops are not adequately tested, because a single pass through the loop body of repelling loops and a single repetition of non -repelling loops for the branch coverage is sufficient.

Simple condition testing ( test)

Here the structure of the decisions within the test blank for the test case specification is used. It is required, the test cases to be determined such that the evaluation of each atomic partial decision once the value true and once yields the value false.

This, however, does not consider that conditions often consist of nested logical combinations of partial decisions. This error situations are masked: In "or" - links, it is sufficient if the first part of condition is true when "and" - links if the first condition is false, in order to determine the further control flow. Errors in the other condition parts are not recognized. Therefore, the simple condition coverage test does not even guarantee a branch coverage.

Multiple condition coverage test ( test)

The maximum multiple coverage test requires that in addition to the atomic partial decisions and the overall decision all combined partial decisions are checked against true and false.

The main disadvantage is the extremely high testing effort: For n partial decisions, test cases arise. The modified condition testing requires test cases that demonstrate that each atomic partial decision can affect the truth value of the overall decision independently of the other partial decisions (This is about required by the standard RTCA DO- 178B for flight- critical software).

For n partial decisions arise minimal, maximal test cases.

Path coverage testing ( test)

In path coverage test all different paths of the test blank are run in at least one test case. In general, however, this is not feasible, as it is very often are infinitely many paths - caused by grinding without a fixed number of repetitions. The path coverage test therefore has standing alone little practical significance.

Data flow oriented tests

Data flow -oriented test methods are based on the data flow, so the access to variables. They are particularly suitable for object- oriented systems developed.

For the data- flow-oriented tests, there are different criteria, which are described below.

All defs criterion

For each definition ( all defs ) of a variable is tested a calculation or condition. For each node and a variable definition -free path must be tested to an item. The error detection rates for this criterion at about 24%.

All p -uses criterion, the " p- uses" used in the formation of truth values ​​within a predicate ( predicate -uses ). The error detection rates for this criterion at about 34%. It will be recognized in particular control flow errors.

All c -uses criterion

Under the "c -uses criterion" is understood within an expression calculation ( computation -uses ) of values. This criterion covers approximately 48% of all c -uses error. It identifies particular calculation errors.

Because of their complexity, the techniques for data flow-oriented test case specification are only usable with tool support. But since there are hardly any tools, they currently have no practical relevance.

Diversifying test methods

The group of diversified tests includes a set of test techniques, test the different versions of a software against each other. There is no comparison between the test results and the specification, but a comparison of the test results of the different versions. A test case is considered successful if the outputs of the different versions are identical. No completeness criterion is specified, in contrast to the function -oriented and oriented structure of the test methods. The necessary test data are created using one of the other techniques, by chance or recording of a user session.

The diversified tests bypass the often time-consuming evaluation of the test results against the specification. This course involves the risk that an error that produces the same result in the versions to compare, is not recognized. Due to the simple comparing to such tests can be automated very well.

Back-to- Back Test

When back-to- back test different versions to be tested against each other arising out of the n- version programming, that is, the programming of different versions of software according to the same specification. The independence of the programming team is a basic requirement.

This procedure is very expensive and only justified in accordance with high safety requirements.

Mutations test

The mutation test is no test technique in the strict sense, but a test of the capacity of other test methods, as well as the test cases used. The different versions created by artificial insertion of typical errors. It is then checked whether the used test method with the test data is available, these artificial errors. If an error is not found, the test data is added to the corresponding test cases. This technique is based on the assumption that an experienced programmer usually makes only " typical " error.

Regression test

Due to the ambiguous use of the term to the regression test a separate article is devoted.

Run time (program lifecycle phase) Static testing Compile time All-pairs testing Regression testing Keyword-Driven Testing Fuzz testing
96443
de