Test automation
Under test automation (and test automation ) is to understand the automation of activities in the test, both the software test and the automated test of hardware, the hardware test.
Motivation
In software development, it is particularly important to know a solid -defined status of the software, such as:
- Is the current, new software version better than the old version?
Automatic tests that test every day at specific times, the software, make software only measurable in terms of their quality and show potential side effects of modifications made directly and recognizable. They serve as direct feedback to developers and testers for that are not able in certain circumstances to survey the entire software system at once, as well as for the recognition of side effects and subsequent errors.
Test automation therefore provides a metric number of successful test cases for each test run. This causes the following questions can be answered:
- When is a new requirement entirely by software?
- When is a bug fixed?
- When is the work of the developer finished?
- Who is responsible for what at what time?
- What quality has a new software version (see stage of development (software) )?
- Is the new software version in better quality than the previous version?
- Has a fixed bug or a new request an impact on existing software ( changing the behavior of the software )?
- Is it guaranteed that the real-time operation with the new software is successfully and safely?
- What really is now on new functionality and bug fixes in the software; can you understand that?
- Can the delivery of the software yet to meet, if an assessment of the current quality of the software is not possible?
For example question: " When is a bug fixed " the answer in this case:
A response provides only the permanent testing, and this is only possible and achievable through automation.
A further advantage of the test automation is the acceleration of the development process. Where are sequentially performed manually in software projects without automation, production, installation and testing, can with fully automated projects ( ie if other than the test and production and installation can be automated ), these three steps are started automatically one after the other, for example in a nightly run. Depending on the scope of the project you may optionally start this process in the evening and have the next morning the test result available.
Automated activities
In principle, the following activities can be automated:
- Test case creation Test data generation
- Test scripting
Test case creation
Depending on the format used to describe a test case are used to automate test case generation by higher linguistic descriptions (test specifications ) can be transformed into this format. To test specification languages of different levels of abstraction are used: simple table-like notations for test data and function calls, scripting languages (such as Tcl, Perl, Python), imperative languages (eg, C, TTCN -3), object-oriented approaches ( JUnit ) and declarative and logical formalisms as well as model-based approaches (eg TPT). Here, a substantial and fully automated as possible translation of artifacts in a machine- language level in remote technical artifacts in a machine-level technical language level is aimed at. Another approach is to generate test cases based on business objects to be declared dynamically. If a test specification is not already in executable form, but in a language not executable (eg UML, Excel spreadsheet, or similar), it can be suitable tools automatically translated into executable under circumstances test cases.
Test data preparation and test scripting
Since the number of possible input values and processes of a program is often very large, must be selected during the generation of test cases from test specifications and procedures according to the input data to be obtained test coverage. To test data generation can often the data model of the software be used for test script creation of the software are in model -based testing behavioral models used. Solutions that do not require scripts are also available on the commercial market.
Test Procedure
The test is performed today largely by fully automated testing tools. Depending here come from the target system unit test tools, test systems for graphical user interfaces, load testing systems, hardware -in- the-loop test rigs or other tools are used.
Test Evaluation
To test evaluation, the obtained test results with the expected value to be compared. In the simplest case, only one table comparison is to be made; if the target behavior is, however, defined by logical constraints or contains extremely complex calculations, the so-called oracle problem can occur. If two software versions or two test cycles and thus two test results against the desired result are compared, it is trend statements and quality statistics can be calculated.
Test documentation
In the test documentation a comprehensible and understandable review is generated from the obtained test results. For this document generators and template tools can be used.
Test Administration
The object of the test administration is the management and versioning of test suites as well as providing an adequate environment of use. In addition to standard tools (eg, CVS, Eclipse) there are a number of specialized tools that are specifically tailored to the needs of software testing.
Universal architecture for test automation
For the automation of the activities listed above, there are various tools. This always focus on the solution of specific tasks and differ in operating philosophy, syntax and semantics. Therefore, it is often difficult to select the right tools for a particular set of activities or use the tools correctly. A structuring and classification of automated activities abstracted solution neutral tool functionality provides the universal test system architecture. To this end, it defines five functional levels: test management, test execution and evaluation, test bed control, test object stimulation and monitoring as well as test object environment. The test system architecture supports the integration of existing test tools and components into test systems and thus represents a universal foundation for test automation