Field of the Invention
The invention relates to a computer-implemented method for testing a real and/or virtual mechatronic automotive system or another real and/or virtual mechatronic system through a test via a test environment interacting with the real and/or virtual mechatronic system, wherein the method includes a test series with different test cases or test types of the test for different execution conditions, wherein these execution conditions are specified in test configurations. The invention also relates to a corresponding computer program product and a corresponding computer-based control system for controlling corresponding tests.
Description of the Background Art
A large amount of data is produced within the framework of a development process for mechatronic systems such as control units and their software. In addition to the test cases, test results are also produced at every test execution. The task of managing the test cases has existed for quite some time. Current developments in the area of standardization and certification have given rise to standards that have placed greater emphasis on management of test results for the purpose of test documentation.
Test execution is increasingly becoming the “bottleneck” in this process. Test execution is very time-consuming because of the complex test environment at the hardware-in-the-loop simulator (HIL) and because of the great number of tests. Due to the growing number of possible combinations (variants) for the end product (motor vehicle, for example) in present-day development projects, it is de facto becoming ever more difficult to carry out all test cases with every environment constellation (variant configuration). Moreover, the question arises as to an “intelligent” selection of the test cases to be executed and the possible combinations for the environment constellation.
The enormous number of test cases already makes it difficult for the manufacturer to execute every test case. If one now additionally wishes to test every combination of the test environment, an enormous expenditure of time and money is incurred. Primarily the time aspect oftentimes leads responsible test managers to select individual combinations for the tests on the basis of their experience. “Gaps” that occur in the process are tolerated of necessity. A suitable overview of the tests carried out as a function of the environment constellations is lacking here. The goal in this case is knowledge of the “gaps.” This knowledge is utilized for a statement about test progress and hence about the maturity of the software. Moreover, suitable means are lacking for determining effective environment constellations and combinations.
A great number of specifications must therefore be made prior to performance of a test in order to specify the test clearly, and thus also reproducibly. This includes the precise definition of the system under test (SUT: system under test) through hardware and software. It is necessary to specify whether only individual components or a connected system are to be tested, and for this purpose the appropriate hardware and its interconnection as well as the software version(s) and their specific parameterization must be defined. Moreover, a clear definition of the test environment is required. This starts with the execution environment of the test. Offline simulations, hardware-in-the-loop tests (HIL), or even a test under “real” conditions (e.g., in the vehicle) may be carried out here. It is also necessary to specify whether execution of the test should be manual or automated, and the appropriate instructions (e.g., sequence plans or test implementations) must be specified. For these instructions as well, different versions or implementations may exist for the same circumstances under test.
Conditional or mutually exclusive dependencies exist between many of these specifications necessary for a test. For example, a control unit that is physically present (hardware) is necessary for an HIL test, while just the algorithm (pure software) may be sufficient for a simulation on a PC. Accordingly, an HIL system test can be performed automatically, while a test in the real vehicle must be performed manually.
A portion of this complete configuration and its internal dependencies is present in the form of a variant model. Here, variant decisions (frequently) derived from the product are taken as points of variation. In addition, as described above, other information that usually cannot be represented with a variant model is required for a test. It is no longer possible to effectively manage all of this information about possible configurations and their dependencies either manually or by simple means such as tables. Consequently, it is often stored in databases and is processed and evaluated by means of tools designed especially for the management of tests.
It is no longer possible to manually select from this confusing mass of “complete” and permitted test configurations, on the basis of objective criteria, the test or tests and test configurations that deliver the greatest possible progress from the testing. Consequently, decisions about the next test configuration or configurations to be executed are frequently made in individual steps. Thus, for example, the SUT is specified first, then the test environment, then the method of execution (manual or automatic), and lastly the variant configuration (in several steps, if applicable). By means of each of these steps, the choice of possible test configurations is limited further through filtering. In this way, decisions concerning the test configurations to be selected are always made only from a subset of all possible test configurations, and a comparison of all test configurations is never made.
Now, a problem resides in representing test progress and resources in a form that is consistent in each case and to place them in relation to one another.