Software applications are increasingly becoming the core of businesses in various industries, as a result of which functional conformance testing of software applications is playing an important part of the development and implementation effort. Functional conformance testing is an activity, which ensures that the implementation conforms to some expected software behavior. A functional test case generally has three main components: set up sequence, test action, and verification sequence. A set up sequence comprises of one or more input actions for the software necessary to drive the software to a state, which can enable a test action. A test action is some input action for the software under test along with an expected output from the software. If the software produces the expected output, the software is said to “pass” the test otherwise it is said to “fail” the test. A verification sequence comprises of one or more input actions followed by an expected output from the software. The purpose of the verification sequence is to determine that the internal state of the software is consistent with its behavior during the test.
Automatic generation of test cases for a given software application helps reduce errors in testing and also makes the process more manageable. Model-Based conformance Test Generation (MBTG) is one technique to automate generation of test cases. In MBTG, the test cases are generated on the basis of a model for the software. Typically, the models are described in terms of operations provided by the system under test (SUT) and include for each operation a description in terms of its Inputs, Outputs, Preconditions, and Effects (IOPEs)
In the past, MBTG techniques required the test designer to additionally provide the test objective specifications to guide the test generation process. This becomes a cumbersome requirement for the test designer for almost all commercial software applications. Also, using the IOPE representation some past techniques have proposed techniques for automatic test generation. However, these techniques only partially automate the test generation process—typically generating a set of transition sequence with constraints on parameters and state. However, the constraints on these transition sequences may not be feasible. The tester has the responsibility of determining feasibility of each sequence and of generating concrete values corresponding to parameters for each transition in a sequence. These techniques do not typically address the issue of generating verification sequence—a sequence, which enables verification of system state (both control and data) obtained as a result of applying each transition sequence.
Petrenko et al. describe a technique for generating such sequences for IOPE models. However, their technique is only applicable for a restricted set of models. For instance, their technique is not applicable to UML object diagrams. Another technique, named the BZ-TT testing technique also recognizes the issue of verification sequence generation. However, it requires the tester to identify a subset of model operations, which do not modify the system state but return some aspect of the system state to the environment as observer operations. An appropriate invocation of each observer operation is appended to each generated test case. While observer operations are very beneficial, a suitable observer operation may not be available for each system state variable.
Thus, it is desirable to have an IOPE model-based test generation environment, which addresses the aforementioned issues of generating verification sequences and automatically selecting the test objectives.