Software development requires extensive repetitive testing as new versions of the software are produced throughout the development process. With each version comes a repetitive series of tests to determine if the changes made to create the new version have affected the software in an unintended manner. Thus, with each new version an operator must repeat all of the input commands tested on the previous version to ensure that the software still functions as intended.
In addition, software development is accomplished in modules which require complete testing as each new module is added to the software product. Thus repetitive testing is required to ensure that the new module, besides adding the functions it was designed to accomplish, will not detrimentally affect the previous version of the software product.
Until recently the repetitive testing was accomplished by a human operator generating each test command manually. Recent advances in technology have made it possible to record the keystroke input of an operator as the first version of software is tested. Subsequent versions are tested by playing back the previously recorded session. A system of this type is disclosed by Kerr et al. in U.S. Pat. No. 4,696,003, issued Sep. 22, 1987. Another relevant keystroke recording and playback system is the Advanced Test System For Software Test as disclosed in IBM Technical Disclosure Bulletin, Vol. 30, No. 9, Feb. 9, 1988.
Systems which record both the key strokes and mouse activity have recently entered the marketplace. However, these systems rely on intrusive hardware inserted into the system being tested. The additional hardware or software typically provides the testing system access to signals on the command bus within the system under test. This alleviates having to interpret mouse activity and keystroke signals in their unprocessed form. However, intrusion into a system under test is not desirable, it adds cost and is not practical for many systems.
To effectively evaluate a software package, the displayed graphical output of the various versions of software under test must be compared and the differences displayed. Kerr et al., as cited above, discloses a method of comparing stored images to determine differences which may arise in executing various versions of software.
However, certain portions of the display may not be relevant to the software-under-test. It would be advantageous for the operator to specify, during the execution of the first version of the software, which areas of the screen should be compared and which areas of the screen should be excluded from the comparison. It would also be advantageous to display the differences within the included areas as hi-lighted images. Moreover, the comparison should not require the compared images to be identical, but should enable the operator to set comparison tolerances which must be exceeded before a difference is noted.