1. Field of the Invention
The invention generally relates to systems, methods, and program products for repeating actions performed on graphical user interfaces (GUIs) of applications on different computer systems. More specifically, the present invention relates to robust ways for automatically and uniquely identifying the same or analogous GUI components across different computers, where differences in configuration, versions of applications, and environment contribute to the difficulty of such task.
2. Related Art
In this document, the following terms and abbreviations are used:
System denotes any arrangement of computer hardware and/or software such as a single software program, a single computer system, and/or a computer infrastructure containing multiple pieces of hardware and/or software;
GUI is an abbreviation for Graphical User Interface; and
Widget denotes any component of the GUI with which the user can interact.
FIG. 1 shows a method known in the art for recording user actions: In Step 101, a recording of user actions and widget information is started. In general, Step 101 ensures that Steps 103, 104, and 105 are executed while the user performs actions in Step 102. In Step 103, an action performed by a user on a widget of a GUI of an application is observed. One of ordinary skills in the art would appreciate how Step 103 can be performed by means of appropriate callback functions invoked by an operating system or by a virtual machine on which the application is running, in response to user actions. Most modern operating systems provide facilities for any application to set “hooks”. Hooks are in fact callback registrations for particular types of events. For example, Microsoft Windows (in all incarnations, including 98, 2000, XP) provides a variety of such hooks (Microsoft, Windows and related terms are trademarks of Microsoft, Inc. in the United States and/or other countries). Similar facilities are provided by the Java Virtual Machine (JVM) on which Java applications run (Java, JVM, and related terms are trademarks of Sun Microsystems, Inc. in the United States and/or other countries.
In Step 104 information about the widget (or widgets) involved in the user action is retrieved. As one of ordinary skills in the art would appreciate, modem computer systems typically provide means for obtaining information about a user interface widgets upon which action is being taken. For example, the various versions of the Microsoft Window operating systems provide means for programmatically acquiring information about UI widgets. A first category of means is provided by Microsoft Active Accessibility, a collection of software interfaces that allow users to programmatically retrieve certain pieces information about most widgets. A second category of means is provided by native Windows calls and messages, which allow the user to programmatically retrieve the certain pieces information. In still another example, it is possible to retrieve information regarding the widgets of applications running on JVMs using a mechanism called introspection. Regardless, Step 105 associates/records the information retrieved in Step 104 to the user action observed in Step 103.
FIG. 2 shows a method taught in the art for automatically executing a sequence of actions on an application GUI or on the GUI of multiple applications. In Step 201 the playback is started and appropriate initializations are performed. Step 202 iterates over the actions to be performed. Step 203 identifies the widget(s) involved in the action to be performed. Step 204 performed the action on identified widgets.
Unfortunately, none of these prior art approaches provide a way automatically and uniquely identifying the same or analogous GUI components across different computers, where differences in configuration, versions of applications, and environment contribute to the difficulty of such task, or within the same computer, where changes in UI configuration or in the user environment, as well as other causes, result in the mentioned difficulties. For example, there have been two ways of performing Step 203. The first consists of recording, in Step 104 of FIG. 1, positional information on the widget(s) upon which the user action was performed. Then Step 203 of Figure identifies the widget(s) at the recorded location and Step 204 performs the action on the identified widget(s). This method suffers from fragility. That is, if for some reason the position of a widget at the time of playback is different from its position at the time of playback, the method can fail to identify the widget. A second method taught in the art consists of recording, for each widget involved in a user action, a list of widget(s) in the path from the widget to its root window (this is, in Microsoft Windows, the window with a title bar that is not contained in any other window, and that contains the widget.) The limitation of this method is that multiple widgets can exist having the same path to the root window. For example, if an application has multiple toolbars, these toolbars can have the same path to the root window, and can be very difficult to distinguish from each other.
In view of the foregoing, there exists a need for a solution that solves at least one of the deficiencies of the related art.