Recently I have been working on recoding some of our Continuous tests, changing from using Fest to using Jemmy in conjunction with JUnit. This has led to me creating easily extendible tests so that creating new tests becomes quick and simple allowing us to easily introduce new tests should we find an area not covered by our existing suite.
These tests use a simple frame work created using JUnit and the actual test that interacts with the application are done using Jemmy. This article covers the use of JUnit to create the simple frame work. Future articles will deal with the implementation of the actual test.
The idea
First, to create a test structure that makes creating new tests quick and simple we need to consider the structure of a test. Generally a test contains the following three stages.
Set up → Functionality → Compare
The Set up stage is where we pass in the file names for any files or folders we wish to test and create / clear any required output folders for the test. Once we have these set up it is just a matter of cycling through a set of files or values that you are testing, passing one at a time into the actual test. This stage can be reused in multiple tests if written correctly, saving the need to rewrite it each time.
The Functionality stage is where we actually test the functionality and save the output. This stage is the stage that is replaced based on the required test. The only part that can be reused is the results output. If you are testing some GUI functionality you may wish to output an image of the GUI, if the output is some extracted text content then you want to output the extracted text.
The Compare stage is where we compare the output from the test against a baseline that is known to correct and can flag if the output matches the baseline or not. Again this stage can be reused. There is no need to rewrite image comparison code for each tests that outputs an image.
The Implementation
Implementing the tests is actually relatively straight forward. It can be done with a single base class and at least a single test class. I will be assuming the test outputs an image of the application on screen and compares that against the baseline. It is important to note that at any point where the test should fail (such if the input folder is empty) you should use a JUnit to cause the test to fail such as org.junit.Asset.fail(String message) with an appropriate message for the cause.
The Base Class
The base class is the class that will contain all the reusable code for the tests. The methods that are required are,
- void testFiles() : This method will check the input folder and loop through all files found and pass the file name to the method void test(String filename). After all the files have been passed to test this method calls the compare method.
- void test(String filename) : This method takes the file name and performs the actual test. In the base class this method is empty. You override this method in the test class later.
- boolean compare(String output, String baseline) : This method compares the output to the baseline, the input are the folders containing the files to compare. We return true is the compare is passed and false if failed.
- void getScreenshot(Component com, String filename) : This method creates the output image to be tested against the baseline.
We also have the following three Strings in the base class each declared as empty Strings.
- outputFolder : File name for the output folder
- inputFolder : File name for the input folder
- baselineFolder : File name for the baseline folder
The Test Class
The test class extends the base class and requires only a Constructor and two other methods.
- The constructor : The Constructor should have the three variables mentioned above set to the folders required for the tests.
- void test(String filename) : This method overrides the method found in the base class and should contain the code that uses the functionality you wish to test and makes a call to getScreenshot(Component com, String filename) with filename being the outputs filename for this test.
- void testFiles() : This method includes the annotation @Test and called the super method ( super.testFiles() ). This is the method that actually begins the test.
That is the basic idea. Below are two classes created using the details above. The classes are incomplete, they just require the test method to be implemented in the test class and the compare method in the base class.
In my next article I will cover additional options such as creating the tests to allow for multiple platforms, other GUI tool kits (mainly Swing and JavaFX) and a look at creating a Jemmy test to add to this basic frame. See you then.
Base Class.java
import java.awt.Component; import java.io.*; import static org.junit.Assert.*; import org.netbeans.jemmy.util.PNGEncoder; public class BaseClass{ protected String outputFolder = ""; protected String inputFolder = ""; protected String baselineFolder = ""; public BaseClass(){} public void testFiles(){ try { /** * open all files */ File filesToTest = new File(inputFolder); String[] fileList = filesToTest.list(); //check we have some files if (fileList == null){ fail("No files in " + inputFolder); } for (String fileName : fileList){ test(fileName); } //Compare all output and check for changes if (compareImages(outputFolder, baselineFolder)){ fail("Current output does not match baseline."); } } catch (Exception e){ e.printStackTrace(); fail(e.toString()); } } public void getScreenshot(Component com, String filename){ PNGEncoder.captureScreen(com, filename, PNGEncoder.COLOR_MODE); } private boolean compareImages(String output, String baseline){ /* ADD CODE TO GET FILES FROM FOLDER NAMES AND COMPARE THEM */ return false; } public void test(String testFile){ //Do nothing for generic test } }
Test Class.java
import org.junit.Test; public class TestClass extends BaseClass{ public TestClass(){ inputFolder = "C:\\testData\\test1\\input\\"; outputFolder = "C:\\testData\\test1\\output\\"; baselineFolder = "C:\\testData\\test1\\baseline\\"; } @Override public void test(String testFile){ /* ADD YOUR TEST HERE */ getScreenshot(component, outputFilename); } @Test @Override public void testFiles(){ super.testFiles(); } }
Tune in next time for ‘Running tests on both JavaFX and Swing using Junit: