title: Certain JUnit test take insane amounts of time in Ant
body: <p>Related: <a href="https://stackoverflow.com/questions/123127/ant-junit-tests-are-running-much-slower-via-ant-than-via-ide-what-to-look-at">Ant Junit tests are running much slower via ant than via IDE - what to look at?</a></p>
  <p>I am having the same problem as the other question, but I have narrowed the scope a lot.</p>
  <p>I have a large suite of unit test (namely 800 test cases) which should be run by Atlassian Bamboo CI. I have rewritten my build script entirely.</p>
  <p>When running all tests in my <code>test</code> folder in IntelliJ IDEA, everything takes a 7-minutes reasonable time, including Spring-context-based tests requiring a new instance of H2 populated with tables.</p>
  <p>When running in Ant, both locally and on CI, the build takes an <strong>insane</strong> 1-hour time to test. Putting 8 threads on duty reduces the time to 20 minutes (the slowest job).</p>
  <p>I had suspected that the heaviest tests were related to Spring and in memory database. I have investigated a lot today and found a shortlits of bad tests that perform awfully, but I am here because I can't understand the reason.</p>
  <p>Most of awfully-performing tests are entirely in-memory with little to none system output. While there is a relatively &quot;large&quot; number of assertion operations, iterations, etc. the same test performs awfully on Ant and performs great on IDE.</p>
  <h2>Pick the following tests:</h2>
  <ul>
  <li><strong>SortedLinkedListTests</strong>: 1.163 seconds</li>
  </ul>
  <p>I have a SortedLinkedList component (linked list with O(n) sorted insertion) along with its unit tests. No logging framework, no Spring context. Nothing.</p>
  <p>Basically the unit tests operate on 10000 iterations. The following very single test took 346 seconds.</p>
  <pre><code>@Test
  public void testAddWithoutComparator()
  {
      final SortedLinkedList&lt;Integer&gt; uut = new SortedLinkedList&lt;&gt;();
  
      final Random random = new Random();
  
      for (int i = 0; i &lt; ITERATIONS; i++)
      {
          uut.add(random.nextInt());
  
          Integer previous = null;
          if (i % 50 == 0)
              for (Integer candidate : uut)
              {
                  assertNotNull(candidate);
                  if (previous != null)
                      assertTrue(previous.compareTo(candidate) &lt;= 0);
                  previous = candidate;
              }
      }
  }
  </code></pre>
  <p>Maybe the code is a bit dumb, can be optimized a lot, but it's not so bad, not <strong>that</strong> bad. On 1 to 10000 iterations, it checks that the list is sorted at every step, not just at the beginning or at the end. Think about 10000 incremental tests.
  This test has a complexity O(n^2), and I am happy with that. I am happy that the test, not writing anything to IO, runs in 4 seconds in memory.</p>
  <p>Again, a lot can be simplified, or optimized, but 4s vs 346s is insane IMO</p>
  <ul>
  <li><strong>BeanUtilsTests</strong></li>
  </ul>
  <p>I have a BeanUtils utility that operates on properties and reflections. This very single test, including an AES encryption operation, takes 25 seconds to run in Ant</p>
  <pre><code>@Test
  public void testMarshal_Password()
  {
      String passwordPlaintext = RandomStringUtils.randomAlphanumeric(25);
      assertThat(BeanUtils.marshal(passwordPlaintext, PASSWORD), is(not(equalTo(passwordPlaintext))));
  }
  </code></pre>
  <p>No iterations!!! Internally, it is a switch statement that redirects to an encryption utility. The key is hardcoded in the code, no need to generate random secure keys. Cipher.Init and then encrypt the random string.</p>
  <p>It takes 1s in IDE, probably a lot more time to initialize testing runtime than perform the test.</p>
  <h2>The Ant build</h2>
  <pre><code>&lt;!-- JUNIT TARGETS --&gt;
  &lt;target name=&quot;debug-junit&quot;&gt;
      &lt;property name=&quot;junit.include&quot; value=&quot;**/*.java&quot; /&gt;
      &lt;condition property=&quot;jarOk&quot;&gt;
          &lt;resourcecount count=&quot;1&quot;&gt;
              &lt;fileset id=&quot;fs&quot; dir=&quot;${project.artifactsDirectory}&quot; includes=&quot;${artifact.name}-junit.jar&quot;/&gt;
          &lt;/resourcecount&gt;
      &lt;/condition&gt;
      &lt;fail message=&quot;No jar file. Please package the project first&quot; unless=&quot;jarOk&quot;/&gt;
  
      &lt;mkdir dir=&quot;${junit.destDir}&quot;/&gt;
  
      &lt;junit printsummary=&quot;yes&quot; haltonfailure=&quot;yes&quot; dir=&quot;${basedir}&quot; forkmode=&quot;once&quot; logfailedtests=&quot;true&quot; maxmemory=&quot;1024m&quot; fork=&quot;true&quot; showoutput=&quot;false&quot;&gt;
          &lt;classpath refid=&quot;project.classpath.test&quot;/&gt;
  
          &lt;jvmarg value=&quot;-Djava.compiler=NONE&quot;/&gt;
          &lt;jvmarg value=&quot;-agentlib:jdwp=transport=dt_socket,address=${junit.debugPort},server=y,suspend=y&quot;/&gt;
  
          &lt;batchtest todir=&quot;${junit.destDir}&quot; haltonerror=&quot;false&quot; haltonfailure=&quot;false&quot; skipnontests=&quot;true&quot;&gt;
              &lt;fileset dir=&quot;${junit.srcDir}&quot; includes=&quot;${junit.include}&quot;/&gt;
              &lt;formatter type=&quot;xml&quot; /&gt;
          &lt;/batchtest&gt;
      &lt;/junit&gt;
  &lt;/target&gt;
  
  &lt;target name=&quot;run-junit&quot;&gt;
      &lt;property name=&quot;junit.include&quot; value=&quot;**/*.java&quot; /&gt;
      &lt;condition property=&quot;jarOk&quot;&gt;
          &lt;resourcecount count=&quot;1&quot;&gt;
              &lt;fileset id=&quot;fs&quot; dir=&quot;${project.artifactsDirectory}&quot; includes=&quot;${artifact.name}-junit.jar&quot;/&gt;
          &lt;/resourcecount&gt;
      &lt;/condition&gt;
      &lt;fail message=&quot;No jar file. Please package the project first&quot; unless=&quot;jarOk&quot;/&gt;
  
      &lt;mkdir dir=&quot;${junit.destDir}&quot;/&gt;
  
      &lt;junit printsummary=&quot;yes&quot; haltonfailure=&quot;yes&quot; dir=&quot;${basedir}&quot; forkmode=&quot;once&quot; logfailedtests=&quot;true&quot; maxmemory=&quot;1024m&quot; fork=&quot;true&quot; showoutput=&quot;false&quot;&gt;
          &lt;classpath refid=&quot;project.classpath.test&quot;/&gt;
  
          &lt;jvmarg value=&quot;-Djava.compiler=NONE&quot;/&gt;
          &lt;jvmarg value=&quot;-ea&quot;/&gt;
  
          &lt;batchtest todir=&quot;${junit.destDir}&quot; haltonerror=&quot;false&quot; haltonfailure=&quot;false&quot; skipnontests=&quot;true&quot;&gt;
              &lt;fileset dir=&quot;${junit.srcDir}&quot; includes=&quot;${junit.include}&quot;/&gt;
              &lt;formatter type=&quot;xml&quot; /&gt;
          &lt;/batchtest&gt;
      &lt;/junit&gt;
  &lt;/target&gt;
  </code></pre>
  <p>Explanations:</p>
  <ul>
  <li><code>junit.debugargs</code> is overwritten when I want to attach a debugger to debug tests. Here you don't have debug-specific properties</li>
  <li><code>fork=&quot;true&quot;</code> just to make sure that classpath is not <strike>infected</strike> affected by Ant-related classes</li>
  <li><code>showoutput=&quot;false&quot;</code> to reduce IO. In particular, Bamboo remote agents transmit <code>stdout</code> to master, reducing performance</li>
  <li><code>forkmode</code> I also have tried perTest mode with parallelism. Now I reverted to single-threading mode because Atlassian Bamboo runs on parallel agents</li>
  <li><code>logfailedtests=&quot;true&quot;</code> failures are important, not all tests fail</li>
  </ul>
  <h2>Question</h2>
  <p>Somebody help please? I need my build to run in a barely decent time</p>
  <h2>Updates</h2>
  <blockquote>
  <p>Update 1: narrowing the scope</p>
  </blockquote>
  <p>I am now trying to use arguments appropriately to run a single test class on my local Ant. I found SortedLinkedListTests to be running slow, so I am now trying to run it alone. It still takes a lot of time.</p>
  <p>I have managed to reduce the execution time of BeanUtilsTests by reducting the number of iterations <strong>of another test method</strong> by a magnitude. And then, even the password test runs like a charm.</p>
  <blockquote>
  <p>Update 2: suite helps</p>
  </blockquote>
  <p>I was trying to recall what has changed since I rewrote my build from scratch, and replicate.</p>
  <p>My previous build had a similar <code>run-junit</code> Ant task that only invoked a single <code>@Suite</code>-annotated class file, which referenced all tests to run manually. This is one of the reasons I rewrote the build. I have checked Bamboo for past logs of tests. <code>SortedLinkedListTests</code> executed in a few seconds with the previous build</p>
  <p>This is the old runner. I can't see anything that would drop the performance so hard and bad</p>
  <pre><code>&lt;target name=&quot;run-junit&quot;&gt;
      &lt;condition property=&quot;jarOk&quot;&gt;
          &lt;resourcecount count=&quot;1&quot;&gt;
              &lt;fileset id=&quot;fs&quot; dir=&quot;${dir.publish}&quot; includes=&quot;${artifact.name}-junit-*.jar&quot;/&gt;
          &lt;/resourcecount&gt;
      &lt;/condition&gt;
      &lt;fail message=&quot;No jar file. Please package the project first&quot; unless=&quot;jarOk&quot;/&gt;
  
      &lt;mkdir dir=&quot;${dir.junitResult}&quot;/&gt;
      &lt;property name=&quot;junit.srcDir&quot; value=&quot;test&quot;/&gt;
      &lt;fail message=&quot;Must set -Djunit.testSuite&quot; unless:set=&quot;junit.testSuite&quot;/&gt;
  
      &lt;junit printsummary=&quot;yes&quot; haltonfailure=&quot;yes&quot; dir=&quot;${basedir}&quot; fork=&quot;true&quot; showoutput=&quot;true&quot;&gt;
          &lt;classpath refid=&quot;project.classpath.test&quot;/&gt;
          &lt;classpath location=&quot;build/compiled-test&quot;/&gt;
  
          &lt;batchtest todir=&quot;${dir.junitResult}&quot; haltonerror=&quot;false&quot; haltonfailure=&quot;false&quot; fork=&quot;true&quot;&gt;
              &lt;fileset dir=&quot;${junit.srcDir}&quot; includes=&quot;**/${junit.testSuite}.java&quot;/&gt;
              &lt;formatter type=&quot;xml&quot;/&gt;
          &lt;/batchtest&gt;
          &lt;formatter type=&quot;xml&quot;/&gt;
      &lt;/junit&gt;
  &lt;/target&gt;
  </code></pre>