Multi-threaded testing with JUnit. Switch to ConcJUnit .

Our JUnit tests deal with components that use multiple threads. Unfortunately JUnit has no built-in support for this kind of testing.

Lets consider a simple thread that interprets commands from input queue. In this example it can only understand "exit" command. All other inputs trigger an error.

 
 
abstract class MainLoop extends Thread { 
 
    private Queue<String> commands = new LinkedBlockingDeque<String>(); 

 
    public void run () { 
        while (true) { 
            String command = commands.poll(); 
            if (command.equals ("exit")) 
                break; 

            else 
                reportError ("Unknown command: " + command); 
        } 
    } 
 
    public void add (String command) { 
        commands.add(command); 

    } 
 
    abstract void reportError (String error); 
} 
 
public class MyTest { 
 
    @Test 
    public void test() { 
        MainLoop thread = new MainLoop() { 

            void reportError(String error) { 
                Assert.fail (error); 
            } 
        }; 
        thread.start(); 
        thread.add ("quit"); // sic! shoudl be "exit" 

    } 
} 

If we run the above test, JUnit completely ignores failure of "Commands Interpreter" thread (Although default uncaughtExceptionHandler prints exception to standard output):

   [junit] Running test.MyTest
   [junit] Exception in thread "Commands Interpreter": java.lang.AssertionError: Unknown command: quit
   [junit]      at org.junit.Assert.fail(Assert.java:91)
   [junit]      at test.MyTest$1.reportError(MyTest.java:36)
   [junit]      at test.MainLoop.run(MyTest.java:19)
   [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.422 sec

As you can see JUnit detected 0 errors. JUnit has at least two problems when it comes to multi-threaded tests:

1. JUnit only cares about errors in its own thread. It completely ignores exceptions or assertion failures in child threads.


2. Left over child threads may produce side effect on subsequent tests. In some cases failures in child threads may happen after the main thread is finished.

We used various custom workarounds until it was clear that we need a generic solution for growing number of multi-threaded tests. Luckily we found an excellent tool - ConcJUnit (AKA ConcuTest) written by Mathias Ricken.

a) ConcJUnit is a drop-in replacement of JUnit. It includes standard JUnit classes and introduces own test runners. Just replace junit.jar by concutest.jar .

b) ConcJUnit detects both failures in child threads and leftover threads.

Switching to ConcJUnit produces expected result:

java.lang.AssertionError: Unknown command: quit
 at org.junit.Assert.fail(Assert.java:91)
 at test.MyTest$1.reportError(MyTest.java:36)
 at test.MainLoop.run(MyTest.java:19)
 at test.MainLoop.run(MyTest.java:19)

If you want to know more about ConcJUnit, read materials from introduction page. For example, the following presentation.