[Note this additional post on this topic]

With the addition of the ClassRule annotation in JUnit 4.9, I thought I’d come back and revisit JUnit test order execution.

@ClassRule fills a gap in the JUnit API, by providing class level rules like @BeforeClass and @AfterClass provide class-wide set up and tear down.

This article explains how to control code execution of unit tests and where @ClassRule fits in.

The terminology I use:

A test implements @Test
A test case in a class with @Test methods

Ordering test methods

The simplest test case you can write in JUnit is to annotate methods with @Test:

package test;

import org.junit.Test;

public class OrderTest1 {

    @Test
    public void test1() {
        println(&quot;@Test test1()&quot;);
    }

    @Test
    public void test2() {
        println(&quot;@Test test2()&quot;);
    }

    private void println(String string) {
        OrderTestUtils.println(OrderTest1.class, string);
    }
}

In OrderTest1, the execution order is:

OrderTest1 @Test test1()
OrderTest1 @Test test2()

Managing test fixtures

If you need to initialize the same data for each test, you put that data in instance variables and initialize them in a @Before setUp method. The setUp method is called before each @Test method.

One test invocation becomes the following call sequence:

Call @Before setUp
Call one @Test method

If that data needs to be cleaned up, implement an @After tearDown method. The tearDown method is called after each @Test method.

One test invocation becomes the following call sequence:

Call @Before setUp
Call one @Test method
Call @After tearDown

The call sequence for a class with two test methods is:

Call @Before setUp
Call @Test method test1
Call @After tearDown
Call @Before setUp
Call @Test method test2
Call @After tearDown

For example:

package test;

import java.io.Closeable;
import java.io.IOException;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class OrderTest2 {

    static class ManagedResource implements Closeable {
        @Override
        public void close() throws IOException {
        }
    }

    private ManagedResource managedResource;

    private void println(String string) {
        OrderTestUtils.println(OrderTest2.class, string);
    }

    @Before
    public void setUp() {
        this.println(&quot;@Before setUp&quot;);
        this.managedResource = new ManagedResource();
    }

    @After
    public void tearDown() throws IOException {
        this.println(&quot;@After tearDown&quot;);
        this.managedResource.close();
        this.managedResource = null;
    }

    @Test
    public void test1() {
        this.println(&quot;@Test test1()&quot;);
    }

    @Test
    public void test2() {
        this.println(&quot;@Test test2()&quot;);
    }
}

In OrderTest2, the execution order is:

OrderTest2 @Before setUp
OrderTest2 @Test test1()
OrderTest2 @After tearDown
OrderTest2 @Before setUp
OrderTest2 @Test test2()
OrderTest2 @After tearDown

Managing expensive test fixtures

When a resource is expensive to manage like a connection to a server, a database, or even managing an embedded server, it’s best to only initialize that resource once for the whole test case. You want to avoid starting and stopping a server for each @Test method. Instead, initialize the server once for all the tests in the class.

To do so, we use @BeforeClass and @AfterClass methods, instead of @Before and @After to get the following call sequence:

Call @BeforeClass setUpClass
Call @Test method test1
Call @Test method test2
Call @AfterClass tearDownClass

Let’s add class-level set up and tear down to our example:

package test;

import java.io.Closeable;
import java.io.IOException;

import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Test;

public class OrderTest3 {

    static class ExpensiveManagedResource implements Closeable {
        @Override
        public void close() throws IOException {
        }
    }

    static class ManagedResource implements Closeable {
        @Override
        public void close() throws IOException {
        }
    }

    @BeforeClass
    public static void setUpClass() {
        OrderTestUtils.println(OrderTest3.class, &quot;@BeforeClass setUpClass&quot;);
        MyExpensiveManagedResource = new ExpensiveManagedResource();
    }

    @AfterClass
    public static void tearDownClass() throws IOException {
        OrderTestUtils.println(OrderTest3.class, &quot;@AfterClass tearDownClass&quot;);
        MyExpensiveManagedResource.close();
        MyExpensiveManagedResource = null;
    }

    private ManagedResource myManagedResource;
    private static ExpensiveManagedResource MyExpensiveManagedResource;

    private void println(String string) {
        OrderTestUtils.println(OrderTest3.class, string);
    }

    @Before
    public void setUp() {
        this.println(&quot;@Before setUp&quot;);
        this.myManagedResource = new ManagedResource();
    }

    @After
    public void tearDown() throws IOException {
        this.println(&quot;@After setUp&quot;);
        this.myManagedResource.close();
        this.myManagedResource = null;
    }

    @Test
    public void test1() {
        this.println(&quot;@Test test1()&quot;);
    }

    @Test
    public void test2() {
        this.println(&quot;@Test test2()&quot;);
    }
}

In OrderTest3, the execution order is:

OrderTestAll @BeforeClass setUpClass
OrderTestAll @Before setUp
OrderTestAll @Test test1()
OrderTestAll @After setUp
OrderTestAll @Before setUp
OrderTestAll @Test test2()
OrderTestAll @After setUp
OrderTestAll @AfterClass tearDownClass

You can see that the setUpClass and tearDownClass wrap the execution of this test case.

Managing resources with rules

Instead of duplicating resource management code in each class, you can reuse your code, but instead of putting this common code in a superclass for all your tests. you can abstract external resource management with a JUnit Rule.

JUnit rules are subclasses of the ExternalResource class.

Let’s do it both ways and compare.

package test;

import java.io.Closeable;
import java.io.IOException;

import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExternalResource;

public class OrderTest4 {

    static class ExpensiveExternalResource extends ExternalResource {

        ExpensiveExternalResource() {
            OrderTestUtils.println(ExpensiveExternalResource.class, &quot;constructor&quot;);
        }

        @Override
        protected void after() {
            OrderTestUtils.println(ExpensiveExternalResource.class, &quot;after&quot;);
        };

        @Override
        protected void before() throws Throwable {
            OrderTestUtils.println(ExpensiveExternalResource.class, &quot;before&quot;);
        };
    };

    static class ExpensiveManagedResource implements Closeable {
        @Override
        public void close() throws IOException {
        }
    }

    static class ManagedResource implements Closeable {
        @Override
        public void close() throws IOException {
        }
    }

    @BeforeClass
    public static void setUpClass() {
        OrderTestUtils.println(OrderTest4.class, &quot;@BeforeClass setUpClass&quot;);
        MyExpensiveManagedResource = new ExpensiveManagedResource();
    }

    @AfterClass
    public static void tearDownClass() throws IOException {
        OrderTestUtils.println(OrderTest4.class, &quot;@AfterClass tearDownClass&quot;);
        MyExpensiveManagedResource.close();
        MyExpensiveManagedResource = null;
    }

    @Rule
    public ExternalResource resource = new ExpensiveExternalResource();

    private ManagedResource myManagedResource;

    private static ExpensiveManagedResource MyExpensiveManagedResource;

    private void println(String string) {
        OrderTestUtils.println(OrderTest4.class, string);
    }

    @Before
    public void setUp() {
        this.println(&quot;@Before setUp&quot;);
        this.myManagedResource = new ManagedResource();
    }

    @After
    public void tearDown() throws IOException {
        this.println(&quot;@After tearDown()&quot;);
        this.myManagedResource.close();
        this.myManagedResource = null;
    }

    @Test
    public void test1() {
        this.println(&quot;@Test test1()&quot;);
    }

    @Test
    public void test2() {
        this.println(&quot;@Test test2()&quot;);
    }
}

The methods are run in the following order:

OrderTest4 @BeforeClass setUpClass
ExpensiveExternalResource constructor
ExpensiveExternalResource before
OrderTest4 @Before setUp
OrderTest4 @Test test1()
OrderTest4 @After setUp
ExpensiveExternalResource after
ExpensiveExternalResource constructor
ExpensiveExternalResource before
OrderTest4 @Before setUp
OrderTest4 @Test test2()
OrderTest4 @After setUp
ExpensiveExternalResource after
OrderTest4 @AfterClass tearDownClass

The ExternalResource before method is called just before the @Before setUp method.

The ExternalResource after method is called just before the @After setUp method.

An ExternalResource is a nice way to abstract out and resuse your resource management code. You can now compose use of your resources instead of subclassing.

Managing expensive resources with rules

Just like you can wrap a test case with @BeforeClass and @AfterClass, there was no way to do this with rules until JUnit 4.9 and the ClassRule annotation.

The following examples shows a ClassRule at work. Deltas from the previous example are highlighted.

package test;

import java.io.Closeable;
import java.io.IOException;

import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.BeforeClass;
import org.junit.ClassRule;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExternalResource;

public class OrderTest5 {

    static class ExpensiveExternalResource extends ExternalResource {

        private final String info;

        ExpensiveExternalResource(String info) {
            this.info = info;
            OrderTestUtils.println(ExpensiveExternalResource.class, &quot;constructor &quot; + info);
        }

        @Override
        protected void after() {
            OrderTestUtils.println(ExpensiveExternalResource.class, &quot;after &quot; + this.info);
        };

        @Override
        protected void before() throws Throwable {
            OrderTestUtils.println(ExpensiveExternalResource.class, &quot;before &quot; + this.info);
        };
    };

    static class ExpensiveManagedResource implements Closeable {
        @Override
        public void close() throws IOException {
        }
    }

    static class ManagedResource implements Closeable {
        @Override
        public void close() throws IOException {
        }
    }

    @BeforeClass
    public static void setUpClass() {
        OrderTestUtils.println(OrderTest5.class, &quot;@BeforeClass setUpClass&quot;);
        MyExpensiveManagedResource = new ExpensiveManagedResource();
    }

    @AfterClass
    public static void tearDownClass() throws IOException {
        OrderTestUtils.println(OrderTest5.class, &quot;@AfterClass tearDownClass&quot;);
        MyExpensiveManagedResource.close();
        MyExpensiveManagedResource = null;
    }

    @Rule
    public ExternalResource resource = new ExpensiveExternalResource(&quot;instance&quot;);

    @ClassRule
    public static ExternalResource StaticResource = new ExpensiveExternalResource(&quot;static&quot;);

    private ManagedResource myManagedResource;

    private static ExpensiveManagedResource MyExpensiveManagedResource;

    private void println(String string) {
        OrderTestUtils.println(OrderTest5.class, string);
    }

    @Before
    public void setUp() {
        this.println(&quot;@Before setUp&quot;);
        this.myManagedResource = new ManagedResource();
    }

    @After
    public void tearDown() throws IOException {
        this.println(&quot;@After tearDown()&quot;);
        this.myManagedResource.close();
        this.myManagedResource = null;
    }

    @Test
    public void test1() {
        this.println(&quot;@Test test1()&quot;);
    }

    @Test
    public void test2() {
        this.println(&quot;@Test test2()&quot;);
    }
}

The test run shows the ClassRule kick in at the start and end of the test case:

ExpensiveExternalResource constructor static
ExpensiveExternalResource before static
OrderTest5 @BeforeClass setUpClass
ExpensiveExternalResource constructor instance
ExpensiveExternalResource before instance
OrderTest5 @Before setUp
OrderTest5 @Test test1()
OrderTest5 @After tearDown()
ExpensiveExternalResource after instance
ExpensiveExternalResource constructor instance
ExpensiveExternalResource before instance
OrderTest5 @Before setUp
OrderTest5 @Test test2()
OrderTest5 @After tearDown()
ExpensiveExternalResource after instance
OrderTest5 @AfterClass tearDownClass
ExpensiveExternalResource after static

Subclassing test cases

What happens when you subclass a test case? For example:

package test;

import org.junit.Test;

public class SubOrderTest1 extends OrderTest1 {

    private void println(String string) {
        OrderTestUtils.println(SubOrderTest1.class, string);
    }

    @Test
    public void testSub1() {
        this.println(&quot;@Test testSub1()&quot;);
    }

    @Test
    public void testSub2() {
        this.println(&quot;@Test testSub1()&quot;);
    }
}

JUnit runs the given test case and then the super class test case:

SubOrderTest1 @Test testSub1()
SubOrderTest1 @Test testSub1()
OrderTest1 @Test test1()
OrderTest1 @Test test2()

If you add a third level with a SubSubOrderTest1 class extending SubOrderTest1, you get:

SubSubOrderTest1 @Test testSubSub1()
SubSubOrderTest1 @Test testSubSub1()
SubOrderTest1 @Test testSub1()
SubOrderTest1 @Test testSub1()
OrderTest1 @Test test1()
OrderTest1 @Test test2()

Method execution order starts with the given test case and goes up the super class chain until you get to the top of the hierarchy.

Subclassing test cases with managed resources

Here, things get trickier.

The order of @Before methods goes from the top to the bottom of the hierarchy: parent, child, child of child.

The order of @After methods goes from the bottom to the top of the hierarchy: child of child, child, parent.

For example:

package test;

import java.io.Closeable;
import java.io.IOException;

import org.junit.After;
import org.junit.Before;
import org.junit.Test;

public class SubOrderTest2 extends OrderTest2 {

    static class ManagedResource implements Closeable {
        @Override
        public void close() throws IOException {
        }
    }

    private ManagedResource managedResource;

    private void println(String string) {
        OrderTestUtils.println(SubOrderTest2.class, string);
    }

    @Before
    public void setUpSub() {
        this.println(&quot;@Before setUpSub&quot;);
        this.managedResource = new ManagedResource();
    }

    @After
    public void tearDownSub() throws IOException {
        this.println(&quot;@After tearDownSub&quot;);
        this.managedResource.close();
        this.managedResource = null;
    }

    @Test
    public void testSub1() {
        this.println(&quot;@Test testSub1()&quot;);
    }

    @Test
    public void testSub2() {
        this.println(&quot;@Test testSub2()&quot;);
    }
}

Here, JUnit runs the given test case and then the super class test case:

OrderTest2 @Before setUp
SubOrderTest2 @Before setUpSub
SubOrderTest2 @Test testSub1()
SubOrderTest2 @After tearDownSub
OrderTest2 @After tearDown
OrderTest2 @Before setUp
SubOrderTest2 @Before setUpSub
SubOrderTest2 @Test testSub2()
SubOrderTest2 @After tearDownSub
OrderTest2 @After tearDown
OrderTest2 @Before setUp
SubOrderTest2 @Before setUpSub
OrderTest2 @Test test1()
SubOrderTest2 @After tearDownSub
OrderTest2 @After tearDown
OrderTest2 @Before setUp
SubOrderTest2 @Before setUpSub
OrderTest2 @Test test2()
SubOrderTest2 @After tearDownSub
OrderTest2 @After tearDown

If you add a third level with a SubSubOrderTest2 class extending SubOrderTest2, you get:

OrderTest2 @Before setUp
SubOrderTest2 @Before setUpSub
SubSubOrderTest2 @Before setUpSubSub
SubSubOrderTest2 @Test testSubSub1()
SubSubOrderTest2 @After tearDownSubSub
SubOrderTest2 @After tearDownSub
OrderTest2 @After tearDown
OrderTest2 @Before setUp
SubOrderTest2 @Before setUpSub
SubSubOrderTest2 @Before setUpSubSub
SubSubOrderTest2 @Test testSubSub2()
SubSubOrderTest2 @After tearDownSubSub
SubOrderTest2 @After tearDownSub
OrderTest2 @After tearDown
OrderTest2 @Before setUp
SubOrderTest2 @Before setUpSub
SubSubOrderTest2 @Before setUpSubSub
SubOrderTest2 @Test testSub1()
SubSubOrderTest2 @After tearDownSubSub
SubOrderTest2 @After tearDownSub
OrderTest2 @After tearDown
OrderTest2 @Before setUp
SubOrderTest2 @Before setUpSub
SubSubOrderTest2 @Before setUpSubSub
SubOrderTest2 @Test testSub2()
SubSubOrderTest2 @After tearDownSubSub
SubOrderTest2 @After tearDownSub
OrderTest2 @After tearDown
OrderTest2 @Before setUp
SubOrderTest2 @Before setUpSub
SubSubOrderTest2 @Before setUpSubSub
OrderTest2 @Test test1()
SubSubOrderTest2 @After tearDownSubSub
SubOrderTest2 @After tearDownSub
OrderTest2 @After tearDown
OrderTest2 @Before setUp
SubOrderTest2 @Before setUpSub
SubSubOrderTest2 @Before setUpSubSub
OrderTest2 @Test test2()
SubSubOrderTest2 @After tearDownSubSub
SubOrderTest2 @After tearDownSub
OrderTest2 @After tearDown

I hope you found this post helpful. I was a nice reminder for me to lay it out and remember the power of JUnit rules. With the new ClassRule in JUnit 4.9, rules can finally be used full strength.

[Note this additional post on this topic.]

22 thoughts on “Understanding JUnit method order execution”

josefB February 14, 2012 at 04:50

Wonderful post! One little nitpick. You did not mention how @BeforeClass behaves when you are using subclassed tests. Or I’m dense and missed it.

This gives me more ideas for a design problem I have. How to structure JUnit tests that are running tests against different browser types. So far, the most tractable way is to write the tests in one class, then subclass for each browser type; that way the @BeforeClass can be used to create the browser specific webdriver.

LikeLike

Reply ↓
Gary Gregory Post authorFebruary 16, 2012 at 20:52

Hi Josef:

The @BeforeClass methods in super classes are run before those the current class.

But… I could not get that to work with JUnit 4.10.

LikeLike

Reply ↓
Colin August 8, 2012 at 17:00

Good post and thanks.

LikeLike

Reply ↓
Francisco Moreno (@morvader) September 21, 2012 at 05:48

Good post!.

But you have to consider that the order or the Test themselves is not guaranteed by the framework.

http://junit.sourceforge.net/doc/faq/faq.htm#tests_2

http://stackoverflow.com/a/9529014/435693

Thanks!

LikeLike

Reply ↓
test2 September 26, 2012 at 02:04

Woah this blog is fantastic i really like studying your posts. Keep up the good paintings! You know, a lot of individuals are looking around for this info, you could help them greatly.

LikeLike

Reply ↓
Kornero October 25, 2012 at 01:40

If you are interesting, you can read my solution about how to user test execution order: just as in your source code:
http://intellijava.blogspot.com/2012/05/junit-and-java-7.html

But remember that if your tests need some special ordering, it means that something goes wrong =)

LikeLike

Reply ↓
Nicolas Barbulesco April 3, 2013 at 07:51

“In OrderTest1, the execution order is:
1 OrderTest1 @Test test1()
2 OrderTest1 @Test test2()”

No. The execution order of JUnit tests in undefined, and it varies at each launch of the tests. “It’s not a bug, it’s a feature.” But it’s annoying. I would happily give up the added value of order variation, and have my list of test results in Eclipse in the order of the tests.

LikeLike

Reply ↓
1. Gary Gregory Post authorApril 3, 2013 at 08:15
  
  Right, JUnit 4.11 had made this more painful, see my update on the subject here: https://garygregory.wordpress.com/2013/01/23/understanding-junit-method-order-execution-updated-for-version-4-11/
  
  LikeLike
  
  Reply ↓
  1. Nicolas Barbulesco April 3, 2013 at 09:54
    
    Now JUnit has added an annotation which we can use to have the tests run in alphabetical order. Well, that’s an idea. But that’s not what I want. I want to have the tests run in their order of appearance in the class. Is there a way to do that ?
    
    Thank you.
    
    LikeLike
2. Kornero May 22, 2013 at 04:38
  
  In my comment you can find link with some hack, and run your tests in order of appearance in the class.
  
  LikeLike
  
  Reply ↓
Pappu June 27, 2013 at 06:22

OrderTestUtils which class does it denotes.

LikeLike

Reply ↓
Andrew Muraco August 26, 2013 at 09:41

Do you know when the various methods in the org.junit.runner.notification.RunListener interface would be fired, particularly when dealing with failures, and @after/@afterclass?

LikeLike

Reply ↓
Krishna September 10, 2013 at 21:49

Did something change in junit after 4.9 with respect to test fixture ordering. For me the super classes fixture is not run automatically unless I call them explicitly from my test’s fixtures.

LikeLike

Reply ↓
Gasan Gus October 16, 2013 at 03:46

Nice post, in section “Managing resources with rules” on second code snippet there is an error: “OrderTest4 @After setUp” should be changed to “OrderTest4 @After tearDown” twice.

LikeLike

Reply ↓
robsonpeixoto November 5, 2013 at 13:19

Can I use @AfterClass to get all tests status?

LikeLike

Reply ↓
Maks December 22, 2013 at 10:33

thnx

LikeLike

Reply ↓
Maja June 11, 2014 at 03:46

Thank you for this article! I just started writing some junit and dbunit tests and I didn’t want to couple my design by using a super class for common initialization code. I didn’t know about @ClassRule and @Rule.

LikeLike

Reply ↓
LZH April 11, 2015 at 05:48

Reblogged this on TechCentral and commented:
Great post on junit method order execution

LikeLike

Reply ↓
DB June 9, 2015 at 09:34

Excellent article that taught me how to use the Before and After annotations. Thank you.

LikeLike

Reply ↓
Pingback: How do you assert that a certain exception is thrown in JUnit 4 tests? - QuestionFocus
Pingback: How Do You Assert That A Certain Exception Is Thrown In JUnit 4 Tests? - Code Utility
Pingback: How do you assert that a certain exception is thrown in JUnit tests?