Assume vs. Assert
Assumptions are an underused feature of modern testing frameworks that should be more widely known. Briefly, an assumption verifies that conditions are right to execute the test. An assertion verifies that the test passes. If an assertion fails, the test failed, and we know the code is broken. If an assumption fails, the test was not run and the code may or may not be broken.
In JUnit 5 assumptions live in the Assumptions class, which has an API that parallels the Assertions class with s/assert/assume. For instance, instead of Assertions.assertEquals(), there is Assumptions.assumeEquals(). Instead of Assertions.assertNotNull(), there is Assumptions.assumeNotNull(), and so forth. Thus the API is very familiar and easy to use.
For example, here’s a test for the Maven enforcer plugin. It creates a file and then immediately deletes it using java.io.File.delete(). Deletion isn’t a perfectly reliable operation so we check the return value of the delete method with assumeTrue before continuing. If the file isn’t deleted, the test halts.
void testFileDoesNotExist() throws IOException {
File f = File.createTempFile("junit", null, temporaryFolder);
Assumptions.assumeTrue(f.delete());
rule.setFilesList(Collections.singletonList(f));
try {
rule.execute();
Assertions.fail("Should get exception");
} catch (EnforcerRuleException e) {
Assertions.assertEquals(f.getPath() + " does not exist.", e.getMessage());
}
}After the assumption is validated, then we execute the code under test. In this example that code is expected to throw an exception so if it doesn’t, we call Assertions.fail(). If it does correctly throw the exception, then the test compares the expected exception message to the actual exception message with Assertions.assertEquals(). The general pattern is check the setup code with assumptions. Check the code under test with assertions.
If you’re still using JUnit 4, the general principle are the same. However you’ll be using org.junit.Assert and org.junit.Assume instead of org.junit.jupiter.api.Assertions and org.junit.jupiter.api.Assumptions. For example:
void testFileDoesNotExist() throws IOException {
File f = File.createTempFile("junit", null, temporaryFolder);
Assume.assumeTrue(f.delete());
rule.setFilesList(Collections.singletonList(f));
try {
rule.execute();
Assert.fail("Should get exception");
} catch (EnforcerRuleException e) {
Assert.assertEquals(f.getPath() + " does not exist.", e.getMessage());
}
}JUnit 3 and earlier don’t distinguish between assertions and assumptions, which is a good reason to upgrade to at least JUnit 4.
When should you use which? Generally assertions verify the model code you’re testing.
Assumptions verify the expected behavior of methods outside your own project, typically a core library method or a third party library method.
For instance, imagine you’re writing testParseInvoice() that tests the parseInvoice() method. parseInvoice() is supposed to read a JSON file and construct an Invoice object from that data. The test might first write JSON data into an invoice.json file that the test can read from. If this code fails, it tells us nothing about whether the parseInvoice() method is correct. The parseInvoice() method was never run. You should verify this with an assumption rather than an assertion.
Good test runners distinguish between assumption violations and assertion failures. Assumption violations are marked as skipped rather than failed. This might not indicate any problem at all. For example, sometimes you might write a test that only applies on Windows. You can write that like this:
Assumptions.assumeTrue(System.getProperty("os.name").contains("Windows"))
Maybe it tests code that only executes on Windows, maybe it tests for idiosyncrasies of the Windows file system, or maybe it’s a case where the expected behavior is different between Windows and Unix. Whatever the reason, if a Unix CI runs the test, it doesn’t pass and it doesn’t fail. Marking it as either one would be incorrect. Instead the tests simply didn’t run. It was skipped. That’s what assumptions enable.
