Unit tests – now they work, now they don’t

Writing unit tests is often quite tricky. However even simple ones can cause problems.

I’ve been merging in my GSP whitespace handling changes into Grails 1.1 and found that for me a bunch of unrelated tests were failing – but the Bamboo CI build and presumably the builds on other developers’ machines were not failing on these.

In a nutshell… beware assumptions of hash key ordering. This is pretty basic stuff but it is very easy to overlook, especially when they work when you run them.

Don’t write test code that asserts the toString() of any object that contains unordered Hash objects, or uses keySet()s from them. For example this test renders some objects to JSON, and the order of the properties of each object cannot be guaranteed:

void testConvertErrors() {
  def c = ga.getControllerClass("RestController").newInstance()
  c.testErrors()
  // @todo this test is fragile and depends
  // on runtime environment because
  // of hash key ordering variations
  assertEquals(
     '{"errors":[{"object":"Book","field":"title",'+
     '"rejected-value":null,"message":'+
     '"Property [title] cannot be null" },'+
     '{"object":"Book","field":"author","rejected-value":null,'+
     '"message":'+
     '"Property [author] cannot be null"}]}',
     response.contentAsString)
}

In that case the solution is a bit more tricky. It is arguably better to test for substrings in the response, rather than parse out the whole JSON again and check the data – after all then you’re also unit testing the parsing code too.

Also don’t assume that an iterator/enumerator you get contains just the element you want:

assertEquals "foo", pageContext.getAttributeNamesInScope(
   PageContext.PAGE_SCOPE).nextElement()

If you need to test for the presence of an object in something that is a list or collection, unless part of your test contract is to ensure it is the -only- entry.