Grails 1.2M1 and why you need to download it NOW

Posted by: on Jul 3, 2009 | No Comments

Some of you may have noticed that I love Grails. You may also have noticed that I bitch about Grails bugs a lot too. You always hurt the ones you love.

Well I was chatting with Graeme the other day as I do like to moan about all the Grails bugs I find. I often joke that I seem to be a magnet for them.

Here’s the deal. Grails stands on the shoulders of giants: Spring, Hibernate, Groovy etc. These techs are really powerful but also complex. That is of course the beauty of Grails – it makes using these techs together so simple.

But here’s the rub. Both the Grails code and those libraries it uses will always have bugs, and sadly regressions are also inevitable. For me regressions are the real enemy – bugs in new features are not so bad, you can just make do without the new feature like you used to.

Why do regressions happen? Because there was no regression test for that code – test code coverage was too low for that use case.

Why aren’t there tests for the regression you just found? Because humans can’t second guess the complex high-level interactions that users of Grails create when they develop with it.

The Grails dev team cannot also write lots of complex applications to be used purely as test cases. There are more and more functional tests, but for true full stack regression testing you need to run new releases of Grails through proven non-trivial real-world applications.

Grails CI server already builds some test applications and runs their functional tests against Grails HEAD. Ideally I would love for some large open source / internal use applications to be “donated” (with high test coverage of their own!) to be run privately on Grails CI – a partnership that would involve developer assistance to the Grails team if the tests start failing.

However, back to the simpler solution for now.

Obviously I’m very passionate about quality of software. That’s why I’m going to beg everybody to download Grails 1.2M1 and run it with their existing apps and plugins locally, especially running all your own tests (you have lots of those, right?).

Then anything that has broken MUST be reported immediately and JIRA’d. It is either a – probably unintentional – breaking change or a regression that is not covered by a test.

Doing this and reporting any problems quickly is so important I cannot stress it enough.

Otherwise what happens is, come 1.2 release, you may suddenly find you can’t use it because of regressions and then we end up in a situation where nobody trusts the “first release” of any new version and you always have to wait for point releases to fix regressions.

That is no way to build confidence in the quality of a product.

Graeme et al gave us Grails to help us do our day jobs. Let’s make life easier for all of us and give back a little test time.

You’d only waste that time later when you try to use a new build and hit problems – and as seasoned developers know, those kinds of things often happen on important product launch days. Ouch.

Trust me – I’m groaning about it too. I have half a dozen plugins to test (with not enough test coverage in them) but if I don’t do it now I’ll only have to firefight it later when 1.2 comes out.

It is a great motivation to increase your own test coverage

Update: PS I forgot to mention. Sometimes, as is true currently, there are some unfixed bugs in underlying libraries like Hibernate. Sometimes the Grails Dev team can’t work around these – so once these problems are found in Grails they are then reported against the underlying library and we need users to request that the maintainers of that code fix the problem. Oh, even better users can submit patches to those maintainers. It’s Open Source after all.

Why G-Func and not Webtest?

Posted by: on Jan 18, 2009 | No Comments

Some people are no doubt wondering why we have the new Grails Functional Testing plugin when we already have Webtest. There are many reasons (Webtest uses Ant to build being one of them) but for me it is primarily the differences in the DSL approach.

Webtest has an established heritage in XML-based test definition. The Grails webtest plugin is a Groovy wrapper around this grammar.

In “G-Func” the philosophy is very much that your test code should “just work” without you having to worry too much about details of the HTML pages returned. For example elements in forms are located by name OR id automatically. I have started putting together a “codex” showing how tasks are achieved in both G-Func and Webtest as a means of comparison and learning.

Another example: buttons are located by name OR id OR value – some forms still have <input type=”submit” value=”Submit”/> and we need to be able to click those in a test. We don’t need to know in the test which approach the web designer has used in the GSP. If we encode knowledge about the specific mechanism of identifying the button, we are making the test more fragile, for little benefit.

G-Func is also smart about field types in forms and “does the right thing”. Whether you’re getting or setting a hidden field, a text area, a select list, a checkbox or a radio button – the mechanism is the same, standard property assignment or method invocation:

form('form1') {
   checkbox1 = true
   checkbox2 true
   optionRadioButton = "OptionC"
   paymentMethodRadioButton "VISA"
   country = "gbr"
   destinationCountry "usa"
   firstName = "Marc"
   lastName = "Palmer"
   click "send"
}

Another strand to this is that many of the assertXXXX variants operate with “loose” comparison by default. This means that case and whitespace are all stripped out before comparing values. This makes it much easier to assert content without getting tripped up by subtle changes in whitespace or case. After all, you rarely need to know the case and whitespace is correct in some html response – its the characters that matter.

For the perfectionists, if you do need to be 100% sure, you use the assertXXXXStrict variant.

Other differences between G-Func and WebTest:

  • G-Func integrates with normal Grails testing conventions. Tests are under ./test/, so are the reports.
  • The tests are JUnit tests and use the familiar testMethod() with assertXXX calls paradigm
  • The G-Func DSL is less verbose than WebTest (see the codex). Setting/getting form fields is simple property access. What really matters is the data getting through, not what kind of field it was in!
  • The DSL, when it does use verbs, is truly specific to the domain. You use get() and post() instead of invoke(method:’GET’ …) and so on.
  • G-Func documentation is all written with direct relevance to the DSL. While WebTest is fully featured, it has documentation primarily based on the XML usage. This makes sense of course, given its heritage. However it makes the learning curve and doc lookups harder work for those working in Groovy
  • G-Func is entirely within the Grails ecosystem – it uses Gant (not Ant) and is a thing pure-Groovy layer over HtmlUnit.
  • Webtest definitely has more functionality and slightly prettier reports than G-Func at this time. However the G-Func reports are consistent with JUnit reports, and the features that users need will be incorporated into G-Func.

Anyway, I hope that clears things up for some people.

If you do have any questions, please do ask on the Grails user mailing list or get in touch with me.

Functional testing in Grails just got a bit sexier

Posted by: on Jan 8, 2009 | 24 Comments

Ok so here’s the 1.0 release in the spirit of “don’t worry be crappy” (can’t use that phrase enough these days) of a new functional testing plugin for Grails.

Actually its far from crappy already.

I have developed this partly as part of my work for my generous client Historic Futures Ltd. who agreed to open source this.

I plan to continue improving and maintaining it personally. It’s surely a bit rough around the edges but the benefits should be pretty obvious to those who use it.

So what are you waiting for? Run:

grails install-plugin functional-test

See the documentation

…but in a nutshell it is HTTP testing with HtmlUnit under the hood but the rest is pure groovy smarts, leveraging standard JUnit.

Example:

import functionaltestplugin.*
class TwitterTests extends FunctionalTestCase {
  void testSearch() {
    get('http://www.twitter.com')

    click "Search"

    assertStatus 200
    assertContentContains "search"

    form('searchForm') {
      q = "#grails"
      click "Search"
    }

    assertStatus 200
    assertContentContains "#grails"
  }
}

What’s special about this?

  1. Simplicity and “elasticity” by default = less fragile functional tests as a result
  2. Simple compact DSL/methods to learn – it tries to “do the right thing”. Let’s face it writing tests is really boring.
  3. Ability to get/post directly without browsing to a page first. eg REST testing
  4. No .properties configs

…and one more thing URL stacktrace. When a test fails the report includes a stack of the URLs at the point of failure. The xxx-out.txt file contains request parameters+headers, content received etc. This feature will be expanded in future to allow a proper request/response chain autopsy.

Caveats – things I definitely want to sort out pronto:

  • currently test output is a bit ropey, I would like to do much nicer stuff with custom output xml and rendering
  • no simple way to set post body
  • no simple way to parse out json/xml responses yet

All this and more will come, with your support! Contribs also welcome!

ENJOY!