Some of you may have noticed that I love Grails. You may also have noticed that I bitch about Grails bugs a lot too. You always hurt the ones you love.
Well I was chatting with Graeme the other day as I do like to moan about all the Grails bugs I find. I often joke that I seem to be a magnet for them.
Here’s the deal. Grails stands on the shoulders of giants: Spring, Hibernate, Groovy etc. These techs are really powerful but also complex. That is of course the beauty of Grails – it makes using these techs together so simple.
But here’s the rub. Both the Grails code and those libraries it uses will always have bugs, and sadly regressions are also inevitable. For me regressions are the real enemy – bugs in new features are not so bad, you can just make do without the new feature like you used to.
Why do regressions happen? Because there was no regression test for that code – test code coverage was too low for that use case.
Why aren’t there tests for the regression you just found? Because humans can’t second guess the complex high-level interactions that users of Grails create when they develop with it.
The Grails dev team cannot also write lots of complex applications to be used purely as test cases. There are more and more functional tests, but for true full stack regression testing you need to run new releases of Grails through proven non-trivial real-world applications.
Grails CI server already builds some test applications and runs their functional tests against Grails HEAD. Ideally I would love for some large open source / internal use applications to be “donated” (with high test coverage of their own!) to be run privately on Grails CI – a partnership that would involve developer assistance to the Grails team if the tests start failing.
However, back to the simpler solution for now.
Obviously I’m very passionate about quality of software. That’s why I’m going to beg everybody to download Grails 1.2M1 and run it with their existing apps and plugins locally, especially running all your own tests (you have lots of those, right?).
Then anything that has broken MUST be reported immediately and JIRA’d. It is either a – probably unintentional – breaking change or a regression that is not covered by a test.
Doing this and reporting any problems quickly is so important I cannot stress it enough.
Otherwise what happens is, come 1.2 release, you may suddenly find you can’t use it because of regressions and then we end up in a situation where nobody trusts the “first release” of any new version and you always have to wait for point releases to fix regressions.
That is no way to build confidence in the quality of a product.
Graeme et al gave us Grails to help us do our day jobs. Let’s make life easier for all of us and give back a little test time.
You’d only waste that time later when you try to use a new build and hit problems – and as seasoned developers know, those kinds of things often happen on important product launch days. Ouch.
Trust me – I’m groaning about it too. I have half a dozen plugins to test (with not enough test coverage in them) but if I don’t do it now I’ll only have to firefight it later when 1.2 comes out.
It is a great motivation to increase your own test coverage
Update: PS I forgot to mention. Sometimes, as is true currently, there are some unfixed bugs in underlying libraries like Hibernate. Sometimes the Grails Dev team can’t work around these – so once these problems are found in Grails they are then reported against the underlying library and we need users to request that the maintainers of that code fix the problem. Oh, even better users can submit patches to those maintainers. It’s Open Source after all.