Closure property conventions for events = anti-pattern

Posted by: on Feb 9, 2009 | One Comment

Having had an interesting time recently playing with flows and GORM events in a couple of Grails applications, as well as looking at some plugins like the Audit Logging plugin, I have some views to share on the use of Closure property conventions.

Background:

In Grails usually the first time you encounter the use of closures as property values is when you are defining actions in a controller, or constraints on a domain class.

This is a pretty good mechanism. It has some problems – regarding inheritance for example – but these are mainly for the “back end” framework code. It affords the framework more flexibility in the way it provide dynamic methods and properties, rather than just using the metaClass of the controller to do this.

The problems:

The difficulties with this approach start to show up with things like GORM events. If you try using domain classes with events in flows you start to discover all kinds of issues that are side effects of using closure properties instead of methods. In summary;

  1. Inheritance is broken. In a closure property you can’t call “super” – so unless the framework provides a mechanism for it, the closure code cannot call the closure from the ancestor class. Frameworks have to also deal with this problem in terms of gathering up all the closures in the hierarchy and merging the results of them eg constraints in GORM – a special case where the code is DSL. This is a problem for more general code eg event handlers.
  2. Serialization problems. The programmer will find out at some point that closure properties should not be serialized. This means marking the properties “transient”. Except that you then find that if you do this, after loading from serialized form, the event property is then null. So actually you have to flag it as “static” and not “transient” to get around this – which restricts what the closure can do. For GORM you also have to explicitly add these properties to the “transients” convention property to stop Hibernate trying to persist them. If you have plugins that expect other closure properties then you also have to mark these as static and in the transients list also. No matter that Groovy 1.6.x will support Closure serialization. You should never be serializing event code in normal applications.
  3. Without reading docs you don’t know what “delegate” is in such a closure, and “delegate” does not read as well as “target” or “this” etc

So what I’m getting at is, I now strongly believe that using closure properties for “events” on objects is generally bad news. The best solution is good old fashioned virtual methods. These support “super”, have no inheritance issues, and no serialization issues.

For example the Audit Logging plugin uses closure properties for events on domain classes, and when evaluating this plugin I immediately thought “Hmmm will it be handling inherited handlers properly or not?” and of course there’s no info in the docs, and there would be code overhead in the plugin to support this – eg instantiating once (nonsensically) every domain class to evaluate its convention. This is something that Grails did (or still does?) do for services which causes all kinds of weirdness if you expect your service to only instantiate once in an application.

Furthermore in GORM, are inherited beforeXXX events on GORM classes called if you define a new one in a descendent? These are the kinds of questions that crop up. You readily assume they will, but you don’t know until you try.

I’m hoping that in Grails 1.1 or soon after the beforeXXX GORM events will transition to methods to avoid all the problems relating to (1) and (2) in the above list.

I’m also hoping plugin developers will stop using closures for event handlers. I’ve done it myself – in the Authentication plugin – but here it is a different case. The events are stored in a singleton map and are meant to be mutable and set by the application. There are no inheritance issues, nor serialization gotchas.

Death to per-instance closure properties for event handling!

An idea to give Grails tags “E.S.P.”

Posted by: on Jan 19, 2009 | 5 Comments

There exists a problem with all current JS UI plugins in that they produce HTML content that is peppered with inline Javascript and/or CSS. This is widely acknowledged as bad practice and greatly increases page load times.

This can be reduced to a problem of the inability for custom tags to define the contents of JS and CSS used by the current GSP page. This can be generalised to and inability for a tag invocation in a page to contribute to the production of any related content.

There is also a separate problem of taglibs invoked within a page body being unable to influence the content of the <head> section of an HTML response. This problem could be solved by introducing a two-phase GSP page execution, but most likely this should be handled by external resources, eg .js and .css files added to te <head> section via <link> tags.

The solution is quite simple, but with some subtleties to make it truly rock.

In a nutshell, we use some generic taglib functionality to gather information about what JS and CSS (or other resource) information is to be gathered and served for the current page, and have a controller with a cache to serve it up.

What we’re doing here is exploiting the fact that the possible race condition of CSS+JS being loaded before the HTML is full loaded by the browser, is not going to happen to pages laid out by SiteMesh, and even for other pages will only be a problem for the first request (see workarounds for this later).

Introducing the concept of “The E.S.P. Plugin”

This plugin would provide a taglib and a controller, and either another tag, or a service (See below).

The taglib would offer:

<esp:resource group="somepage" type="css"/>

This tag is used in the <head> section of layouts or pages, to indicate that you want to pull in a resource. The group attribute is optional. If there is none the group will default to “${controllerName}.${actionName}”. This is how you control where resources are served up. You can have a resource per page, or you can be smarter and have them grouped in any combinations you like, eg a single CSS file but separate JS files. Some pages may even have more than one such inclusion:

<html>
<head>
  <esp:resource group="common" type="css"/>
  <esp:resource group="common" type="js"/>
  <esp:resource group="contactForm" type="js"/>
</head>
...
</html>

This tag simple outputs a <link> tag with the mimetype + relation set according to the “type” value, and a link to the ESPController action with params “group” and “type” and “template” – if there was a “template” attribute specified. There could be a URL mapping to prettify this to something like:

/esp/contactForm/js

Then we have another tag or service to store strings against the group and type, and to detect if such data has already been registered (to prevent tag slow-down for every request).

For example as a tag called as a method from UI taglibs:

if (!esp.has('contactForm', 'js',
       "grailsui.richtext.${attrs.id}")) {
    esp.store('contactForm', 'js', 
       "grailsui.richtext.${attrs.id}") << "some text"
}

Whether this is a tag or service makes little difference other than convenience. The data would ultimately need to be stored in a cache in a service – the value returned from the store is a StringWriter that can be put in a local var and reused across multiple “<<” invocations.

The page-unique id is required to prevent the same content being cached more than once for that group.

Order of data stored must be preserved so they are written out in the same order.

Tags will use this form for writing out their JS or CSS, while they simultaneously – in most cases – also write out some vanilla and entirely unobtrusive HTML.

NOTE: There should be a form of store() that does not take the “group” parameter, only type and id. This would default to the last ‘group’ value used in the esp:resource tags in <head> (which would be stored as a request attribute).

Next, the ESPController:

This controller would work by retrieving from the internal datastructure all the strings registered for the specified group and resource type, and then:

1) if there is no “template” attribute, simply render them out with a default content type based on the “type” stored

2) if there is a “template” attribute, it will render the GSP template and put all the strings into a list in the model that can be iterated in the GSP – so that you can have a base template and do any decorating you want

3) Set appropriate caching headers (probably subject to Config.groovy settings). Although this is not intended to be a substitute for high performance front end caching of the resources.

4) If there is a request for a group and resource type combination for which there is currently no data, we presume this is due to a request race condition or a first request with an eager browser so it either:

i) returns a “Server busy” or similar status code

or

ii) uses a filter to know when such a request has first been served (checking request attribute for groups and types used in the page), and have the /esp/ request for same groups+type “sleep” until the first request is complete.

or

iii) use a <esp:done/> tag used at the end of the GSP/layout such that this sets a flag in the global data indicating that requests can now be served for the groups+types used in the page.

Hopefully this makes sense. In essence it is quite simple.

It also gives us a new feature – the ability to define “static” data used across different GSP pages and resources, and text fragments for reuse without <g:render>. All it needs is an extra <esp:get> tag to get the content.

I have to say that I prefer option (i). Frankly I couldn’t give a damn about request zero

Why G-Func and not Webtest?

Posted by: on Jan 18, 2009 | No Comments

Some people are no doubt wondering why we have the new Grails Functional Testing plugin when we already have Webtest. There are many reasons (Webtest uses Ant to build being one of them) but for me it is primarily the differences in the DSL approach.

Webtest has an established heritage in XML-based test definition. The Grails webtest plugin is a Groovy wrapper around this grammar.

In “G-Func” the philosophy is very much that your test code should “just work” without you having to worry too much about details of the HTML pages returned. For example elements in forms are located by name OR id automatically. I have started putting together a “codex” showing how tasks are achieved in both G-Func and Webtest as a means of comparison and learning.

Another example: buttons are located by name OR id OR value – some forms still have <input type=”submit” value=”Submit”/> and we need to be able to click those in a test. We don’t need to know in the test which approach the web designer has used in the GSP. If we encode knowledge about the specific mechanism of identifying the button, we are making the test more fragile, for little benefit.

G-Func is also smart about field types in forms and “does the right thing”. Whether you’re getting or setting a hidden field, a text area, a select list, a checkbox or a radio button – the mechanism is the same, standard property assignment or method invocation:

form('form1') {
   checkbox1 = true
   checkbox2 true
   optionRadioButton = "OptionC"
   paymentMethodRadioButton "VISA"
   country = "gbr"
   destinationCountry "usa"
   firstName = "Marc"
   lastName = "Palmer"
   click "send"
}

Another strand to this is that many of the assertXXXX variants operate with “loose” comparison by default. This means that case and whitespace are all stripped out before comparing values. This makes it much easier to assert content without getting tripped up by subtle changes in whitespace or case. After all, you rarely need to know the case and whitespace is correct in some html response – its the characters that matter.

For the perfectionists, if you do need to be 100% sure, you use the assertXXXXStrict variant.

Other differences between G-Func and WebTest:

  • G-Func integrates with normal Grails testing conventions. Tests are under ./test/, so are the reports.
  • The tests are JUnit tests and use the familiar testMethod() with assertXXX calls paradigm
  • The G-Func DSL is less verbose than WebTest (see the codex). Setting/getting form fields is simple property access. What really matters is the data getting through, not what kind of field it was in!
  • The DSL, when it does use verbs, is truly specific to the domain. You use get() and post() instead of invoke(method:’GET’ …) and so on.
  • G-Func documentation is all written with direct relevance to the DSL. While WebTest is fully featured, it has documentation primarily based on the XML usage. This makes sense of course, given its heritage. However it makes the learning curve and doc lookups harder work for those working in Groovy
  • G-Func is entirely within the Grails ecosystem – it uses Gant (not Ant) and is a thing pure-Groovy layer over HtmlUnit.
  • Webtest definitely has more functionality and slightly prettier reports than G-Func at this time. However the G-Func reports are consistent with JUnit reports, and the features that users need will be incorporated into G-Func.

Anyway, I hope that clears things up for some people.

If you do have any questions, please do ask on the Grails user mailing list or get in touch with me.

Groovy, let me count the ways in which I love thee

Posted by: on Jan 16, 2009 | 6 Comments

Oh the wonder and the joy. I’ve been fixing some issues and adding features to the Grails Functional Testing plugin (now informally known as The G-Func) and had cause to marvel at some Groovy treats.

The problem: HtmlUnit, which the plugin uses for HTTP interactions, has an extensive hierarchy of classes representing page and form elements. There is however no way to indentify some of these as specific functional groups eg there are 4 different button types, which have no interface or ancestor that is not also shared by some other non-button classes.

So a little ugly logic is needed. For example when the plugin’s form wrapper uses Groovy dynamic property resolution to get a form field for you, it is now doing something like this:

private getFieldValue(f) {
  // Note this switch is polymorphic, order of cases is IMPORTANT
  switch (f.class) {
    case BUTTON_CLASSES:
    case HtmlFileInput.class:
      return f // return the field object itself, not its value
    case HtmlSelect.class:
      return f.selected
    case HtmlRadioButtonInput.class:
      return f.checked
    case HtmlCheckBoxInput.class:
      return f.checked
    case HtmlTextArea.class:
    case HtmlInput.class:
      return f.value
    default:
      throw new RuntimeException("Don't know how to get a value from form element of type [${f.class}]")
  }
}

Now, what is special about this is:

  1. Groovy’s switch statement is very cool. It calls isCase(value) on the objects used in the case statements. That’s pretty basic Groovy stuff, but is still news to most Java developers.
  2. The use of BUTTON_CLASSES in the case is very cool. That property is a list of HtmlUnit classes that are “buttons” in conceptual terms. Groovy will call isCase on the list, which will return true if the list CONTAINS the switch value. That saved me 4 case lines and made my code more DRY as this check is also done elsewhere.
  3. The isCase implementation on Class added by the Groovy GDK is polymorphic. So the “case HtmlInput.class” actually catches any descendent of HtmlInput – of which there are several. This is also why the case BUTTON_CLASSES is at the top of the switch, as some of those classes extend HtmlInput and would be incorrectly caught by this HtmlInput clause.

Many people will no doubt say I can simplify the code above further – the “.class” are largely not necessary and I could merge the “return f.checked” cases. I may well do in future, but this is there for clarity at the moment

We already knew those Spring guys were smart…

Posted by: on Nov 18, 2008 | No Comments

Well it came out of the blue, but it really was a nice surprise to hear that SpringSource (formerly Interface21) have bought the Groovy and Grails company (G2One).

It seems only fitting that the people behind the best web/EE solutions for Java end up backing the best rapid web development platform for the Java VM, namely Grails.

Congratulations to the team at G2One for brokering this deal. I am very excited about the opportunities this gives Grails to make important inroads into all those “serious” Java development shops who, to date, have felt that Grails/Groovy are “too out there” for them. They couldn’t be more wrong.

You know this to be true when you’ve had a meeting with a manager about some sorely needed features, and within an hour or so of the end of the meeting you’ve got the code working, and as a result some very happy people. Nothing even comes close to Grails in the Java realm.

In these financially difficult times there’s never been a better reason to choose a massively productive framework like Grails – get twice as much (or more) out of your staff, for no extra cost!