Excluding files from a WAR with Grails – the right way

Posted by: on Jan 21, 2009 | 12 Comments

I have seen people try to remove JARs shipped with Grails from their WAR by completely redefining their list of grails dependencies in Config.groovy. This is understandable because on the surface there seems to be no way to “exclude” just one or two JARs as in Config.groovy you do not have access to the DEFAULT_DEPS list.

However, I had cause to do this last night to make a new Mor.ph deployment plugin that actually works, and I knew there was a better way.

If you have an application that needs to exclude one or more resources, you can do this in Config.groovy just by supplying a closure that uses Ant calls to delete the resources, in the grails.war.resources property:

// Remove the JDBC jar before the war is bundled
grails.war.resources = { stagingDir ->
  delete(file:"${stagingDir}/WEB-INF/lib/jdbc2_0-stdext.jar")
}

However if you are writing a plugin, this option is not available to you. Never fear – Grails scripting events to the rescue.

All you need to do is add a _Events.groovy file to your plugin in the ./scripts folder, and add code like this:

// Remove the JDBC jar before the war is bundled
eventWarStart = { warName ->
    if (grailsEnv == "production") {
        println "Removing JDBC 2 Extensions JAR"
        Ant.delete(
           file:"${stagingDir}/WEB-INF/lib/jdbc2_0-stdext.jar")
    }
}

This WarStart event is called after the whole staging area has been prepared, just before zipping it up into a .war. So you can do whatever you like to the tree using Ant!

I never thought I’d enjoy the start of a war…

An idea to give Grails tags “E.S.P.”

Posted by: on Jan 19, 2009 | 5 Comments

There exists a problem with all current JS UI plugins in that they produce HTML content that is peppered with inline Javascript and/or CSS. This is widely acknowledged as bad practice and greatly increases page load times.

This can be reduced to a problem of the inability for custom tags to define the contents of JS and CSS used by the current GSP page. This can be generalised to and inability for a tag invocation in a page to contribute to the production of any related content.

There is also a separate problem of taglibs invoked within a page body being unable to influence the content of the <head> section of an HTML response. This problem could be solved by introducing a two-phase GSP page execution, but most likely this should be handled by external resources, eg .js and .css files added to te <head> section via <link> tags.

The solution is quite simple, but with some subtleties to make it truly rock.

In a nutshell, we use some generic taglib functionality to gather information about what JS and CSS (or other resource) information is to be gathered and served for the current page, and have a controller with a cache to serve it up.

What we’re doing here is exploiting the fact that the possible race condition of CSS+JS being loaded before the HTML is full loaded by the browser, is not going to happen to pages laid out by SiteMesh, and even for other pages will only be a problem for the first request (see workarounds for this later).

Introducing the concept of “The E.S.P. Plugin”

This plugin would provide a taglib and a controller, and either another tag, or a service (See below).

The taglib would offer:

<esp:resource group="somepage" type="css"/>

This tag is used in the <head> section of layouts or pages, to indicate that you want to pull in a resource. The group attribute is optional. If there is none the group will default to “${controllerName}.${actionName}”. This is how you control where resources are served up. You can have a resource per page, or you can be smarter and have them grouped in any combinations you like, eg a single CSS file but separate JS files. Some pages may even have more than one such inclusion:

<html>
<head>
  <esp:resource group="common" type="css"/>
  <esp:resource group="common" type="js"/>
  <esp:resource group="contactForm" type="js"/>
</head>
...
</html>

This tag simple outputs a <link> tag with the mimetype + relation set according to the “type” value, and a link to the ESPController action with params “group” and “type” and “template” – if there was a “template” attribute specified. There could be a URL mapping to prettify this to something like:

/esp/contactForm/js

Then we have another tag or service to store strings against the group and type, and to detect if such data has already been registered (to prevent tag slow-down for every request).

For example as a tag called as a method from UI taglibs:

if (!esp.has('contactForm', 'js',
       "grailsui.richtext.${attrs.id}")) {
    esp.store('contactForm', 'js',
       "grailsui.richtext.${attrs.id}") << "some text"
}

Whether this is a tag or service makes little difference other than convenience. The data would ultimately need to be stored in a cache in a service – the value returned from the store is a StringWriter that can be put in a local var and reused across multiple “<<” invocations.

The page-unique id is required to prevent the same content being cached more than once for that group.

Order of data stored must be preserved so they are written out in the same order.

Tags will use this form for writing out their JS or CSS, while they simultaneously – in most cases – also write out some vanilla and entirely unobtrusive HTML.

NOTE: There should be a form of store() that does not take the “group” parameter, only type and id. This would default to the last ‘group’ value used in the esp:resource tags in <head> (which would be stored as a request attribute).

Next, the ESPController:

This controller would work by retrieving from the internal datastructure all the strings registered for the specified group and resource type, and then:

1) if there is no “template” attribute, simply render them out with a default content type based on the “type” stored

2) if there is a “template” attribute, it will render the GSP template and put all the strings into a list in the model that can be iterated in the GSP – so that you can have a base template and do any decorating you want

3) Set appropriate caching headers (probably subject to Config.groovy settings). Although this is not intended to be a substitute for high performance front end caching of the resources.

4) If there is a request for a group and resource type combination for which there is currently no data, we presume this is due to a request race condition or a first request with an eager browser so it either:

i) returns a “Server busy” or similar status code

or

ii) uses a filter to know when such a request has first been served (checking request attribute for groups and types used in the page), and have the /esp/ request for same groups+type “sleep” until the first request is complete.

or

iii) use a <esp:done/> tag used at the end of the GSP/layout such that this sets a flag in the global data indicating that requests can now be served for the groups+types used in the page.

Hopefully this makes sense. In essence it is quite simple.

It also gives us a new feature – the ability to define “static” data used across different GSP pages and resources, and text fragments for reuse without <g:render>. All it needs is an extra <esp:get> tag to get the content.

I have to say that I prefer option (i). Frankly I couldn’t give a damn about request zero

Why G-Func and not Webtest?

Posted by: on Jan 18, 2009 | No Comments

Some people are no doubt wondering why we have the new Grails Functional Testing plugin when we already have Webtest. There are many reasons (Webtest uses Ant to build being one of them) but for me it is primarily the differences in the DSL approach.

Webtest has an established heritage in XML-based test definition. The Grails webtest plugin is a Groovy wrapper around this grammar.

In “G-Func” the philosophy is very much that your test code should “just work” without you having to worry too much about details of the HTML pages returned. For example elements in forms are located by name OR id automatically. I have started putting together a “codex” showing how tasks are achieved in both G-Func and Webtest as a means of comparison and learning.

Another example: buttons are located by name OR id OR value – some forms still have <input type=”submit” value=”Submit”/> and we need to be able to click those in a test. We don’t need to know in the test which approach the web designer has used in the GSP. If we encode knowledge about the specific mechanism of identifying the button, we are making the test more fragile, for little benefit.

G-Func is also smart about field types in forms and “does the right thing”. Whether you’re getting or setting a hidden field, a text area, a select list, a checkbox or a radio button – the mechanism is the same, standard property assignment or method invocation:

form('form1') {
   checkbox1 = true
   checkbox2 true
   optionRadioButton = "OptionC"
   paymentMethodRadioButton "VISA"
   country = "gbr"
   destinationCountry "usa"
   firstName = "Marc"
   lastName = "Palmer"
   click "send"
}

Another strand to this is that many of the assertXXXX variants operate with “loose” comparison by default. This means that case and whitespace are all stripped out before comparing values. This makes it much easier to assert content without getting tripped up by subtle changes in whitespace or case. After all, you rarely need to know the case and whitespace is correct in some html response – its the characters that matter.

For the perfectionists, if you do need to be 100% sure, you use the assertXXXXStrict variant.

Other differences between G-Func and WebTest:

  • G-Func integrates with normal Grails testing conventions. Tests are under ./test/, so are the reports.
  • The tests are JUnit tests and use the familiar testMethod() with assertXXX calls paradigm
  • The G-Func DSL is less verbose than WebTest (see the codex). Setting/getting form fields is simple property access. What really matters is the data getting through, not what kind of field it was in!
  • The DSL, when it does use verbs, is truly specific to the domain. You use get() and post() instead of invoke(method:’GET’ …) and so on.
  • G-Func documentation is all written with direct relevance to the DSL. While WebTest is fully featured, it has documentation primarily based on the XML usage. This makes sense of course, given its heritage. However it makes the learning curve and doc lookups harder work for those working in Groovy
  • G-Func is entirely within the Grails ecosystem – it uses Gant (not Ant) and is a thing pure-Groovy layer over HtmlUnit.
  • Webtest definitely has more functionality and slightly prettier reports than G-Func at this time. However the G-Func reports are consistent with JUnit reports, and the features that users need will be incorporated into G-Func.

Anyway, I hope that clears things up for some people.

If you do have any questions, please do ask on the Grails user mailing list or get in touch with me.

G-Func Grails Functional Testing Plugin 1.2

Posted by: on Jan 17, 2009 | No Comments

Not much more than a week has passed, and we’re onto 1.2 already. 1.1 had excessive debug output I left in by mistake…

It is in the plugin repo now, and “grails install-plugin functional-test” should work.

The docs at http://grails.org/Grails+Functional+Testing have been updated – there were several errata but there are also new features and one breaking change. I know, we started at 1.0 but … its a good change and few people will be affected at this point

New are… Javascript on/off, Grails 1.1 compatibility, text area support, cookie support, proper redirect handling (some limitations currently), and lots of output improvements and fixes.

BREAKING CHANGE:

Setting the value of selects, checkboxes and radio button groups is now just a simple property assignment. So where before you had:

form(‘myform’) {
radioOptions.checked = “option3″
}

You now just have:

form(‘myform’) {
radioOptions= “option3″
}

Which is more consistent with other field access.

Also the test output files contain useful information about followed redirects and values set and items clicked, for when you need to debug a failing test!

Here’s the changelog from JIRA:

GRAILSPLUGINS-772     FIXED     Add support for disabling JavaScript
GRAILSPLUGINS-771     FIXED     Add support for getting/setting textarea fields
GRAILSPLUGINS-773     FIXED     Enable redirect following by default, and add property to turn it off
GRAILSPLUGINS-779     FIXED     Sometimes requests will not update the current page data
GRAILSPLUGINS-783     FIXED     Allow resolution of form buttons by VALUE if no element can be found by id/name
GRAILSPLUGINS-781     FIXED     Allow user to clear the CSS+JS cache during a test
GRAILSPLUGINS-784     FIXED     Breaking change – make it so that setting value of checkbox/radio/selects does not need x.checked or x.select
GRAILSPLUGINS-786     FIXED     Improve output captured so that headers and assignments to fields are easily seen
GRAILSPLUGINS-785     FIXED     Provide access to cookies and enabled/disable cookies
GRAILSPLUGINS-788     FIXED     Rework it so that redirects that are followed show in output and correct request method shown for redirect after POST

Enjoy! Thanks to Peter for the work on Grails 1.1 compat.

Groovy, let me count the ways in which I love thee

Posted by: on Jan 16, 2009 | 6 Comments

Oh the wonder and the joy. I’ve been fixing some issues and adding features to the Grails Functional Testing plugin (now informally known as The G-Func) and had cause to marvel at some Groovy treats.

The problem: HtmlUnit, which the plugin uses for HTTP interactions, has an extensive hierarchy of classes representing page and form elements. There is however no way to indentify some of these as specific functional groups eg there are 4 different button types, which have no interface or ancestor that is not also shared by some other non-button classes.

So a little ugly logic is needed. For example when the plugin’s form wrapper uses Groovy dynamic property resolution to get a form field for you, it is now doing something like this:

private getFieldValue(f) {
  // Note this switch is polymorphic, order of cases is IMPORTANT
  switch (f.class) {
    case BUTTON_CLASSES:
    case HtmlFileInput.class:
      return f // return the field object itself, not its value
    case HtmlSelect.class:
      return f.selected
    case HtmlRadioButtonInput.class:
      return f.checked
    case HtmlCheckBoxInput.class:
      return f.checked
    case HtmlTextArea.class:
    case HtmlInput.class:
      return f.value
    default:
      throw new RuntimeException("Don't know how to get a value from form element of type [${f.class}]")
  }
}

Now, what is special about this is:

  1. Groovy’s switch statement is very cool. It calls isCase(value) on the objects used in the case statements. That’s pretty basic Groovy stuff, but is still news to most Java developers.
  2. The use of BUTTON_CLASSES in the case is very cool. That property is a list of HtmlUnit classes that are “buttons” in conceptual terms. Groovy will call isCase on the list, which will return true if the list CONTAINS the switch value. That saved me 4 case lines and made my code more DRY as this check is also done elsewhere.
  3. The isCase implementation on Class added by the Groovy GDK is polymorphic. So the “case HtmlInput.class” actually catches any descendent of HtmlInput – of which there are several. This is also why the case BUTTON_CLASSES is at the top of the switch, as some of those classes extend HtmlInput and would be incorrectly caught by this HtmlInput clause.

Many people will no doubt say I can simplify the code above further – the “.class” are largely not necessary and I could merge the “return f.checked” cases. I may well do in future, but this is there for clarity at the moment