The great thing is that actually, with only a little knowledge, doing this stuff is really easy. The Arduino board designs and software are open source, which is a good thing. However as usual this means the docs can be problematic.

There’s a confusing array of Arduino hardware out there. The boards have the ATmega micro controller chips on them that you program, and lots of digital and analog input/outputs so you can control and measure things. There are boards with Ethernet, Wi-fi, USB and even LED matrix drivers.

Basic Arduino (UNO)

Using this Arduino which has just a USB port and the I/O connectors, plus a bread board and 4-digit 7-segment LED display from an Earthshine starter kit, I managed after digging out the spec sheet on the display, to create a counter from 0001 to 9999. This sounds trivial, but it has a few challenges:

1. The LED display is basically 4 x 8 LEDs (7 segment + decimal point)
2. There are only 12 pins on it. Huh?
3. If you don’t send power to one of the LEDs, it goes off.

The spec sheet showed that 4 pins control which of the 4 digits you are addressing – set them LOW (common cathode) to enable the corresponding digit.

Then you set some of the other 8 pins HIGH to represent the digit or symbol you want to show.

So to set all 4 digits without any shift register chips or other smarts to “remember” the state of segments for each digit, you have program the Arduino to continually loop through the 4 selector pins, set the segment pins HIGH, wait a few milliseconds, then set the selector pin for that digit HIGH (to turn it off), and then repeat the same process for the next digit’s selector pin. What happens is that each digit is illuminated for say 5ms and then it moves to the next one.

They are continually going on and off, but because you constantly loop at high speed over them, it appears to be a continuous display.

Much more fun – the Rainbowduino

The Rainbowduino is a self contained Arduino board and the latest v3.0 one has built in mini-USB. So its just like an Arduino UNO but has a bunch of other controllers to drive an 8×8 RGB LED matrix with constant current (which I think means when lots of LEDs are on, they stay the same brightness). CAUTION: the LEDs are also seemingly as bright as the sun and staring at them is likely to leave you with a disturbing 8×8 grid of dots burnt into your retina.

So I bought the Rainbowduino plus an 8×8 RGB LED Matrix. The Rainbowduino is built to have the matrix directly plugged into the top of it, which is very handy.

Rainbowduino board plus LED matrix mounted on top

There are some tricks to getting this working however:

1. Unlike the Arduino UNO, my Mac would not recognise the Rainbowduino when plugged in to USB. I had to install the “FTDI” drivers from here. Thanks to @andydavies for the tip there
2. The docs on Rainbowduino are sketchy, and seem to be mainly targeted at an LED 3D cube rather than the matrix, and using the Processing language (which is not what the Arduino IDE uses).
3. It turns our there are several options for controlling this matrix. You can flash your own Arduino “sketches” (programs) directly onto the Rainbowduino, or you can use the I2C serial protocol to drive the Rainbowduino from another Arduino – I am guessing this means the Rainbowduino ATMega has a custom boot loader that supports serial comms to drive the display. I may be wrong.

To solve (2) I had to find the driver code that would work with the Arduino IDE. The v3 Rainbowduino driver is here. If you save that to ~/Documents/Arduino/Libraries/Rainbowv3 or similar (for Mac OS X at least that is where it is) and restart Arduino IDE, you will see you can now insert it into your Arduino sketch using the “Import Library…” menu option.

You will then find it does not compile, complaining it cannot find “WProgram.h”. This is I think because the library is built to run with “Wiring” programming language. To fix this you need to edit the Rainbowduino.h file from the driver that you saved, and change the #include <WProgram.h> to <Arduino.h>.

After that you can use the driver, which is incredibly simple:

1. in setup() call Rb.init()
2. in loop() call some Rb methods such as: Rb.blankDisplay() and Rb.setPixelXY(x, y, rgbcolour)

Rainbowduino with diagonal spectrum

Why so dark? LEDs are too bright to photograph without a filter. I put lots of tissue paper over them.

So here’s an example to apply the colour spectrum diagonally to the entire grid in action:

Note that if you find your reds are not working you probably have a C typing problem, and need to explicitly cast your values to uint32_t to avoid the RR of your RRGGBB being lost to 16bit accuracy.

And seeing as most of my life is spent in Grails… this wouldn’t be complete without a video of the Grails cup shining in all its 8×8 glory.

And here’s the code, if you want some Grails glory yourself:

It was meant to be 1.1 but there was a bug that needed fixing immediately after releasing so 1.1.1 is where we’re at. This version is tested with Grails 1.3.7 and Grails 2.0 RC1 which is set to drop this weekend.

What’s new in this release of Resources?

Well aside from some useful bug fixes there’s a few interesting changes:

1. If you forget to include r:layoutResources tags, you will get an error in the console. The error may or may not display in the browser currently depending on whether you used site mesh layouts or not. We may be able to improve this in future. This means that people who wondered where their JS code was disappearing to after installing Resources into a 2.0 app will now get an error telling them that they need to add r:layoutResources.
2. There is now support for laying out arbitrary dispositions by passing a “disposition” to the r:layoutResources tag. This means you can position sets of resources wherever you like in the output, using multiple dispositions. Currently this is only relevant for JavaScript but in future we should be able to support images so that you can e.g. have a bunch of “preloaded” images that are hidden in the page somewhere.
3. The integration with Grails 2.0 has been “finalized” now. The ResourceService has changed to a “grailsResourceProcessor” bean, and all g:javascript invocations result in called to r:require or r:script as necessary. The default layouts in 2.0 now use layoutResources and a default “application” module exists in new projects.

There are still some outstanding important bugs that I will endeavour to fix soon, as well as improving the error reporting on all fronts.

Enjoy!

The start of this article will go over some of the basics again, as they are a prerequisite for understanding the more advance topics covered later in the piece.

Recap: What is Resources framework for?

It solves the growing problems around application and plugin static resources. Your CSS, JS and other files have dependencies on each other, and must load in the correct order. You also want to optimize these so that your application loads quickly and caches well for your users.

It provides a way to declare your resources, their dependencies, and a processing chain to modify the resources and response headers to perform all manner of tasks such as bundling multiple resource files into one file, zipping the response, applying long-term client-side caching, or compiling Less or Sass files to regular CSS on the fly.

Where do I start?

First you install the “resources” plugin. To actually see some magic happening you probably want to also install the cached-resources and/or zipped-resources plugins also, but this is not a requirement.

Then you declare some resources in your app, or pull some in from a plugin that supports Resources. Let’s start by declaring a resource that exists in your application, say main.css. To do this you create a file in grails-app/conf/ with a name that ends in “Resources.groovy“. So we might create AppResources.groovy and use the DSL to declare the resource:

This is a module declaration. For full details see the Resources documentation, but in essence this declares a resource module called “common” and it has one CSS resource in it, the file supplied by your application in web-app/css/main.css, and on JS file.

So far, so good. Now you need to pull this resource into a page. To do this requires using some Resources tags.

Using resources in your pages

There are two main tags involved in using resources, with some others that provide extra features. The main two are r:require and r:layoutResources.

The r:require tag tells the framework which resource modules the current GSP requires. The framework will make sure that all the resources declared in those modules are pulled in in the correct order.

To achieve this, you need to include r:layoutResources tags in your <head> and <body> sections. Typically you do this in your SiteMesh layout. You use two calls because one will render the resources that you need to link to in the <head> section of your page (all your CSS, and some of your JavaScript, maybe some favico or iPhone icons). The other goes at the end of the <body> of your page, to render any “deferred” resources, which is usually just your JavaScript code.

So here’s an example of a Grails Sitemesh layout using these tags:

Here you see that you still use the normal Grails sitemesh tags to layout the head and body. These provide the route for you GSP page to actually affect the output, including the output of the Resources plugin’s r:layoutResources tags. That’s because your GSP will call r:require and r:script tags something like this:

This GSP is set to use a Sitemesh layout like the example one given earlier. It uses the r:require tag to say that this page needs the “app” resource module. Using this information, the r:layoutResources tags will render links to all the CSS, JS and other resources of that “common” module, in the correct places.

This means that some may end up in head, and some may end up in body. Remember that we invoked r:layoutResources twice, once in each location? Well, that’s why.

What is this “disposition” stuff?

The disposition of a resource tells the framework which part of the page it belongs – its a form of arbitrary grouping. The default dispositions supported are “head” and “defer” but in fact resources can define any disposition they like – and this can be useful for declaring resources that are not rendered by the normal layoutResources calls (more about this later).

The framework applies a default disposition based on the type of the resource – so CSS always defaults to “head” but JavaScript always defaults to “defer”, so that it runs only when the rest of the page has loaded. Images and icons are also supported to represent favicon and iPhone icons, which render in “head”.

Resource modules usually contain multiple resources and the main reason to specify a disposition when declaring a resource is to force JavaScript to load in the <head> section of the page. To achieve that you just update your resource declaration to include a “disposition” attribute:

Now your page will load with jQuery in the head section, with no code changes to your layout or GSP!

Dependencies

So far, so good but how do we control the order of resources when they are pulled into the page? How can I make sure that my application JS or CSS code always loads after the JS and CSS from a plugin that I’m using?

This is where Resources dependencies come in. There are two simple rules

1. Resources from a given module are loaded in the order they are defined in the module
2. Resources from modules are loaded in module dependency order

The first rule ensures that when you declare resources you can intuitively see which CSS or JS loads before another.

The second rule uses the “dependsOn” method in module declarations to list the names of other resource modules on which the module depends. Those will always be loaded before the module that depends on them. Here’s an example declaration that alters our specimen module to depend on the jQuery plugin’s query module instead, and jQuery UI:

Notice that we have specified both jQuery UI and jQuery in the dependsOn. We don’t need to. However notice that they are also declared in the “wrong” order. This is to illustrate that the order you use here is not important – the ‘jquery-ui’ module declared by the Grails jQueryUI plugin depends on on ‘jquery’ (provided by the Grails jQuery plugin) as well – so it will always make sure that is loaded first.

As your application does not define those two modules, you would need to install the Grails jQuery-UI plugin – or define them yourself. Including the jQuery-UI plugin automatically pulls in the jQuery plugin for you, because of Grails’ transitive plugin dependencies. TheResources framework was designed to work hand in hand with the other Grails dependency mechanisms.

So now you have done this, you still have no changes to make to your GSP! It still just requires the “common” module. Notice how this has decoupled your GSPs and layouts from the details of the resources that they need. This is an important feature that gives you great scope to modularise your resources, and even have page-specific modules if you like that contain nothing other than dependencies themselves:

With this, an edit.gsp could just r:require the “editPage” module and never care about the explicit details of what is required.

Dealing with your ugly inline Javascript

You already know that inline Javascript is pretty horrendous, but some things are just plain shameful and yet we can’t help ourselves. Frequently you want to write a Grails taglib that simplifies using a JS library and to do that it must render some code inline when the tag is invoked.

With Resources you can trivially throw this JS code around to wherever you like. You can have the code render in <head> or at the end of the body, and it will automatically do so after loading the resources that the page depends on. To do this you just use the <r:script> tag instead of a normal <script> or <g:javascript> tag:

In this example the script code specified in <head> does not run immediately. The r:script tag also uses “disposition” and the default is “defer” so you will find that the alert only pops up once the rest of the page has loaded, and the HTML source will show the <script> tag at the end of the body, after all other deferred JS code has been pulled in.

All such fragments in the same disposition are concatenated together, so you don’t get a long list of <script> tags.

To force a fragment of inline JS to execute in <head>, just add disposition=”head” to the r:script tag invocation.

To write your own taglibs that generate code in this way, you just output the result of calling the r.script(attrs, Closure) tag from your tag.

Linking to images

Images often need to be optimised or set for long-term caching. However you don’t normally need to declare them in modules. To render images correctly you should use the <r:img> tag anywhere in your pages as you would and HTML <img> tag. All you do differently is supply a uri or dir/file attribute to tell it where to load from. Other attributes are passed through:

That’s all there is to it.

There are however some extra tricks you can pull. Any tag attributes specified in a module resource declaration are honoured when rendering the link. As such this means that you can specify your width & height values if you declare your image – and never have to specify them when rendering links:

Now any r:img links to that logo will automatically include those attributes in the output. Notice that we had to include the disposition “image” which is not a predefined disposition – it is just anything other than “head” or “defer” to prevent the image being rendered by the r:layoutResources tags.

Bundling: combining files to reduce the number of requests

Page load time is a combination of many factors. Those that you can control from your application itself include how many requests are needed to load your page and whether you load resource before or after your HTML content.

To reduce the number of requests made, you need to combine one or more resources into a single file. This is called bundling and what you do is tell Resource which files can be bundled together. This is usually dependent on how your application uses the code, and inevitably requires some compromise.

For example you could bundle all your resources into one file across your entire app – resulting in just one CSS and one JS file for the entire site. With aggressive browser caching this could yield good results, but if you have a few hundred KB of code in these files it could mean it takes a long time for people to see your site the first time they visit or empty their cache.

To specify the bundle that a file belongs to, just add the “bundle” attribute to resource declarations. The Resources framework will not bundle together resources that are incompatible: different content types, or those with differing “attrs”. For example CSS with media=’print’ will not be bundled with screen media.

It does however allow you to bundle resources across module boundaries. This is important as you will see shortly. It also automatically bundles using the module name by default, and maintains separate bundled files for head and defer dispositions automatically.

That is a lot to take in, but it basically means you don’t need to worry about anything.

Just set the bundle attribute:

Here the main.css, which previously might have auto-bundled into bundle “common” if there was more than one resource in the module, will now be bundled into bundle “core”.

The difference has no effect for now, but what you can do is override the module declarations that came from the jQuery plugins and make them go into the same bundle. To do this we use the “overrides” mechanism to change the properties of the resource modules supplied by the plugins. We can call the defaultBundle method to change the default used by them:

Now all the modules will throw their resources together, and you will find your page (again with zero GSP changes) is now pulling in just three files: bundle_core_head.js, bundle_core_head.css and bundle_core_defer.js

Changing how links to resources are rendered

Under the hood, the r:layoutResources tag uses the r:external tag to render links. The r:external tag is a smart tag that writes out the “right kind” of link for a resource. So for JS files it renders <script src=…> tags, for CSS it renders <link href=…>, and so on for favicons and a few other types.

The key thing is that you can pass any other attributes to r:external and they will pass through to the output. It so happens that r:layoutResources pass the “attrs” Map defined in your resource declaration when calling r:external for that resource. This means you can pass any extra attributes you like to the declaration and they make it to the output.

A typical usage for this is to specify media:’print’ on a CSS resource, because the default for this is “screen, projection”:

Note that if you specify these extra properties, those files will not be bundled with others, as by definition this isn’t going to work out well for you.

Wrapping the link in some other code. AKA MS IE workarounds

The resource declaration can also supply some wrapper markup that is used to render the link, with the generated link being passed in as a string. You normally use this to wrap a resource such that it is commented out for all browsers except MS IE:

The wrapper Closure is passed the link text as an argument. You are then free to pre/post-fix it with anything you like.

Linking to specific resources without using the dependency mechanisms

On occasion you may need to link directly to a resource that is handled by the framework, without using r:layoutResources or r:require.

To do this you can use r:resource to get the URL of a resource, or r:external to create a link to it. You can use these to generate absolute links to those resources, for example in an HTML email.

These tags accept the regular uri, url or plugin/file/dir attributes. The r:external tag also supports a “type” attribute to give it a clue what kind of link to render if it is not obvious from the file extension.

Overriding resources from plugins

Plugins that expose modules normally do the right thing, but sometimes you might need to supply different content for a given file, or change some of the tag attributes or the bundle used. To do this you use the overrides mechanism shown earlier in the defaultBundle example.

The “overrides” closure lets you specify new values for some of the module defaults for a given module, but also lets you tweak individual resources if they can be addressed uniquely. Developers must add an “id” attribute to the resource when it is declared for this to work:

This changes the bundle on just the resources with id “js” in the original jquery module. There might be one or more resources with that id in the module. Resources in other modules are unaffected.

For full details of what you can override, please see the Resources plugin documentation.

Using r:require in GSP fragments

Using g:render to render GSP templates to build up your pages is a powerful technique. What you may not have realised is that you can completely modularise these templates now because the enclosing GSP no longer needs to know what libraries they need.

Yes, you can call r:require at any time in GSP rendering – including inside nested GSP templates – and the dependencies of your code will be stored along with your request and the resources all tallied up when the r:layoutResources called in your Sitemesh layout execute.

The only pre-requisite is that your r:require must be invoked before the corresponding r:layoutResources that you’d like to use to render it.

Picture how easy it is now to create portlet-like user interfaces where only the resources needed by UI elements displayed to the user will be loaded, and when they are they will be optimised. The portlets they have not installed or disabled will not load their resources if you do not render their GSP template.

Dynamic requirements

It is important to realise that the r:require tag can support normal GSP EL values in the modules list attribute. This brings great power because you can change the code pulled in by a page at runtime. For example you can make yourself a trivial per-user theme-switching system by storing the name of the theme they want to use in the session, and using a tag like this:

This will locate the theme module and all its dependencies with no further coding.

You could even switch the entire JS library and supporting code used by your plugin’s UI so that, for example, the user could choose between jQuery UI or YUI, or their own implementation.

Including common resources in your layout

Most likely you have some resources that are used in all pages using your Sitemesh layout. This is no problem because you can call r:require inside your Sitemesh layout, with the same proviso as “Using r:require in GSP fragments” – they must be invoked before the r:layoutResources call that would render them.

This means that in practice your individual GSP pages are likely to have only one or two module requirements as the rest is handled by the Sitemesh layout.

Patterns for plugin authors

Here are a few emerging patterns for plugin developers to heed when supporting the resources plugin.

• Specify id attributes on your resources when declaring them, to maximise the user’s opportunity to override specific details
• Avoid monolithic modules, opting were possible to separate out functional areas such as the jQuery UI theme being separate from the jQuery UI code. This gives developers more flexibility. Obviously you must maintain correct dependencies between these
• Put “development” un-minified versions of resources in “<yourmodulename>-dev” modules, so the user can pull in un-minified versions on demand, per-environment if they wish
• Do not specify the bundling of your modules – at most use defaultBundle. No per-resource bundle attributes. Let the developer control this easily by overriding your defaultBundle settings in each module.
• Use module names prefixed with the name of your plugin followed by a hyphen i.e. “jquery-main”, “myplugin-theme”, “acmeplugin-ui-styling”

Summing up

All the things we’ve tackled here are core Resources framework features – no extra plugins are required. The resource mapper that performs bundling is even built into the framework.

The real fun starts when you install some of the other mapper plugins – they may be renamed, re-encoded or even compiled from one form into another. It all happens at startup for declared resources, and at runtime for ad-hoc resources. Once it’s been done it is stored on disk on the server in that state and no more processing is done.

I hope you have fun with the Resources framework!

There are some nice refinements and maybe some new features coming in point releases at the time of writing.

The most interesting figures for me are that almost 50% of respondents do think there is a need for commercial plugin support, and that a lot of respondents’ companies already pay for commercial tools and support.

Also it was interesting to see the breakdown of usage of my existing plugins, as well as the spread of Grails versions in use. This survey happened just after the release of Grails 1.3.7.

View the survey results here.

The interview can be found here.

I used to build it with a custom Java app that processed XML using WebMacro to produce multiple HTML formats.

Yeah, XML.

This time I decided to write it with less emphasis on all the “7 years this”, “4 months that” so adored by recruitment agencies. Turns out I’ve actually really been a consultant for circa 75% of the last 14 years, and this level of skill detail is not needed by my prospective clients.

So now I have built it trivially using Markdown in TextMate. A little pain on the work history requires some inline HTML, but aside from that its trivial to build a new version. Using a butchered Blueprint CSS framework and some little CSS3 and Google Font sugar, its come out simple and clean, I hope.

Oh yes, if you want an extra hand on your Grails project, need some custom plugins, support or other Grails/Groovy related work let me know. I’m still paid to work on Weceem, but the time there is flexible – variety being the spice of life!

In my experience over the years it became obvious that one of the major pain-points of using Java to deploy web apps is the hassle of updating a bit of content here or there, replacing an image – those kinds of things that should be simple but aren’t. Also a lot of the time this stuff is not subject to the same strict versioning processes that we are using for our code.

As you may know I am paid to work on the Weceem CMS plugin and application, that is also Grails based.

For NoticeLocal I decided that it would suit us well to use an instance of Weceem CMS running on one Tomcat, to supply the marketing/PR site for the service, as well as the multi-language content for UI elements in the main application.

It turned out this was really easy to do, even though Weceem currently has no API to speak of.

To do this we:

1. Have a vanilla WeceemApp WAR running in one Tomcat instance
2. Have our NoticeLocal application WAR running in another Tomcat instance
3. Wrote a trivial taglib in the NoticeLocal application to retrieve content nodes from the CMS instance using vanilla HTTP requests to URLs with a predefined prefix, and stores the text them in a local ehcache.

So, in Weceem we have a bunch of HTML content nodes that represent different elements of the main NoticeLocal app’s UI – we have several different Weceem templates that these different nodes use – e.g. one for side panels, one for full page content (but laid out inside the main NoticeLocal app sitemesh layout, and one for inline UI messages such as those shown on demand when the user performs some action.

So in our NoticeLocal application we use our taglib to render a “remote CMS message” in our pages using the not-so-well-known Grails/SiteMesh g:applyLayout tag that, just like Grails layouts, will parse out the body of the content that we’ve cached, and render it in situ.

Oh, and for bonus points we use a simple regex replace to give us GSP-like variable expansion when rendering the content from the local cache, so we can perform variable substitutions like ${userName} – so you can write that in a HTML node in Weceem, and by the time the NoticeLocal app renders it, it has your name in it.

At any point we can log into the CMS, update some UI message, and when the cache expires the users see the new updated stuff. No app redeploys.

Oh, and because its in a CMS we solve the hard problem – i18n translations of rich content with styling, images etc.

In a nutshell, it’s great!

Until I installed OS X 10.6.5 update last night.

Safari under 10.6.5 does not handle cookies stored against IP addresses in the same way anymore, presumably as a side effect of a cookie-related security “fix”.

So to state it simply, if you have a web app that sets a cookie with the Domain attribute set to an IP address, this cookie will be stored but never sent to your server. What appears to happen is that cookies without a domain set e.g. JSESSIONID set by Tomcat, are successfully stored against the IP.

However as soon as you include domain= in your Set-Cookie header, this is stored with a “.” at the start of the domain in the browser cookie list (even if you didn’t put a “.” at the start of your domain) and it is then never sent to the server on subsequent requests.

The workaround is to update your code to not set the domain to the IP address of your host, but to the local host name. This is not normally a problem except when you are using a development tool locally.

By changing your code to use the bonjour name of the local host e.g. “marcmacbook.local” the problem goes away again – provided you browse to marcmacbook.local to test your site.

Something tells me this is a regression in OS X, not a deliberate change relating to the CFNetwork security bug.

Here’s a complete example:

This will render five news articles from the BBC news feed. It is incredibly naïve:

1. It doesn’t make any attempt at error handling
2. It will likely blow up if there are less than 5 items in the feed
3. It doesn’t do any caching – the feed is retrieved (by the server) every time the widget is rendered

However with a few tweaks this can be turned into something more robust. I imagine we will add a specific tag for this that hides the error handling and caching issues into a future build of Weceem.

This obviously needed improvement, and this tied in well to work I had already done on the small but beautiful Grails cache-headers plugin.

The problem: with a CMS you generally can’t make the client cache any content, not without some config for the CMS author/administrator to specify how long to cache it for – this is a problem we will tackle later. However at the very least you want to avoid re-generating content when requested if the client already has the same content.

The solution: Implement ETag and Last-Modified handling correctly so that when you receive requests with If-None-Match or If-Modified-Since you can return “304 Not Modified” if the content is the same as that the client last downloaded. This can give you major improvements in client response time and also server load, even in applications with highly dynamic content.

The thing is, implementing this from scratch in every Grails controller action you have is rather dull.

So in steps the new withCacheHeaders method in the Cache-Headers plugin. This method works a little like Grails’ excellent withFormat method, using a simple DSL to specify:

• How to generate the ETag for this request
• How to calculate the Last-Modified date for this request
• How to render the response, if we need to do this because the content has changed since the client last downloaded it.

Using this information that you provide, the cache-headers plugin will automatically work out if it can return “304 Not Modified” or if it has to render your response for you.

Here’s a working example from Weceem’s content rendering controller:

As you can see there are three “method” calls inside the withCacheHeaders call: etag, lastModified and generate. You simply implement these and the plugin handles the rest.

In the “generate” closure we’re also calling the lastModified method added to controller actions by the cache-headers plugin. This lets us easily set the Last-Modified header of the response, and then we make a token effort at setting client/proxy caching headers with the “cache” method. This will be subject to later refinement.

For more details see the docs.