Rails and Grails performance compared

Posted by: on Mar 23, 2007 | 12 Comments

Graeme and I put a little work into getting some initial figures comparing Grails and Rails performance.

Grame has put these together into a benchmark document on grails.org

I fully expect a bit of a firefight over some of this but tried to be as fair as we know how – we are certainly not Rails experts so there may be some idiomatic approaches/tuning that can be done. However we didn’t necessarily optimise the Grails tests maximally either… so I am sure debate will range for some time. We always believed we would see this kind of result because of Java and Groovy’s non-interpreted nature – despite performance improvements being required in some areas of Groovy dynamic method invocation and string handling. Hibernate and the mature Java VM are also paying dividends for us.

We did find during the tests that there is some performance tuning to do with GSP view rendering, as there appear to be some bottlenecks related to heavy taglib usage.

Here’s the requests per second result for 1000 requests across 50 concurrent clients, updating 3 random records (doman objects) and returning a programmatically generated XML response:

In a nutshell, in each of these tests, Grails mean request time is lower than Rails in every case, in some by quite a bit. Grails has not been optimized to any degree yet.

I just hope people realise this isn’t intended to start a flame war. People are asking us for figures, and here are some. Of course if comparing Grails to a pure Java application we might see better performance for pure Java than Grails, but this is not a like for like test. To a degree the mantra with these new-generation frameworks like Rails an Grails is that you are happy to trade off a little performance for massively enhanced productivity. Shorter time to market etc.

12 Comments

  1. Paul Barry
    March 23, 2007

    Did you use only one mongrel instance?

  2. Marc Palmer
    March 23, 2007

    If you mean was this a single server test, yes. Single server for Rails and single server for Grails. All tests were to localhost

  3. Paul Barry
    March 23, 2007

    A rails application is not threaded like a Java app. What this means is that when you only have 1 mongrel process, all requests must be processed serially. This is not true for 1 tomcat instance. Since tomcat is a java app, it is threaded and process multiple requests at once, when gives it better performance under a concurrent load.

    The way you handle more concurrent requests with a Rails app is to run multiple mongrel instances (on the same server, a different port for each) using mongrel_cluster[1]. Then you use something like Apache 2.2′s mod_proxy_balancer to load balance the requests across the multiple mongrel instances.

    It would be interesting to see these tests run again with mongrel_cluster running 2, 4, 6, 8, et.c instances to see what impact that would have. for this kind of test, I would guess that it actually wouldn’t be a tremendous boost, because each requests should be very fast. But when testing requests that can take longer than a few milliseconds, especially where uploading or downloading of large amount of data is concerned, the performance difference would probably be huge with using multiple mongrel instances.

    http://mongrel.rubyforge.org/docs/mongrel_cluster.html

  4. Marc Palmer
    March 23, 2007

    These tests are being done by a Rails/Grails user. So far, Rails gets worse with more mongrels, but they are still looking into it.

    Threading boosts concurrency, not performance. A single Rails mongrel running at full pelt and with an unloaded (single DB sessions at a time) MySQL is likely to be faster than any threaded solution, as far as I can see currently. No timeslicing, bottom line. Multi-threading slows things down if anything, but improves the concurrency under load.

    We did not detail # concurrent sessions per second so this aspect of performance, is arguably not relevant.

  5. John Wells
    March 23, 2007

    Marc,

    Well, I’m that user who’s doing the testing. And I need to clarify. Rails doesn’t get worse when going from 1 to 10 mongrels…it actually gets much better. However, going from 10 to 20 had a slightly negative impact:

    http://thoughtmining.blogspot.com/2007/03/grails-versus-rails-comparison.html

  6. Ravi
    March 24, 2007

    I’m not surprised that Grails is faster on most tests, even after you reran your tests using a cluster of mongrels. You’re still really comparing apples to oranges– Java runs as jit’ed bytecode and Ruby is interpreted directly. Bottom line, in my tests Java can be over an order of magnitude faster in the general case. I think its a credit to Rails that it actually won even one of the benchmarks! ;-) Just goes to show that at the end of the day most db driven web apps are io bound.

  7. Stephan
    March 26, 2007

    Ravi,

    I thought most Ruby code is C? (Not the Rails code). Which means this compars a.) DB I/O performance b.) C versus Java

  8. Marc Palmer
    March 27, 2007

    Stephan – this is not quite correct. C vs. Java is an oversimplification. If anything it is testing the implementation of the languages (Ruby in C, Groovy in Java), not the underlying languages.

    You can’t compare the max speed of two car engines unless both cars have identical weight, aerodynamics etc and Groovy and Ruby are very different implementations, so your comparison is meaningless.

    This is measuring what happens when you run the full application stack, that’s all.

  9. Erik van Oosten
    March 30, 2007

    Thanks for these nicely done measurements.

    Could you please, for the next round, repeat the Rails tests with erubis (http://www.kuwata-lab.com/erubis/). Erubis replaces the default erb to do the rhtml parsing and rendering.

    Instructions. First install erubis:
    # gem install erubis

    Then add the following line to “config/environment/production.rb”:
    require “erubis”

    Good luck!

  10. Erik van Oosten
    March 30, 2007

    And if I may add, please make sure you use Ruby 1.8.5. It is considerably faster then earlier versions.

  11. Adrian
    December 6, 2008

    Can you confirm the JVM options you used when starting tomcat – particularly the Xmx?

  12. Marc Palmer
    December 6, 2008

    Adrian – this was a long time ago – 1.5years. Can’t remember sorry