Always The Critic

Groovy and Grails thoughts

I've been slowly banging away at a small Grails app on the train to and from work, to teach myself Groovy the language and Grails the web framework. I already had a good familiarity with Java, Spring and Hibernate so it is mostly a case of learning what Groovy and Grails add on top of those. I've still got a long way to go, but I thought it worth jotting down some thoughts in case they help somebody else work their way into this particular world.

Overall, it's been fairly pleasant and I'm optimistic that it's probably as good a way to go about your average web app as any I've found before, and I've tried plenty over the past decade, including a few fairly obscure ones. I should point out that I don't believe in silver bullets any more. There are always going to be frustrations along the way when using any technology for a non-trivial project involving real-world customer requirements, but it's instructive to see just how many of those frustrations there are, and how hard they are to get past. I'm pleased to say that each time I've got a bit stuck with Grails I've been able to dig myself out fairly swiftly with the help of the existing documentation, other web resources, the #grails channel on freenode IRC and the grails-user mailing list. Furthermore, the principle of least surprise (POLS) is alive and well, meaning that when I hope that I'd find a particular feature to solve my problem du jour, it's usually there for me to find.

It's not all sunshine and roses though. For instance when I upgraded to Grails 1.3.3 my unit tests started failing with a strange and unhelpful stack trace emanating deep from within the guts of the plugin framework. Turns out this is a bug introduced in 1.3.3 that plenty of people have been caught out by. I've had to go back to 1.3.2 for now. It's a shade surprising and disappointing that a release could contain a bug like this, and that it's not been fixed and re-released yet. I worry that people trying Grails out for the first time with 1.3.3 will have a bad experience and give up early because of issues like this.

I had a struggle to find half decent tools. I've been a huge fan of Netbeans over the past several years and it claims to have great Groovy and Grails support, but I found it to be woeful I'm afraid. Common GSP tags completely confused the GSP editor (it claims the syntax is wrong) so I struggle to believe that those claiming it has this great support have actually tried developing a non-toy Grails app with it. I've settled on SpringSource Tool Suite (a bastardised Eclipse) which isn't bad - though as of this writing you definitely want to get the 2.3.3 M2 version for the latest Groovy and Grails support that's not yet in the stable release.

Groovy and Grails

Overview

I've spent the past couple of weeks reading up on Grails, a web application framework inspired by Ruby on Rails. It takes advantage of Groovy, a dynamic programming language that runs on the JVM and is best described as a cross between Java and Ruby. Groovy has support for most existing Java syntax, but also a lot of Ruby-style syntax and dynamic features. To be honest I find that the Java aspects of Groovy (and Grails) drag the whole experience down a bit compared to writing Ruby, but it should be significantly nicer to work with than pure Java.

The big selling point of Groovy and Grails over Ruby on Rails comes from the compatibility with the whole Java ecosystem. You can have your clever dynamic language without giving up the Java libraries and deployment environments that your projects and customers may demand. This is clearly quite a compelling idea if like me you've got a long history of doing big Java web apps but have recently been converted to the dynamic cause.

One thing that is very important to realise is that Grails is not a port of Rails. It is in fact built on top of the Spring stack, including Spring MVC, REST support, dependency injection, transactionality and Hibernate amongst others. If you're already familiar with Spring this is probably a good thing, though I do worry that the framework on top of framework approach could lead to having more to learn (and mess up) overall.

Practical findings

My initial experiments showed an interesting result that rather worried me. I set up a domain class and a controller with def scaffold = true to provide built-in CRUD for my domain class. However the CRUD pages this presented were noticeably sluggish in the browser and benchmarking confirmed it could only manage about 2 requests per second. This is lamentably poor and even now I can't figure out how it's managing to spend so much time achieving not very much.

After a bit of experimentation I determined that if I use grails generate-all to put the scaffolding code in place in my classes (rather than using the dynamic scaffolding) then things speed up enormously. In fact I can now get about 200 requests per second. It's hard to see how there can be such a massive performance gulf between the two scenarios as I would have thought that the only extra overhead for dynamic scaffolding is the initial dynamic intercept of the missing action method and calling into the scaffolding code. Hopefully a Grails expert can explain this, or I will eventually figure it out myself. At least now I have determined that for real-world scenarios there isn't going to be a performance issue, though I am surprised that the Grails docs don't have a massive "Warning - dynamic scaffolding is really slow" box.

For now I haven't done much but read the docs and try a few things. I hope to get deeper into it soon.

RVM and NetBeans

I've been having a lot of trouble with my Ruby 1.9.1 install on Mac OS X. Mostly it works fine, but I struggle when installing gems that require native extensions. I think this is because the way my install was built causes linkage problems, perhaps due to 32 vs 64 bit issues, or due to linkage with other libraries. I'm not entirely sure what's causing the problems, but recently I decided enough was enough and tried out rvm since I've heard a lot of good things about it. I got the impression that by compiling from my own source I was stubbornly making a lot of my own trouble.

Rvm is trivial to install: it's a gem that installs some of its own executables. I did hack my PATH first, to remove /usr/local/bin (where my custom Ruby lived) so that I'd be using the stock Mac OS X Ruby for the rvm install.

> sudo gem install rvm
> rvm-install

Note that rvm-install added the following to the end of ~/.bash_profile automatically, so I could ignore the instruction it gave me about adding it myself:

if [ -s ~/.rvm/scripts/rvm ] ; then source ~/.rvm/scripts/rvm ; fi

I then used rvm to install a fresh version of Ruby 1.9.1: >> rvm install 1.9.1

Actually that failed with an error about libsqlite3.dylib being the wrong architecture - perhaps another hangover from my old manual installs, or a problem I'm going to have to solve sometime in the future! For now I moved the old version of that file and tried again:

> sudo mv /usr/local/lib/libsqlite3.dylib /usr/local/lib/libsqlite3.dylibOLD
> rvm install 1.9.1

And that left me with a decent ruby 1.9.1 install. Which brought me back to one of the things that I was originally frustrated by: getting NetBeans Ruby debugging working with the fast debugger. With my old install the ruby-debug-ide gem would not install, but I'm pleased to report that it does with this new setup.

However getting NetBeans to actually use my new rvm ruby required a bit of a trick. The Ruby Platform management GUI in NetBeans doesn't show you hidden folders in its file picker, so you can't navigate to the ~/.rvm/ruby-1.9.1-p243/bin/ruby file that it wants. The trick is to create a non-hidden symlink, so you can then find it from NetBeans (and it's also handy to get at your rvm files from Finder):

> ln -s ~/.rvm ~/rvm

One word of warning: once you're using an RVM Ruby install, do not use sudo for gem installs, as the gems (and every part of rvm) live in ~/.rvm so sudo is not required. In fact using sudo will knacker your gems quite badly as it gets its PATH wrong and its permissions and you end up deleting a bunch of stuff to get back to a known good state. I learnt this the hard way!

MacRuby 0.5 beta 1 Released

I was very excited to see MacRuby 0.5 beta 1 had been announced, complete with ahead of time compilation via LLVM. It has been long while since the previous update on the MacRuby blog in March, but clearly a lot of work has been taking place. At the moment this beta shows the promise of things to come but isn't yet fit for much more than anticipatory experimentation. If you want to try the macrubyc compiler, Antonio Cangiano's blog post on the topic is a must-read.

The MacRuby notes suggest that compiled ahead of time or not, it uses LLVM for a big speed win, but my own quick experiment showed the macruby interpreter to be about 3 times slower than the standard MRI Ruby 1.9.1. This was with a single small benchmark app only though, just to prove things were working, so I can't draw conclusions. I can't pretend I wasn't a little disappointed not to see MRI blown out of the water though, even though I know it's unscientific and wrong of me!

I couldn't get a fully compiled version to produce any output, though it appeared to run without barfing, so I couldn't tell if it was really working or not. It was notable that the compiled binary was nearly 15MB so there must be a lot of statically linked code being included to swell my couple of KB of Ruby code so much. I'm hopeful that this can be improved in the future in order to support my dream of iPhone apps being written with Ruby hooking into Cocoa. In fact more than a dream - I'm hopeful and optimistic that in the long-run Apple will make Ruby a heavily promoted first class citizen for Mac and iPhone development, sitting on top of Objective-C but hiding it for the most part. The whole world has moved on from primitive C-based languages to higher levels of abstraction and I think Apple really needs a successor to Objective-C within the next 5 years. Is MacRuby it?

Setting an app path prefix in Ramaze

Say you've got a Ramaze web application quite happily running at http://foo.com/ but you want to have the whole app running at http://foo.com/myapp/. It's not unusual to want to do this, for various reasons, and it's relatively easy to arrange by setting a single piece of Ramaze config.

The exact way to do this depends on the version of Ramaze you're using, as the options system changed between the 2009.03 and 2009.04 gem versions. Here's the syntax for Ramaze version 2009.03 and previous:

Ramaze::Global.prefix = '/myapp'

And here's the syntax for Ramaze 2009.04 and later, which requires setting the prefix for links (as before) and a routing rule for stripping it off incoming requests (since it no longer does it automatically):

app_prefix = '/myapp'

Ramaze.options.prefix = app_prefix

Ramaze::Route[ /^#{app_prefix}(.*)$/ ] = '%s'

The prefix will be automatically added onto the front of all URLs created by Ramaze's built-in URL and link creator methods (R, Rs and A from the link helper). It will also be stripped off the front of all incoming requests by the dispatcher (for Ramaze 2009.03) or by the routing rule (for later versions). So your own application code doesn't need to be aware that it's running with a prefix at all, unless you create absolute URLs via means other than the link helper methods. In that case you will need to use Ramaze::Global.prefix or Ramaze.options.prefix in your own link construction code to get them right.

One thing which might catch you out is forgetting to put the forward slash on the front of the prefix. If you set the prefix to 'myapp' it won't work - it has to be '/myapp'.

Update: Note that static files (those in your public directory) are not affected by any of the above changes. Furthermore, chances are that you refer to them with absolute paths from your HTML anyway - e.g. for images, CSS, JavaScript. So further thought will be required to handle these.

New Ramaze

My favourite Ruby web application framework Ramaze has been updated, and in a major way. As of the 2009.05 gem release Ramaze is built on top of sub-project Innate - a lean mean core that in turn builds on top of Rack. Pistos explains the major changes well in his blog post. For me, although I'm yet to properly get to grips with the new code, the most promising things to look forward to are:

• A fresh start with a lot of accumulated cruft thrown out, resulting in cleaner, more straightforward code for Ramaze itself. I end up reading the Ramaze code a lot (it's very accessible and empowering) so this makes it even easier to delve into the internals and figure out what's what.
• Rack middleware is front and centre, making it easy to chain HTTP processing components into your request handling flow - like Java servlet filters. Not required very often, but a godsend when it is.
• A much tidier sessions implementation, with lazy initialisation.
• A rethought render helper that brings a bit more sanity to the many ways to bring in view fragments.
• Layout support is more of a core concern, with more facilities to deal with layouts sensibly.
• Built-in support for serving different content representations from the same controller.
• An interesting new codebase for me to poke my nose into :-) 

Now if only I can find time to get my Ramaze CRUD framework going in this new world order, then package it up to share with the rest of the world. One day!

Install Ramaze and Innate on Mac OS X's stock Ruby

Mac OS X comes with a lot of fantastic stuff built-in, like Java, Ruby, Python etc. The downside to this is that users tend to stick with the stock installs of those items and never upgrade them, which can leave them a long way behind the curve. I just ran into that problem when trying to install the latest version of Ramaze (my favourite Ruby web application framework) from the official gems. The IRC channel #ramaze was useful as always and provided the solution. The problem is that rubygems on OSX 10.5 is old and tired, but you can tell it to install a very specific version of innate that sorts it out. So from scratch, assuming Mac OS X 10.5 with the stock Ruby 1.8.6 install:

> sudo gem install innate -v 2009.04
> sudo gem install ramaze

Obviously as new versions of innate and ramaze come out, the exact version of innate required will change. You can tell which verison you need by trying the ramaze install and looking at the error message.

Update: Actually now Ramaze and Innate 2009.05 have been released, it seems things work just fine without any special incantations. So you can just sudo gem install ramaze and you're away.

NetBeans' Auto-formatting

I've endeavoured to rely on NetBeans' auto-formatting feature before now, but I've never been able to configure it quite to my tastes. In particular I had struggled to get rules for braces and line wrapping setup how I wanted for Java, with them almost always on a new line. I'm not sure whether I had simply failed to find the right options or if they're new in 6.5.1 or another recent release.

It's perfect for cleaning up code that you're copying and pasting from the web, taking the labour out of it. All I need now in NetBeans is a less tedious way of generating getters and setters that requires less mouse clicks, remembers your last preferences (put at end of file, no comments) and can have m_ prefix on the vars but not the accessor method names.

That said, better still would be for Java to grow up a bit and gain a better way to do boiler plate accessors. Which is better:

Java:

public int getFoo()

{

  return m_foo;

}

public void setFoo(value)

{

  m_foo = value;

}

public String getBar()

{

  return m_bar;

}

public void setBar(value)

{

  m_bar = value;

}

Ruby:

attr_accessor :foo, :bar

Adventures in Ruby 1.9.1

Spurred on by Antonio Cangiano's recent campaign, I've been trying to make a permanent move to Ruby 1.9.1. I've been on 1.8.6 by default for a good while, as that's what you get with Mac OS X 10.5, though it's actually not too hard to get going compiling your own Ruby from source - no special tricks are required and it works great. I hope Apple makes 1.9.x the default for OS X 10.6 when it appears later this year. Go on - be brave!

So what have I found in this exciting new world?

• It's a bit faster, as expected, though nothing to get especially excited about.
• Some very popular gems just don't work straight off the shelf. Mongrel for instance - though I'm told it's really easy to get it working with a couple of tweaks. Still, it's a shame for the sake of the community that the official latest gem can't have those tweaks made to it and save a lot of people an awful lot of fuss. Actually I hear that Mongrel 2 is on the way, taking advantage of more of Rack's infrastructure. I look forward to it. 
• ri now works really quickly compared to my 1.8.6 install, which always thinks for a couple of seconds before giving me a result. I think this is due to a new caching mechanism that was introduced around 1.8.7. Simple things like this make a huge difference to my life!
• ri actually finds docs for the standard libraries, which it didn't for my 1.8.7 install. I think was due to some issue in the standard makefile for 1.8.7, but I'm not really sure. Again, for someone like me who only just knows what he's doing, this makes a huge difference.
• Proper string encoding support! Finally the promised land, as long as you can take the hit of string indexing now being incredibly slow.
• Rubygems at 1.3.1, which is much more refined than that bundled with ruby 1.8.6 on Mac OS X. You can get this for yourself on any version of Ruby with gem update --system BTW. No more "Bulk updating Gem source".

I recommend everyone tries to move on up, at least with a parallel install that they can easily switch, and applies pressure to the holdouts to get it happening. Commercial pressures will make it tough for many, but when we break on through to the other side it will all be worth it I reckon. 

Install Ruby 1.9.1 on Windows

Update: There is a new effort to create a decent Windows Ruby installer. You can get the new downloadable installers from http://rubyinstaller.org/ though as of August 2009 these are still in preview form. It seems from my brief experiment that you'll still have to set your PATH manually, as per my manual instructions below. This installer also promises a faster Ruby as it builds using superior tools, so might well be worthwhile if only for that. Below are my original manual instructions for installing Ruby on Windows, which should still be just as valid as they always were. The installer is probably worth a look though.

The following worked for me on Windows XP, but maybe I'm yet to run into problems I don't know I've got:

• Download the Windows binary release from http://www.ruby-lang.org/en/downloads/
• Unzip it, in my case to C:\ruby191\
• Also download readline.dll (e.g. from http://www.econsultant.com/dll-database-r/readline.dll-download.html) and put the DLL in C:\ruby191\bin
• Put C:\ruby191\bin and C:\ruby191\lib on your PATH

That got ruby working in a basic sense for me, but I wasn't able to install gems. If I tried, I got two errors in sequence: "The ordinal 277 could not be located in the dynamic link library SSLEAY32.dll" and "This application has failed to start because zlib.dll was not found."

The ssleay32.dll problem was solved by getting hold of a couple of DLLs and putting them in the right place, as follows:

• Download "Win32 OpenSSL v0.9.8k Light" from http://www.slproweb.com/products/Win32OpenSSL.html.
• Run the installer, but take note of the following:
• You can ignore any initial warning about not having Microsoft Visual C++ 2008 Redistributables.
• You can ignore any warnings about command prompts being open.
• Install to somewhere like C:\OpenSSL - it doesn't really matter, we just need a couple of DLLs.
• When prompted, choose to Copy OpenSSL DLLS to The OpenSSL binaries (/bin) directory.
• Once the installer has completed, you can go to C:\OpenSSL\bin (or wherever you installed it) and copy the two files ssleay32.dll and libeay32.dll to your Ruby bin directory, C:\ruby191\bin in my case.

The zlib.dll problem was solved by in much the same way:

• Download the compiled zlib 1.2.3 DLL from http://www.zlib.net/ (about two thirds of the way down the page).
• Unzip it to get the zlib1.dll file.
• Put that file in your Ruby bin directory, but rename it to zlib.dll

Et voila - gems can now be installed successfully! Note that you may have had copies of some of these DLLs in your \WINDOWS\system32 directory already (I certainly did) but the problem is that they are too old. I prefer to put the newer ones directly in the Ruby bin directory, so only Ruby picks them up and it can't break anything else on the system.