A quick evaluation of testing frameworks : htmlunit

I had a quick look at htmlunit. It's a kind of a browser emulator, it parses html sites for you and even has the abliity to run some javascript. Then you could 'click links', 'fill in fields' and 'press buttons' on your website by calling some functions on the emulator.

The framework, which is written i java, seems kind of "oldschool" and doesn't seem to handle all the ajax we're doing on our site.It has some support for (XMLHTTPRequest) but that is kind of it. One nice thing about it is that it doesn't provide testingmethods (such as assertions) so you have to use an external testingframework to do this. This lets you use the same framework for all of you application and homogenise your tests. Nice.

Because we're using ruby on rails to develop our site and htmlunit is written in java we could have some trouble integrating it and probably have to use JRuby to get it to function. This is doable but think about it, first were using JRuby (a ruby runtime implemented in java) instead of ruby, this could be the cause of numerous bugs. Then on top of this we should use a browser emulator instead of a real browser, which will introduce even more bugs and glitches that, in reality, isn't there. The problem with this framework for us seems to be that we could spend to much time 'chasing ghosts'; even if the tests fails that doesn't mean that we have a bug. This is of course always a problem with testing, but this seems even worse.

Instead of this framework I stumbled over the Hpricot_forms framework. Hpricot is a nice framework for parsing html/xml content and the hpricot_forms is kind of a htmlunit port for ruby. It's really nothing fancy but it lets you "click links" and "fill in and submit forms" in your tests. I downloaded the project and after some troubles, including manually updating some code, I had it running and wrote some tests in it! It's nice and really supply a valuable addition to our testframework Rspec, but I think it would be even better if combined with the Storyrunner included in the next Rspec release.