The Great Test Framework Dance-off

— June 1, 2008 at 15:19 PDT

Got my RailsConf talk done yesterday. Seems to have gone over well, though my slides didn't show color well in the brightly lit room so they turned the lights down all the way to see them better and I ended up doing the talk as a ghost story for half of it. Guess I have to revise my rules for slide creation to account for ambient light.

Anyway, I have a PDF of the slides posted: The Great Test Framework Dance-off. I've also opened up the example code and tests on github as the teldra_prime project. That project is based on the application that runs this blog, but you should probably avoid deploying it to run your own blog. I'll be releasing the Teldra blog software as its own project very soon, and that will be the project where I continue development and people can contribute changes.

Also, it turns out I have a craaaaazy fan. Here's a picture of us at the end of the talk, me posing with his ballpoint tattoo of me on his bicep. Yes, somebody drew picture of me on his body! That so beats people introducing themselves at the urinal!

UPDATE: Alright, it turns out there is a way to significantly simplify the be_sorted custom matcher I showed as an example for extending RSpec. David Chelimsky sent me this alternate implementation that uses the simple_matcher helper:

def be_sorted
 return simple_matcher("a sorted list") do |actual|
   actual.sort == actual

That doesn't have all the power of the example I gave, but it is pretty much equivalent in operation to the test/unit example I gave and said was so much easier to write than one in RSpec. Nice one, David.

11 commentspresentation, railsconf, railsconf2008, sightings, testing

  1. bronson2008-06-02 13:57:31

    Unfortunately I couldn't be around on Saturday. Very briefly, did you arrive at a conclusion about the test frameworks? I'm leaning towards Shoulda -- less syntactic sugar == less cavities -- but it seems like Shoulda or RSpec would both work fine. Hope the talk went well!

  2. carlivar2008-06-03 13:41:27

    Yeah I'm curious too about general consensus, or if you have your own opinions. I just went through the slides but they seem to be more of the spark-discussion general variety rather than coming to a specific conclusion.

    I am also leaning towards Shoulda. I like that it is more backwards-compatible and I think it's just as easy to read. I seem to grok the nested contexts a little more.

    I think output is meaningless. I would think it would be trivial to make the Shoulda (default rake test) output look as nice as rspec.

  3. Clinton R. Nixon2008-06-03 14:06:16

    Josh - I really enjoyed your talk. Like you, I ended up liking Shoulda a lot after my testing journey. I didn't want to say, "Hey, what about my framework?" at your talk for obvious reasons, but I will now: Spect, a helping library for Test::Unit.

    I wrote it because I wanted to keep using shoulda, but really liked the way test/spec wrapped assertions. I think you might enjoy using it.

  4. Trevor Turk2008-06-03 15:00:08

    So... was there a winner? :)

  5. Rechnung2008-06-05 14:15:18

    simple_matcher - very usefull!

  6. Rev. Dan2008-06-05 23:34:42

    So... was there a winner? :)

    The winner is every developer who writes tests, regardless of which test framework(s) they use (or like). :)

    Testing Good!

  7. Trotter Cashion2008-06-06 07:32:27

    Josh, I really liked your talk. I wrote a post about the macro test stuff I'm doing in rSpec. It's available here: Macro Tests. Lemme know if this isn't what you meant by macro tests.

  8. David Chelimsky2008-06-07 13:02:15

    @Josh - thanks for posting the update about the simple matcher. And thanks for doing the dance off in the first place!

    @Rev.Dan - I agree wholeheartedly!

    @Trotter - thanks for blogging that material as well. Good stuff.

    @carlivar - the backwards compatibility thing is solved with rspec now. If you have an existing test/unit suite, just require 'spec' after you require 'test/unit' and rspec will run your existing test/unit tests using it's own output formatters. Then you can gradually move whatever parts you want to over to rspec's DSL, mixing and matching as you see fit.

  9. carlivar2008-06-08 00:01:32

    @David - thanks for the info.

    Actually after using shoulda for a week I'm ready to switch over to rpec. Shoulda is fine in theory but it seems to be quite buggy. For example, should_require_unique_attributes does not work if :allow_nil => true. I like the shortcuts to plug in good and bad values into models, but it seems like half of those shortcuts are buggy, or make me sync up the validation error message.

    Maybe it's just a Rails 2.1.0 compatibility thing though.

  10. George2008-06-10 16:31:55


    This was interesting--I needa go look at shoulda!

    It's also worth mentioning that if anyone is clinging to rspec for the expectation syntax (vs the assertion syntax of test/unit or shoulda), take a look at assert2. This lets you do:

    assert{ == 'Smith'}

    I think I prefer this syntax to rspec, actually, and a neat bonus is that if the assertion fails, assert2 will print out all the intermediate expressions in the parse tree, which can be a time saver. (It uses evil tricks with rubynode to accomplish this, but it has been very stable in my experience.)

    Thanks for the slides, Josh!

  11. David Chelimsky2008-06-19 08:18:43

    I just updated RSpec's simple_matcher in git to support configurable messages. This gives simple_matcher a LOT more power. See the RDoc for details.

Sorry, comments for this article are closed.