Migration Concordance

— March 2, 2008 at 16:23 PST

If you are a solo developer, Rails' migrations are the neatest thing since sliced bread. If you work on a team, you know that often it can be a real pain dealing with migrations. Someone on your team checks in a new migration and you don't notice it when you svn up or git pull, and suddenly all your tests are breaking. Or even worse, someone modifies an old migration and you need to reset and migrate up from zero (we're talking development here, not production). I'm not a fan of automatically running migrations (I'll leave the reasons for you to guess), but I do like to be informed of when I'm about to run headfirst into a wall. Saves so much wear and tear on my noggin.

And so I give you the migration_concordance plugin. It's pretty darn simple. From the README:

This plugin extends Rails migrations to provide notification when you need to run migrations. It will detect both new migrations and modifications to previously run migrations. It is primarily of use for team development, but is also useful when deploying a release to a new environment to determine when migrations need to be run. This plugin does not run migrations automatically, but will notify you whenever you need to run them.

How it works

Whenever you run rake db:migrate, the plugin takes a snapshot of the state of your migrations by generating an MD5 hash of each migration file and dumping them all in a YAML file. Put a 1-line check in your environment.rb file, and then whenever you fire up the Rails environment, a notification is printed telling you if you need to run migrations. The snapshot strategy allows it to tell when old migrations are modified, solving one of the most annoying things about using migrations on a team.

Why would anyone ever edit an old migration? Well, I think there are legitimate reasons to do so. First off, let me say you should never modify a migration that has been deployed and used in your production environment. That's just asking for trouble with a side of getting fired for incompetence. But during a development cycle, it can make sense to modify a migration. On my current project we keep a migration file for each model (or cluster of related models), and as we evolve and refactor the design of the model, we go back and modify its migration to update its schema. This makes more sense to me than having the definition of a model's schema spread out over a zillion migration files, and in practice it works really well. It's nice to only have to look in one place for a model's schema definition, and more importantly if two people modify the model's schema at the same time, it's easier to find the conflict if both changes are in the same file rather than in two different migrations. That said, there are those issues with noticing when you have to run an old migration again. Thus the plugin.

I haven't figured out exactly how to integrate this with Capistrano yet, but I think it will be handy to get a notice when migrations have to be run as part of a deployment. Stay tuned, or if you get inspired and tackle it yourself, please let me know.

How to get it

The code for migration_concordance is up on github at http://github.com/joshsusser/migration_concordance. If you're not running git (and why aren't you?), you can grab a tarball from the github page to ease your plugin installation. You can just untar into vendor/plugins - there's no install code to run. After you git clone or untar the plugin, add the following line to the end of your config/environment.rb file:

puts ActiveRecord::Migrator.check_concordance

Then whenever you run script/server, script/console, or whatever, you'll get a notice of whether you need to migrate. If you want to check manually, run rake environment to get the notice.

Lastly, make sure you add the file db/migration_snapshot.yml to your svn:ignores property or .gitignore file. Putting the snapshot file under version control will defeat the plugin, since the snapshot will reflect the migrations that were checked in, rather than the ones that were last run on your computer.

By the way...

You've probably heard this at least twelve times in the last week, but github is totally badass. I've never had a reason to put my code up on a hosting service like that before, but now I do. github is social networking for code geeks. If you want to hack on migration_concordance yourself, get yourself a github account and fork my repo and have at it. If you do something cool, let me know so I can grab your mods. DrNic has a very useful writeup of how to manage that, if you're not already up on the planet-smashing powers of git. And Chris and Tom have said that github will be free for open source repos (of reasonable size), so there isn't even anything to pay. Sweet.

15 commentsmigrations, rails

  1. Dr Nic2008-03-02 18:28:59

    I've been using http://pastie.textmate.org/160359 (in config/initializers/check_migration_version.rb) as a "hey you need to run rake db:migrate" warning; can't remember where I got it from, but its a nice hack.

  2. Josh Susser2008-03-02 18:40:08

    DrNic: That will work if you're only ever migrating upwards. Won't work for me, for all the reasons described in the article. I like that unless defined?(Rake) thought - I may have to steal that to silence the warning when running rake db:migrate.

  3. Trevor Turk2008-03-02 18:50:56

    Nice. Also, the Advanced Rails Recipes book has something similar that I've been using since I picked it up there: http://pragprog.com/titles/fr_arr

  4. Trevor Turk2008-03-02 18:55:56

    Sorry - I should have mentioned the recipe is titled "Fail Early" (it's kind of hard to find).

  5. wzph2008-03-02 21:48:13

    Running into these problems on a team project right now. Out of curiosity, why isn't schema.rb sufficient for you as the "one place [to look] for a model's schema definition"?

  6. Josh Susser2008-03-02 22:35:41

    @Trevor: now we know where the snippet DrNic mentioned came from (or was based on at least). Same comments apply. If you are only concerned with new migrations, that approach works fine.

    @wzph: schema.rb is fine to see the what a model's table looks like, but you can't edit it. You have to find the migration(s) where the model's table is defined, and it's easier if there's just one to look at.

  7. Tim Harper2008-03-03 00:05:03

    Awesome write-up, I'll check this plugin out.

    We've (Lead Media Partners) made the jump to git, and we've felt a lot of pain from merging migrations and having conflicts in revision numbers. The "Enhanced Migrations" plugin, which was written by gentlemen at Revolution Health. It assigns time-stamps instead of incremental numbers (In a sense, it's a bit like git - ugly long revision numbers, but very nice for merging :).

    So far, it's been phenomenal.

    Any idea of how this plugin would play with Enhanced Migrations?

  8. Tim Harper2008-03-03 00:07:24

    I love the premature freudian-slip posts. Insert "we started using" before "The \"Enhanced Migrations\" plugin".

  9. Lawrence Pit2008-03-04 14:09:24

    Imho there is NO legitimate reason at all to edit an old migration. You say it makes sense in a dev cycle, but only as long as you haven't deployed your first version to production yet. Once it is in production, even in dev cycle you can not go back to the migrations file having a definition of model X and modify it there.

    The definition of a model's schema is not spread out over a zillion migration files. It is in your current schema.rb (and in your database(s) if you like). No, you can't edit it there, but why would you want to anyways? You might as well modify your databases directly.

    If you simply stick to the rule that you can not ever edit an old migration, I find there's no pain at all, incl when working with a big team. I think the pain comes exactly because you allow old migration files to be edited, and you try to remedy the pain with a plugin. I assure you, if you keep allowing that misbehavior, the pain will be back.

    I think this plugin has an interesting use though. I would use it exactly for the opposite purpose: as a check to see if anyone edited an old migrations file, in which case that person would find himself hanging up on his ears to dry for a bit.

  10. Josh Susser2008-03-04 20:29:31

    Lawrence, to be clear, I wasn't advocating changing a migration that had already been deployed in production, even in a dev cycle. I was referring only to migrations that had been created since the last release. My typical dev cycle involves creating a number of migrations. Before I deploy a release, I collapse the migrations from that dev cycle into a single migration for production. It's just part of the refactoring/cleanup we do as part of our dev process. "Make it green, then make it clean." Answering the question of "what changed in this release?" is so much easier if you just have to look at one migration file instead of several dozen.

    I totally respect your take on things though. If it works for you, I'm not going to take issue with it. And if my plugin helps you rain unholy vengeance down on people breaking your team's rules, I'll be more than satisfied :-)

  11. Carl Youngblood2008-03-07 09:39:38

    +1 for having one migration per release. That seems to solve the problem of working in teams, and it also tames the migrations directory, which otherwise can fill up with many migrations that are often just additions and fixes for previous migrations.

  12. Wallen2008-03-18 04:40:51

    You've described our exact situation. And you have solved our exact situation.




    Thank you.

  13. Jonno2008-03-25 01:41:33

    Hi, sorry to post an off topic comment, the right post was closed for comments.

    Could you please do a post on has _and _belongs _to _many of the same type as the model. For example:

    class Friend
    #where the relationship is always both ways.
    hasandbelongs_to_many :friends

    What is the "rails way" of doing this?

  14. Nathan de Vries2008-03-31 05:03:03

    As Brooks once said:

    Therein lies madness

    1. Your schema.rb is the "single source of the truth". If you want to know what a table looks like, you should be looking there.
    2. You should never migrate from zero, that's what rake db:schema:load is for.
    3. Migrations should never be edited, only pruned (for migrations < production schema version).

    If you stick to those rules, it's very easy to work in a team & plugins like this become much less necessary.

  15. David Krmpotic2008-04-15 17:37:05

    I sent you a pull request...

    Suppose you install this plugin on the existing project and the database is up-to-date. You try to take a snapshot so that you're notified on the very next migration update, but you cannot do it without generating some dummy migration first!

    I made it so that if there is no "snapshot to compare", one is created immediatelly. The next run is then able to tell you if 'schema is in sync' or there were updates to migrations.

    I think this is especially nice for someone that decides to try the plugin on the existing project - with this little patch 'it just works'.

Sorry, comments for this article are closed.