Rails framework upgrades with git-bisect
When Rails 2.2 hit the shelves, the features were not compelling enough for us to move Poll Everywhere from 2.1 to 2.2. Now that 2.3 is out, we've got a number of reasons to upgrade and as expected, there was a lot of breakage. To make matters work, we have some code that does some really hacky things to the Rails framework. We tried running regression tests but that only helped us fix about 80 of 100 failing tests (out of 600 total tests). For the last 20 some failing tests, we needed to dive deep into the Rails framework and find out what specifically changed to help us find what might be breaking in our code. Fortunately my new best friend, git-bisect, was there to save the day. Here's how:
First, clone Rails from github somewhere on your local machine. We're going to symlink this into your project vendor/rails directory.
my_project$ ln -s /local/github/rails_clone/ ./vendor/rails
You're going to want to checkout the version of rails you're testing against from the rails git repo. Fortunately the rails team does a great job tagging their releases so its pretty straight forward
rails $ git checkout v2.1.2
Do a sanity check by running your test suite to make sure everything is passing. Now the fun starts; checkout the version of rails you plan on upgrading to:
rails $ git checkout v2.3.2
Run your test suite again and see what's broken. In the case of Poll Everywhere, we had 100 tests failing and were able to fix 80 before we had to dive deep. If you're luckier than we are, you might be able to fix everything and verify it through automated tests. If not, get ready for a bunch of server restarts.
Now we want to start git-bisect to see what change in Rails broke your application. Finding out what commit in Rails broke your app makes it a hell of a lot easier to hunt down the offending code in your app. Lets start git-bisect:
rails $ git bisect start
rails $ git bisect good v2.1.2
rails $ git bisect bad v2.3.2
Bisecting: 1104 revisions left to test after this
[550fbcceddceabdb4fe000ee52142e8461b28d54] Fix test warnings
Git will checkout a commit that you need to mark as good or bad. The determining factor for good or bad could be the tests you wrote (do they all pass/fail?) or does your application crash when you are running it. If all is well, you'll mark the commit as "good" and you'll get the next commit to test:
rails $ git bisect good
Bisecting: 552 revisions left to test after this
[3b35366d5df8c8d8a7b216c42dd96b0cfa38fee4] Use more generic test env flag
If the next test run is bad:
rails $ git bisect bad
Bisecting: 275 revisions left to test after this
[d8a555e1376ed509c9f81c42229c5e923153eeb3] Mocha 0.9.0 compatibility for test setup/teardown callbacks
And so on until you find the offending commit:
Bisecting: 1 revisions left to test after this
[f7f113610e7cdca8ef03e206f2cbeb8400cdfefa] Add a rake task to apply a template to an existing application.
rails $ git bisect good
Bisecting: 0 revisions left to test after this
[e4eadf3910d04fcdcab3e084d249f3a881a3ca35] Fix message when running TemplateRunner#git. [#1526 state:resolved]
rails $ git bisect good
ebec9d43e262d28d742ff10acd828bad6cbb28ed is first bad commit
commit ebec9d43e262d28d742ff10acd828bad6cbb28ed
Author: Joshua Peek <josh@joshpeek.com>
Date: Sun Dec 7 16:37:48 2008 -0600
Make integration test runner more Rack friendly and clean out old CGI cruft
:040000 040000 66590983a84e62e949b326db9b8d7da717a15a79 d333e0e0580849752bf4d192b8061a64c73e49fc M actionpack
Now you can take a look at the patch, see what code is changed, and hopefully follow it to a point in your code that is blowing up.
When you're done fixing everything, remove the vendored version of rails and call it a day.
my_project $ rm -rf ./vendor/rails
