Reflections on a Failed Upgrade

On a bright sunny day, with a sweeping view of beautiful Port Stephens, I made a terrible mistake. Instead of playing golf, swimming, or sitting in a local restaurant sipping a good wine from a nearby Hunter vineyard, I decided to upgrade to Rspec 3. I know, I know, but what true developer can refuse the opportunity to upgrade their tools when they have downtime?

Of course, I did fall for the siren call of an easy victory. The Relish upgrade instructions do say 'but our hope is to make this the smoothest major-version gem upgrade you've ever done'. I had recently done the Rails 4 upgrade, and that went pretty smoothly, so I thought that an Rspec 3 upgrade would be the perfect thing for a vacation hack. Moderately interesting but not stressful. It will only take me an hour or two. Little did I know...

My target app was The Golf Mentor. This is a few years old, has survived from Rails 2, and has about 2,000 tests that take 6 minutes to run. It was using Rspec 2.14, and in general I like to keep aligned with current releases, so Rspec 3 seemed a good idea.

The upgrade process has been well thought through. It involves first upgrading to rspec 2.99. All your tests should pass at this point, but there will be a long list of deprecations. You can then wave a magic wand called transpec to upgrade almost all your syntax to rspec 3. At this stage all your tests should work, and you might have a few deprecation warnings. You then fix the deprecations, upgrade to rspec 3, and Bobs your Uncle, all tests should pass. I imagine that for most lucky people, this is what happens.

The troubles first started on rspec 2.99, all my tests failed. But I was not worried, generally when all the tests fail at once there is a single failure point. The error message was undefined method `example' for #.

I tracked this down to this code in my rails_helper.rb file:

  
    if example.example.metadata[:js] 
      DatabaseCleaner.strategy = :truncation
    else
      DatabaseCleaner.strategy = :transaction
    end
  

I could not remember why I had inserted this code, so I just went to the database_cleaner github readme and pasted in the example rails code from there. My first big mistake.

I ran rspec spec and it started off well, green dot after green dot, then one red F, then nothing. I waited, still nothing. I hit one ^C, then another one, then another, but the three of them hung there - ^C^C^C, like tear drops waiting to run down my monitor's cheek. I terminated the terminal window, opened a new window and typed rspec rails. Now there was not even any green dots, it just hung from the get go. There must have been some rspec process still running. The only way I could fix it was to reboot my system. I then had to experiment, trying to find which test was causing the problem. Each time I tried something I had to reboot my system.

I eventually tracked the problem down to my feature specs that used javascript. When a bit more searching on the web, I found the correct way to set up database_cleaner for rspec 3 was:

  
    config.before(:suite) do
      DatabaseCleaner.clean_with(:truncation)
    end 
    config.before(:each) do
      DatabaseCleaner.strategy = :transaction
    end
    config.before(:each, :js => true) do
      DatabaseCleaner.strategy = :truncation
    end
    config.before(:each) do
      DatabaseCleaner.start
    end
    config.after(:each) do
      DatabaseCleaner.clean
    end
  

So with a bit more work, I had all my tests passing and was ready for the next step, which was to let transpec do its magic. And indeed it worked well. There were only a few problems, which I fixed relative quickly. Now I had a fully green test suite and no deprecation warnings, so I was ready for the next step, which was to upgrade to rpsec 3. The upgrade guide said that at this stage 'If anything fails, please open a Github issue'. Wow, if they are that confident, then my tests must pass, right?

Wrong, a wave of red Fs amongst placid green dots, a turbulent sea of trouble. There not one cause, there were many. Most of them seem to be connected to selectors, e.g.

  
    expect(response).to have_selector("title", content: "Unsubscribe Newsletter")
  

I searched the issues on rspec-rails github repository, and did find mention. Someone had raised the issue, but was told th at it was a capybara-webrat issue, and to stop using the webrat gem. The issue was then closed. In a nutshell, rspec 3 no longer supported webrat and capybara 2.0 did not support request specs.

In a sense the advice was good, but the solution was not easy. It was not a few copy and pastes. I had used many different selectors with different formats. Worse still, I needed to convert my hundreds of request spec tests to feature tests. The real difficulty is that even if the syntax is the same, capybara is more fussy. For example:

fill_in :email, with: '[email protected]' works with webrat, even though the id of the input is user_email presumably because the text was 'Email'. Capybara would not accept the inaccuracy. From a general viewpoint this is good, but from the viewpoint of upgrading it meant that I would have many, many test failures.

I was already up to 6 hours, with no golf and no wine, I could see days and days of tedious editing and debugging to complete the upgrade. It was too much for me to face, I gave up. Actually I rationalized it as I was now working on a phased upgrade strategy. I resolved to write feature specs instead of request specs for any further integration tests, and when I did any work on old request spec, I would convert it to a feature spec. Above all, I would wait and hope for the day that someone clever writes a trans_webrat_to_capybara gem. The webrat gem has not been updated since 2011, but there have been 1.6 million downloads and it is still being downloaded at the rate of 5,000 a month. There must be a few other outcasts around.

By the way, none of this is a criticism of the rspec team, in the ruby on rails ecosystem, you can only work your part of it, otherwise the scope becomes too wide.

So I have wasted 6 hours, or 7 if you count the time writing this lament. Today is raining here, so I have decided to use rubocop to clean up all my syntax. There is a neat to-do procedure and an auto-correct feature, so it should only take one or two hours...

Author: Dr. Chris Drane
Stack Overflow: Obromios
Date: August 8, 2019
Feedback: We welcome feedback, please contact us.