Winning a Battle with Turbolinks

Introduction

Turbolinks is a topic that creates strong feelings within the Rails community. You either love it or take it out of your Gemfile. I have thrice tried to get turbolinks working on different rails apps and three times failed. Finally on the fourth try, I won, it is now working on a medium sized app. So why did I succeed this time? Basically because I took a different attitude. Below I detail some of the steps needed to work with Turbolinks, starting with a change of attitude.

Change your Attitude

In my past efforts, I allowed a couple of days to get Turbolinks working. I would quickly try things, google stackoverflow and try the first few answers. After passing all tests, I would deploy to production, and then find that most of the site worked, but some things did not. At this point I would fiddle a bit more, try a few time more, then give up.

This time, I decided that working on Turbolinks was a great opportunity for me to learn. I love ruby but only have a basic familiarity with coffeescript/javascript. My knowledge of how a browser loads a page loading and browser developer tools was strictly on a need to know basis. In other words, I had just enough to do some simple coffeescript enhancements to the web page. I realized that this was not enough to get Turbolinks working on a legacy app. Accordingly, my change of attitude was to work on Turbolinks in an effort to improve my understanding of how page loading, browser developer tools, coffeescript, and improve the overall quality of my code. I decided to spend a week on this project knowing that even if I did not end up using Turbolinks, I would have improved by abilities as a developer. This is the attitude I recommend.

Understand how Turbolinks works

At a high level, Turbolinks works by intercepting clicks on links. Normally when you click on a link, the page's head and body are loaded. In a rails app, the head points to the javascript and css files used by the page, and the body has the html that describes the page content. If you are using a frameworks like jquery and bootstrap, the javascript and css files can be very large, and take a long time to load.

When Turbolinks is enabled, things happen differently. First, if you have already visited that page, then the old version of the page is loaded from cache. This means that for a static page, the load is almost instantaneous. If the page's content has not changed since last time it was loaded, then the page appears instantaneously, though there is a refresh event a little later. If the page's content has changed, loading from cache gives the impression that progress is being made quickly, and in a lot of cases, by the time the viewer has focused on the relevant part of the new page, all the changes are done. The results are dramatic unless you have already spent a lot of time optimizing your page loads.

That is the quick story. The official Turbolinks Site has an excellent README file. It is well worth reading, but it is written by experts, so it assumes a reasonable level of knowledge. So read the earlier parts carefully, scan the api section, and be prepared to go back many times as you gradually increase your understanding and so are able to understand better different aspects of the README.

Another site worth scanning before stating work is the Turbolinks Compatability project. This lists the compatibility status for many popular web page inserts, such as Google Analytics and Facebook. You need to scan this to see if there are any show stoppers. For instance, Google Adsense has canceled support for AJAX requests, so it cannot be compatible with Turbolinks. See the section below for more advice on this.

Get Started

The instructions here assume that you are using jQuery, and are based on the excellent GO RAILS tutorial.

Add gem 'turbolinks', '~> 5.0.0' to your gemfile.
Add gem 'jquery-turbolinks' to your gemfile
Type bundle install
Add the following to your app/assets/javascripts/application.js {% highlight javascript %} //= require jquery.turbolinks //= require turbolinks {% endhighlight %} Note that the order is important.
Unfortunately, Turbolinks 5 does not play well with jQuery out of the box, but in the Turbolinks 5 repository there is some code that translates turbolinks events so that jQuery will work. Copy the compatability file to app/assets/javascripts/turbolinks-compatibility.coffee and you should be ready to start testing Turbolinks in your app.

The Set up for Testing Turbolinks

I am a 'write tests before code' sort of person, but I did not approach Turbolinks installation that way. My experience is that Turbolinks can act differently between the test, development and production environment. If it works in development or testing it may not work in production. Also in my experience even a local production server may not work the same way as a production environment on Heroku (you will see one reason for this later). Now one approach is to spend a lot of time making all these environments respond to Turbolinks in the same way, but if you are not an expert with Turbolinks, then that is not productive at this point. After all, if you are an expert with Turbolinks you would not be reading this blog.

My approach was to work mainly in development but to do periodic pushes to a staging server on Heroku. Given I was not using my usual approach of test driven development, I took note of the sort of manual tests that I was doing. Essentially I developed a list of the tests to make sure all of my coffeescript code still worked with Turbolinks. With each test, you need to take three actions;

Go to the page, refresh it with the browser reload button and then test the code.
Click away from the page and then back to the page and then test the code.
Click away from the page, press the browser back button and then test the code

It can be quite time consuming to find which page uses a particular coffeescript function, so I developed a special page that had code that needed key features of my coffeescript code to work. That way I could quickly checkoff most key features.

Triggering Your Code

When you test your code in develoment, you will probably find that your coffeescript features work when you refresh the page, but will not work when you click away from the page and then click on a link on another page that goes back to original page. When you click back, you are visiting the page via Turbolinks and your coffeescript is not triggering. If you look at your coffeescript it will either start with $ -> or wrapped with a document.ready. This is what causes your coffeescript to run when the page is ready for it. The trouble is that when the page is visited by Turbolinks, the document.ready event is not triggered. However, Turbolinks does have its own turbolinks:load. This even should trigger when the page is first loaded, but as well it works when the page is visited via Turbolinks. So instead of wrapping your code in document.ready, you should do the following

  
     func ->
      # your code goes here
     $(document).on('turbolinks:load', func)

In words, put your code into a function called func (or whatever you it) and call it when the turbolinks:load is triggered. This event is triggered when the page is refreshed, when you click to the page from another page in the app, or you get to the page by the browser back or forward function.

The good news is that you have Turbolinks integrated into your app and you are able understand what is going on with developer tools. The bad news is that some of your coffescript is now broken. You are now ready to start debugging.

Become familiar with a Browser Developer Tools

Trying to debug coffeescript without using the developer tools is like trying to debug ruby code without being able to see the development.log and not having pry or the rails console. But the developer tools provide even more; with the Network tab, you can gain much greater insight into how and when the page is being loaded. Using this tab, you can work out if Turbolinks is loading the page and what it is loading. If Turbolinks is loading the page, the console is not cleared and the key elements will be loaded with xhr requests. If you are not familiar with all this, find a tutorial on the different aspects of how a page loads, and simultaneously use the Network tab of the browser development tools to what is actually happening in your app. I use the Chrome developer tools, but the Safari tools seem to have similar functionality. Firefox does have developer tools but I am not as familiar with them.

Idempotent is not just a Fancy Word

By far the most common problem with Turbolinks that I have seen is failures because the coffeescript or javascript function is not idempotent. This can be caused by your own code or by plugins that you are using. Idempontent just means that if a function is applied more than once, the result is the same when it was applied the first time. In general, idempotence is a robust attribute for code, so although it is painful to track fix these idempotent problems, it does improve the quality of your code. Lets take a few examples and see how to fix them.

I find the Morris.js plugin is a great way to add attractive graphs to an app. The code to apply a graph looks something like this:

  
   graph ->
    Morris.Line
    element: 'progress_chart'
    data: window.progress
   $(document).on('turbolinks:load', func)

and in the pages's html there is be a div with an id of 'progress_chart'.

When the page is refreshed, the graph function is triggered and a graph is displayed in the div with id of progress_chart using a svg element, but when you click to the page from another page, a second graph is displayed. What is happening is that when you visit the page using Turbolinks, the document is not cleared, so the first graph is still there, but the turbolinks:load is still triggered, so a second identical graph is added. If you visit a third time, a third graph is added and so on. I hope you can see now why it is important to have your code idempotent when using Turbolinks. What you want is that no graph is added if there is already one there.

The solution is to make the code idempotent using a conditional that checks if there is already a svg element there

  
    func ->
     pid = $('#progress_chart')
     svg_missing = (pid.length > 0) && (pid.find('svg').length == 0)
      if svg_missing
      graph ->
       Morris.Line
        element: 'progress_chart'
        data: window.progress
    $(document).on('turbo:load', func)

Sometimes it is not possible to use this approach because it is not easy to test for the change. In this case, you can mark the target element with a data attribute to indicate that the function as already operated on the element. Then the conditional checks if the target element has been marked. Here is an example.

  
   example ->
    if $('body').attr('data-pages-loaded') == 'T'
    return
    # code that operates on on div#target
    $('body').attr('data-code-loaded','T')
   $(document).on('turbolinks:load', example)

with a target element that looks like this

    
   <div id="target"></div>

Once this code has run once, then the target div will look like this;

  
   <div id="target" "data-code-loaded='T'" ></div>

so the second time, the code will not be run because the data-code-loaded attribute is detected.

Selectively Disabling Turbolinks

For some reason, you may decide to disable Turbolinks on some pages. In the case of the project I was working on, there are Adsense ads on the public pages. As discussed above, Adsense is not compatible with Turbolinks. However, I still felt it was worthwhile proceeding with turbolinks for the performance boost to members who do not see ads when they are signed in. This can be done by marking a link with data-turbolinks="true". On my past attempts I had done this on a case by case basis, but it is messy and very easy to leave out links. The way I solved it this time was by overloading the link_to view helper.

  
  def link_to(*args)
    if turbolinks_white_list.include? args[1]
      super
    elsif no_turbolinks_if_ads
      h = {data: {turbolinks: "false"}}
      args[2] = args[2].try(:deep_merge, h) || h
      super
    else
      super
    end
  end

This allowed me to confine almost all changes to this single location. The whitelist was for public pages that displayed no ads, so it might look like ['/','/sign_up','/sign_in'], and the no_turbolinks_if_ads method was true if a page had ads displayed on it. I did carefully consider the wisdom of overloading such an important helper as link_to. However, it is being overloaded in a manner that the overloading is immediately apparent as a data-turbolinks="true" will appear in the link on the page, i.e. the overloading is not causing subtle changes to behaviour.

Other Issues

Some code is inherently difficult to make idempotent. For instance a function that is a toggle has to be involutary, applying it twice brings it back to the original stage. A quick work around is to use stopImmediatePropagation(). This stops immediate propagation, so for instance, the code stopped the class from being toggled twice.
```
  
$(document).on "click","#menu-toggle", (e) ->
  e.preventDefault()
  e.stopImmediatePropagation()
  $("#wrapper").toggleClass("toggled"
```
If you are using global variables stored on the window object, then these persist when you click between pages. If your code does not assume this to be the case, then this can cause problems. A solution to this is on stackoverflow.
No doubt you may have other issues. If you cannot find a fix on stackoverflow, it is worth looking carefully through the Turbolinks issue tracker. Also bear in mind sometimes bug you are having will affect very few users, because of the sequence needed to trigger it, and can be overcome by the user refreshing the page. In cases like this you might just post a question on stackoverflow and either live with it or disable turbolinks for that page.

Test Environment

Now you have your code working, you should try your test suite. Unfortunately, you are likely to see quite a few errors. The most common reason for this is that page has not fully loaded before you carry out your test. This is a general problem with Capybara, but Turbolinks seems to exacerbate it.

The obvious solution is to wait for a unique element to load on a page, but difficult to implement if you have a large test suite, because you have to find a unique element on each page. If you are a graduate from the Michael Hartl school of rails there is an easy way to solve this. Hartl teaches setting a different @title instance variable for every controller action. This is then used in application.html.erb to set an html title element for every page.

I certainly do, and it allowed a very easy way to fix up all my flaky tests that failed because the page had not loaded properly. You cannot find the title tag because it is not visible. So what I did was put at the bottom of the application.html.erb, the following code;

  
  <%= content_tag :div, "[#{title}]", class: 'rspec_eyes_only' if Rails.env.test? %>

This inserts, if you are in test mode, a div with title text, surrounded by square brackets. Then I wrote a helper that goes in rails_helper.rb

  
  def wait_for_page_load(title)
    find('div.rspec_eyes_only', text: title)
  end

I already has another helper to test for the correct title, so I modified it to add in wait_for_page_load i.e;

  
  def title_is_correct(title)
    full_title = "#{BaseTitle} | #{title}"
    wait_for_page_load "[#{full_title}]"
    expect(page.title).to eq full_title
  end

These simple changes cleared up almost all of my flaky tests, even the ones that were being caused by Turbolinks.

Difficulties in Production

If you are using a CDN then make sure that you have not enabled a feature that can interfere with your Javascript. For instance there is a documented issue with Cloudflare's rocket loader. The simplest workaround is to turn the rocket loader off.
There is a known issue with using async: true. A workaround is to use defer: true.