A JSON rabbit hole

I fell down a really curious rabbit hole the other day and thought I'd share the adventure. I was using a gem that handles a response from an API (pretty standard, right?).

The response was handled inside the gem like so:

def get(path, options = {})
  response = request(path, options)
  JSON.parse(response.body, :symbolize_names => true)
end

The :symbolize_names => true argument here is crucial. The response from the API would look something like "{\"princess\": \"bubblegum\"}", and when parsed using the :symbolize_names => true option, the keys in the returned hash would be symbols:

{ :princess => "bubblegum" }

All throughout the gem, symbols were used when accessing this response hash.

JSON vs Yajl-ruby

Long story short, a legacy version of Yajl-ruby (1.1.0) doesn't look for :symbolize_names in the opts hash of the parse method. Instead, it looks for :symbolize_keys. So passing along :symbolize_names => true has no effect.

Here's a nice example taken from an issue raised against Yajl-ruby for this release:

string = '{"bla": "blub"}'

require 'json'
JSON.parse string, :symbolize_names => true # => {:bla=>"blub"}
JSON.parse string, :symbolize_keys => true  # => {"bla"=>"blub"}

require 'yajl/json_gem'
JSON.parse string, :symbolize_names => true # => {"bla"=>"blub"}
JSON.parse string, :symbolize_keys => true  # => {:bla=>"blub"}

And obviously our project was using this legacy version of Yajl-ruby. When the parse method was called inside the gem, it did not symbolise the keys, and so nil was returned everywhere the hash was accessed.

  require 'yajl/json_gem'
  json_string = "{\"princess\":\"bubblegum\"}"
  hash = JSON.parse(json_string, :symbolize_names => true)
  # => {"princess"=>"bubblegum"}
  hash[:princess]
  # => nil

It looked like the gem was not working at all. So I dove into it, I ran the tests, I used it in the console; everything worked fine. Yet when I used the gem in our project no joy was had.

Eventually, (with the help of another pair of eyes) it was clear that the parse method was being handled differently in the legacy version of Yajl-ruby. Our fix was to patch the gem to send both :symbolize_names => true and :symbolize_keys => true, which seemed to do the trick for both Yajl-ruby 1.1.0 and the standard json gem.

def get(path, options = {})
  response = request(path, options)
  JSON.parse(response.body, :symbolize_names => true, :symbolize_keys => true)
end

Morning well spent? ✅