🚤

Using Redis Hashes for Caching in Ruby on Rails

Mirror of this post on engineering.mrsool.co, co-authored by: Alex Golubenko

Note: We created a Ruby gem to easily share the lessons we’ve learned in this article.

We’ve been using Redis with Ruby on Rails’s caching methods for a couple of years at Mrsool and it’s been a great experience. We have particularly been using a function called #delete_matched to delete keys starting with a certain prefix. Let's take a look at an example:

Let’s imagine we have SQL table called stores and each store has many branches, and we need to show store branches near a certain user using their latitude/longitude (with varying degrees of precision), but we have millions of users in many places around the world, and we don’t want to overload our database with the same query for users in the same area, so let's cache this value per area.

def nearest_branches(store_id:, latitude:, longitude:)
  Rails.cache.fetch("nearest_branches_#{store_id}_#{latitude}_#{longitude}") do
    # Logic to get nearest open branches
  end
end

Whenever certain kinds of updates happen to stores or branches or a certain amount of time passes, we cleared out all the caches for a specific store using #delete_matched.

This worked well for a long time but as the amount of content we had stored in Redis grew, this function kept performing noticeably slower. At some point we also had to move from a single Redis node to a cluster, and we found out that #delete_matched is only operating on a single node, and not the entire cluster.

After doing some research, we found that we weren’t the only ones who were facing these issues at scale, and we found a data type in Redis that’s a great solution to our problems: Redis Hashes. Compared to #delete_matched, deleting a hash is much faster and there is no risk of leaving out undeleted data across multiple nodes.

It’s very similar to Ruby hashes:

{ hash_key => { sub_key => value } }

Unlike Ruby, hashes in Redis are flat, i.e. we can only have one level of sub-keys and it is not possible to create deeply nested structures.

From the structure above we can see the answer to our question of how to solve the problem of #delete_matched:

store_nearest_branches_hash_key = "nearest_branches_#{store_id}"branches_in_area_a_key = "#{latitude}_#{longitude}"branches_in_area_b_key = "#{latitude2}_#{longitude2}"

# Which can be cached like so:
redis = Rails.cache.redis

redis.hset(store_nearest_branches_hash_key, branches_in_area_a_key, branches_in_area_a)

redis.hset(store_nearest_branches_hash_key, branches_in_area_b_key, branches_in_area_b)

# To clear the cache for some store, all we need to do is delete its hash:redis.del(store_nearest_branches_hash_key)

but it’s not that simple: How can we store objects like ActiveRecord models in the Hash? (by default it’s only possible to store strings)

I will try to show you how to do it. First of all, let’s create a wrapper class/module for the Redis client that will operate on hashes. I decided to put it in lib, but it can be a service object instead — it’s up to you.

# lib/redis_hash_store.rb
module RedisHashStore
  extend self

  def write(hash_key, sub_key, value)
    redis.hset(hash_key, sub_key, value)
  end

  def read(hash_key, sub_key)
    redis.hget(hash_key, sub_key)
  end

  def delete(hash_key, sub_key)
   redis.hdel(hash_key, sub_key)
  end

  def delete_hash(key)
    redis.del(key)
  end

  private

  def redis
    Rails.cache.redis
  end
end

Now it’s a bit easier to work with hashes:

store_nearest_branches_key = "nearest_branches_#{store_id}"

branches_in_area_a_key = "#{latitude}_#{longitude}"

RedisHashStore.write(store_nearest_branches_key, branches_in_area_a_key, value)

RedisHashStore.read(store_nearest_branches_key, branches_in_area_a_key)

RedisHashStore.delete(store_nearest_branches_key, branches_in_area_a_key)
RedisHashStore.delete_hash(store_nearest_branches_key)

We still can’t store objects yet, let’s change that! After some investigations of ActiveSupport source I decided to use a lighter version of their implementation:

class Entry
  attr_reader :value

  def initialize(value, expires_in:)
    @value = value
    @created_at = Time.now.to_f
    @expires_in = expires_in
  end

  def expired?
    @expires_in && @created_at + @expires_in <= Time.now.to_f
  end
end

Unfortunately, Redis doesn’t have expire for Redis Hashes, so we have to implement our own expired? function.

Let’s move back to our RedisHashStore, which now looks like this:

module RedisHashStore
  extend self

  class Entry
    attr_reader :value

    def initialize(value, expires_in:)
      @value = value
      @created_at = Time.now.to_f
      @expires_in = expires_in
    end

    def expired?
      @expires_in && @created_at + @expires_in <= Time.now.to_f
    end
  end

  def write(hash_key, sub_key, value)
    redis.hset(hash_key, sub_key, value)
  end

  def read(hash_key, sub_key)
    redis.hget(hash_key, sub_key)
  end

  def delete(hash_key, sub_key)
    redis.hdel(hash_key, sub_key)
  end

  def delete_hash(key)
    redis.del(key)
  end

  private

  def redis
    Rails.cache.redis
  end
end

Wait a minute, we missed something here! Now we need to somehow convert Entry to a String and then safely move it back to the object. For this purpose we can use Marshal.

In our case we need 2 methods: #dump and #load, let’s add it to our RedisHashStore:

module RedisHashStore
  extend self

  class Entry
    attr_reader :value

    def initialize(value, expires_in:)
      @value = value
      @created_at = Time.now.to_f
      @expires_in = expires_in
    end

    def expired?
      @expires_in && @created_at + @expires_in <= Time.now.to_f
    end
  end

  def write(hash_key, sub_key, value, **options)
    entry = Entry.new(value, expires_in: options[:expires_in])
    redis.hset(hash_key, sub_key, serialize_value(entry))

    entry.value
  end

  def read(hash_key, sub_key)
    entry = deserialize_value(redis.hget(hash_key, sub_key))

    return if entry.blank?

    if entry.expired?
      delete(hash_key, sub_key)

      return nil
    end

    entry.value
  end

  def delete(hash_key, sub_key)
    redis.hdel(hash_key, sub_key)
  end

  def delete_hash(hash_key)
    redis.del(hash_key)
  end

  private

  def serialize_value(value)
    Marshal.dump(value)
  end

  def deserialize_value(value)
    return if value.nil?

    Marshal.load(value)
  end

  def redis
    Rails.cache.redis
  end
end

What about Benchmarks?

indexes = 1..1_000_000

indexes.each do |index|
  Rails.cache.write("some_data_#{index}", index)
  RedisHashStore.write("some_data", index, index)
end

Benchmark.bm do |x|
  x.report("delete_matched") {
    Rails.cache.delete_matched("some_data_*")
  }
  x.report("delete_hash") { RedisHashStore.delete_hash('some_data') }
end

# user             system      total        real
# delete_matched  0.571040   0.244962   0.816002 (3.791056)
# delete_hash     0.000000   0.000225   0.000225 (0.677891)# Machine info

# OS: macOS Big Sur
# Processor: 2,6 GHz 6-Core Intel Core i7
# Memory: 16 GB 2667 MHz DDR4
#
# Runned on
# OS: Docker Desktop (linux)
# Architecture: x86_64
# CPUs: 4
# Total Memory: 4.827GiB

Thanks for reading through this article, and don’t forget to check out our Ruby gem which includes all of the problems we’ve solved in this article and more.