Benchmarking In Ruby

This post will demonstrate simple usage of Ruby's benchmarking tool from the standard library.

Benchmarking helps us to better understand the performance and impact of the code that we write.

Source code can be found here

Getting started

We will create the project directory ruby-benchmarking and use Bundler to initialize the project:

$ mkdir ruby-benchmarking
$ cd ruby-benchmarking
$ touch benchmark.rb iterate.rb main.rb
copy

We will write two methods in the iterate.rb file to compare against in our benchmark.rb file.

Writing our iteration methods

We are going to make this example as contrived as possible by comparing a function that makes use of each and map to add a new property to each object in an array.

Add the following to iterate.rb:

def add_property_one(arr)
  arr.each do |x|
    x.store('new_property', 'new_value')
  end
end

def add_property_two(arr)
  fn = ->(x) { x.store('new_property', 'new_value') }

  arr.map(&fn)
  arr
end
copy

At this point, we can check that the two methods produce the same results.

In main.rb, we will write a script that will run the functions and print the results:

require './iterate'

arr = [{ name: 'John', age: 23 }, { name: 'Jane', age: 18 }, { name: 'Joe', age: 21 }]
arr2 = arr.map(&:clone)

res1 = add_property_one(arr)
res2 = add_property_two(arr2)

puts "res1: #{res1}"
puts "res2: #{res2}"
copy

When we run this, we see that new_property with value new_value is added to the hash:

$ ruby main.rb
res1: [{:name=>"John", :age=>23, "new_property"=>"new_value"}, {:name=>"Jane", :age=>18, "new_property"=>"new_value"}, {:name=>"Joe", :age=>21, "new_property"=>"new_value"}]
res2: [{:name=>"John", :age=>23, "new_property"=>"new_value"}, {:name=>"Jane", :age=>18, "new_property"=>"new_value"}, {:name=>"Joe", :age=>21, "new_property"=>"new_value"}]
copy

At this point we are ready to run our benchmark.

Writing a benchmark check

In benchmark.rb, add the following code:

require 'benchmark'
require './iterate'

input = ('a'..'z').map { |letter| [[letter, letter]].to_h }
n = 1_000_000

Benchmark.bm do |benchmark|
  benchmark.report('add_property_one') do
    n.times do
      add_property_one(input)
    end
  end

  benchmark.report('add_property_two') do
    n.times do
      add_property_two(input)
    end
  end
end
copy

Some points on the above code:

I am running one million iterations just to accentuate the difference, but really could drop it down to about 50_000 or less depending on what is a more realistic benchmark for you.

Running the benchmark.rb file yields the following:

$ ruby benchmark.rb
        user     system      total        real
add_property_one  5.152850   0.024035   5.176885 (  5.251898)
add_property_two  6.112460   0.027414   6.139874 (  6.177714)
copy

The benchmarks come with four columns:

Wall clock time can be defined as the "real" amount of time it took to execute the code. This time includes system + user + time spent waiting for I/O, network, disk, user input, etc.

Looking strictly at the user times, we can ascertain that add_property_one is faster than add_property_two.

Summary

Today's short post demonstrated how to write a benchmark test for Ruby to compare the performance of two different methods.

This method may help you debug and benchmark improvements to your code in a quantifiable manner.

Resources and further reading

Photo credit: dnevozhai