SimpleCov With Ruby And GitHub Actions

The SimpleCov gem can be used in conjunction with RSpec to easily denote the test coverage that we have with our code.

It is a useful way to visualize how much of our code has been covered by the tests that we write.

In this tutorial, we will implement the gem and then demonstrate how we can setup a GitHub Action that will also add comments to a pull request on what the coverage looks like.

Source code and final results can be found here.

Getting started

Let's start by create a folder to work in ruby-simplecov and adding some files:

# Create a new project
$ mkdir ruby-simplecov
$ cd ruby-simplecov

# Initialize the project and add the required gems
$ bundle init
$ bundle add rspec simplecov simplecov-lcov

# Add our required folders and files
$ mkdir lib
$ touch lib/contrived_math.rb

# Add in our test files
$ mkdir spec
$ touch spec/math_spec.rb spec/spec_helper.rb

# Prepare our GitHub Action file
$ mkdir -p .github/workflows
$ touch .github/workflows/rspec.yml
copy

Note that we added three required gems for this project:

At this stage, our project is now ready to start working with.

Seeing SimpleCov in action

Let's write a contrived math class and some tests that we can use for generating the coverage.

In lib/contrived_math.rb:

module ContrivedMath
  def self.add(a, b)
    a + b
  end
end
copy

In spec/math_spec.rb:

require 'spec_helper'
require 'contrived_math'

RSpec.describe Math do
  describe '#add' do
    it 'adds two numbers' do
      expect(ContrivedMath.add(1, 2)).to eq(3)
    end
  end
end
copy

Where the magic happens is in the spec/spec_helper.rb file.

require 'simplecov'
require 'simplecov-lcov'
SimpleCov::Formatter::LcovFormatter.config.report_with_single_file = true
SimpleCov.formatter = SimpleCov::Formatter::LcovFormatter
SimpleCov.start
copy

In this file, we effectively start SimpleCov with some configuration to add in the LcovFormatter as well as tell that formatter to report to a single file (by default it will do each file).

Run our test using bundle exec rspec and you will see a new coverage folder generated.

Inside, you will see a lcov folder where the output for a lcov formatter goes.

We can also see the results of our test coverage in a browser with open coverage/index.html:

Coverage

Awesome! Our coverage is generating locally. The next logical step is to implement our GitHub Action to comment the coverage on pull requests.

Code coverage comments on our GitHub pull requests

Setting up our GitHub action is quite trivial (after nutting out a couple of attempts).

The configuration that ended up working is the following:

name: Run RSpec tests
on: [pull_request]
jobs:
  run-rspec-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Set up Ruby
        uses: ruby/setup-ruby@v1
        with:
          # Not needed with a .ruby-version file
          ruby-version: 2.7
          # runs 'bundle install' and caches installed gems automatically
          bundler-cache: true
      - name: Run tests
        run: |
          bundle exec rspec
      - name: Add code coverage comment
        uses: romeovs/lcov-reporter-action@v0.2.16
        with:
          lcov-file: ./coverage/lcov/ruby-simplecov.lcov
          github-token: ${{ secrets.GITHUB_TOKEN }}
copy

In this GitHub action, we are telling GitHub to run our tests on a push to a pull_request.

I won't be explaining much about the first steps since I will infer you have some working knowledge of GitHub Actions, but the last two steps comprise of running the tests and then using the romeovs/lcov-reporter-action action to comment the results.

For the romeovs/lcov-reporter-action action, we are passing the path to the output file (which I figured out with some trial and error) and also passing our GITHUB_TOKEN in order to be able to comment.

At this point, this is all that we need!

Create a remote repo and push the code. Afterwards, let's create a new branch for our PR.

# Initialise a new git repo
$ git init

# I added `coverage` to .gitignore beforehand
$ git add --all
$ git commit -m "feat: init commit"

# I will infer that you have created your GitHub repo
$ git remote add origin https://github.com/okeeffed/ruby-simplecov.git

# Push the contents
$ git push

# Create new branch
$ git branch pr-example
$ git checkout pr-example
copy

At this point, we should be in our pr-example branch. Let's just update some code for the sake of creating a PR.

In lib/contrived_math.rb:

module ContrivedMath
  def self.add(a, b)
    a + b
  end

  def self.subtract(a, b)
    a - b
  end
end
copy

In spec/math_spec.rb:

require 'spec_helper'
require 'contrived_math'

RSpec.describe Math do
  describe '#add' do
    it 'adds two numbers' do
      expect(ContrivedMath.add(1, 2)).to eq(3)
    end
  end

  describe '#subtract' do
    it 'subtracts two numbers' do
      expect(ContrivedMath.subtract(2, 1)).to eq(1)
    end
  end
end
copy

At this point, commit this code and push it to GitHub.

$ git add --all
$ git commit -m "feat: add subtract functionality"
$ git push
copy

Inside of the repo, open a new PR and fill out the required fields (I won't be illustrating how to do this here).

Once you open the PR, you will see that our action Run RSpec tests is running!

Our GitHub action is running

It took me a couple of goes in my example (thanks to mucking around to see what works), but once your action completes successfully you will see a new comment added to the PR from github-actions that will have our coverage!

Coverage report

Success! We now have our coverage successfully shared for other reviewers to see.

Summary

Today's post demonstrated how to implement the SimpleCov gem into our RSpec tests to generate coverage.

We followed up on that by creating a GitHub Action to comment our coverage when we have open pull requests.

Resources and further reading

Photo credit: still_loony