Getting Started With Sidekiq In Rails 7

Sidekiq makes it easy to run efficient background jobs in Ruby. It is amazingly useful in your Rails projects for jobs that require processing.

This post will cover a basic example of running jobs with Sidekiq and Redis.

Source code can be found here.

Prerequisites

Getting started

We will use Rails to initialize the project demo-hello-sidekiq:

# Create a new rails project
$ rails new demo-rails-with-react-frontend -j esbuild
$ cd demo-rails-with-react-frontend

# Create some required files
$ touch Procfile.dev config/initializers/sidekiq.rb bin/dev

# Add permissions to run bin/dev
$ chmod u+x bin/dev

# Add required gems
$ bundler add sidekiq

# Generate a controller route for our demo
$ bin/rails g controller jobs create
      create  app/controllers/jobs_controller.rb
       route  get 'jobs/create'
      invoke  erb
      create    app/views/jobs
      create    app/views/jobs/create.html.erb
      invoke  test_unit
      create    test/controllers/jobs_controller_test.rb
      invoke  helper
      create    app/helpers/jobs_helper.rb
      invoke    test_unit

# Generate a basic job
$ bin/rails g sidekiq:job hello
      create  app/sidekiq/hello_job.rb
      create  test/sidekiq/hello_job_test.rb
copy

With Sidekiq, we have some helpers to help generate our jobs with bin/rails g sidekiq:job <job_name>.

At this stage, we are ready to start updating and editing our code.

Setting up our development environment

Inside of Procfile.dev:

web: bin/rails s
worker: bundle exec sidekiq
copy

Inside of bin/dev:

#!/usr/bin/env bash

if ! gem list --silent --installed foreman
then
  echo "Installing foreman..."
  gem install foreman
fi

foreman start -f Procfile.dev "$@"
copy

This will enable us to run bin/dev to start both our Rails server and our Sidekiq process.

Updating our routes

Rails.application.routes.draw do
  resources :jobs, only: [:create]
end
copy

Updating our configuration file

We are going to turn off our default_protect_from_forgery in development so we can demonstrate creating the job with our CLI tools.

Update our config/application.rb to the following:

require_relative "boot"

require "rails/all"

# Require the gems listed in Gemfile, including any gems
# you've limited to :test, :development, or :production.
Bundler.require(*Rails.groups)

module DemoHelloSidekiq
  class Application < Rails::Application
    # Initialize configuration defaults for originally generated Rails version.
    config.load_defaults 7.0

        # Enable us to send requests without auth token
        config.action_controller.default_protect_from_forgery = false if ENV['RAILS_ENV'] == 'development'
  end
end
copy

Configuring our Sidekiq initializer file

Let's update our config/initializers/sidekiq.rb file to set up our server configuration. Note that this will only be for our local server URL.

# inside config/initializers/sidekiq.rb

Sidekiq.configure_server do |config|
  config.redis = { url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/1') }
end
Sidekiq.configure_client do |config|
  config.redis = { url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/1') }
end
copy

Writing our first job

Update what is in app/sidekiq/hello_job.rb:

class HelloJob
  include Sidekiq::Job

  def perform(*_args)
    # Do something
    p "HelloJob started with args #{_args}"

    # Sleep to simulate a time-consuming task
    sleep 5

    # Will display current time, milliseconds included
    p "HelloJob #{Time.now.strftime('%F - %H:%M:%S.%L')}"
  end
end
copy

Writing our controller to run a basic job

Finally, let's update our app/controllers/jobs_controller.rb:

class JobsController < ApplicationController
  def create
    HelloJob.perform_async('job', 5)
    render json: { message: 'Accepted' }, status: :accepted
  end
end
copy

Our simple job will take 5 seconds to complete and will log out the arguments we've passed to the job as well as the time the job completes.

Testing our job

Now we can start the server with bin/dev.

$ bin/dev
10:36:56 web.1    | started with pid 77188
10:36:56 worker.1 | started with pid 77189
10:36:58 web.1    | => Booting Puma
10:36:58 web.1    | => Rails 7.0.2.2 application starting in development
10:36:58 web.1    | => Run `bin/rails server --help` for more startup options
10:36:58 worker.1 | 2022-03-02T00:36:58.612Z pid=77189 tid=1r01 INFO: Booting Sidekiq 6.4.1 with redis options {:url=>"redis://localhost:6379/1"}
10:36:58 worker.1 | 2022-03-02T00:36:58.913Z pid=77189 tid=1r01 INFO: Booted Rails 7.0.2.2 application in development environment
10:36:58 worker.1 | 2022-03-02T00:36:58.913Z pid=77189 tid=1r01 INFO: Running in ruby 2.7.3p183 (2021-04-05 revision 6847ee089d) [x86_64-darwin20]
10:36:58 worker.1 | 2022-03-02T00:36:58.913Z pid=77189 tid=1r01 INFO: See LICENSE and the LGPL-3.0 for licensing details.
10:36:58 worker.1 | 2022-03-02T00:36:58.913Z pid=77189 tid=1r01 INFO: Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org
10:36:59 web.1    | Puma starting in single mode...
10:36:59 web.1    | * Puma version: 5.6.2 (ruby 2.7.3-p183) ("Birdie's Version")
10:36:59 web.1    | *  Min threads: 5
10:36:59 web.1    | *  Max threads: 5
10:36:59 web.1    | *  Environment: development
10:36:59 web.1    | *          PID: 77188
10:36:59 web.1    | * Listening on http://127.0.0.1:5000
10:36:59 web.1    | * Listening on http://[::1]:5000
10:36:59 web.1    | Use Ctrl-C to stop
copy

To start a job, we can make a POST call to http://localhost:5000/jobs:

$ http POST http://localhost:5000/jobs
HTTP/1.1 202 Accepted
Cache-Control: no-cache
Content-Type: application/json; charset=utf-8
Referrer-Policy: strict-origin-when-cross-origin
Server-Timing: start_processing.action_controller;dur=0.108642578125, process_action.action_controller;dur=2.332763671875
Transfer-Encoding: chunked
Vary: Accept
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Request-Id: 4b7ed09f-f449-42a6-abcd-6a2c8fb3487a
X-Runtime: 0.062683
X-XSS-Protection: 0

{
    "message": "Accepted"
}
copy

Our logs will show us that the web processes the controller, then the job is handled by the working (with job completion occurring 5 seconds later).

10:42:57 web.1    | Started POST "/jobs" for ::1 at 2022-03-02 10:42:57 +1000
10:42:57 web.1    | Processing by JobsController#create as */*
10:42:57 worker.1 | 2022-03-02T00:42:57.674Z pid=79840 tid=2264 class=HelloJob jid=2d01fd26fa36e584afe284a3 INFO: start
10:42:57 web.1    | Completed 202 Accepted in 2ms (Views: 0.3ms | Allocations: 425)
10:42:57 web.1    |
10:42:57 web.1    |
10:42:57 worker.1 | "HelloJob started with args [\"job\", 5]"
10:43:02 worker.1 | "HelloJob 2022-03-02 - 10:43:02.752"
10:43:02 worker.1 | 2022-03-02T00:43:02.752Z pid=79840 tid=2264 class=HelloJob jid=2d01fd26fa36e584afe284a3 elapsed=5.078 INFO: done
copy

Hooray!

Demonstrating other job methods

We also have the capability of using the perform_in and perform_at API methods to schedule jobs.

Update the controller to see this in action:

class JobsController < ApplicationController
  def create
    HelloJob.perform_async('perform_async job')
    HelloJob.perform_in(10.seconds, 'perform_in job')
    HelloJob.perform_at(20.seconds.from_now, 'perform_at job')
    render json: { message: 'Accepted' }, status: :accepted
  end
end
copy

Again, send a POST request to our /jobs routes results in the following:

10:51:38 web.1    | Started POST "/jobs" for ::1 at 2022-03-02 10:51:38 +1000
10:51:38 web.1    | Processing by JobsController#create as */*
10:51:38 worker.1 | 2022-03-02T00:51:38.630Z pid=79840 tid=2214 class=HelloJob jid=e38a834662f55fec2fc0c0a8 INFO: start
10:51:38 web.1    | Completed 202 Accepted in 46ms (Views: 0.3ms | Allocations: 622)
10:51:38 web.1    |
10:51:38 web.1    |
10:51:38 worker.1 | "HelloJob started with args [\"perform_async job\"]"
10:51:43 worker.1 | "HelloJob 2022-03-02 - 10:51:43.710"
10:51:43 worker.1 | 2022-03-02T00:51:43.710Z pid=79840 tid=2214 class=HelloJob jid=e38a834662f55fec2fc0c0a8 elapsed=5.08 INFO: done
10:51:55 worker.1 | 2022-03-02T00:51:55.054Z pid=79840 tid=2214 class=HelloJob jid=56a274fa9a9db51df8591674 INFO: start
10:51:55 worker.1 | "HelloJob started with args [\"perform_in job\"]"
10:52:00 worker.1 | "HelloJob 2022-03-02 - 10:52:00.067"
10:52:00 worker.1 | 2022-03-02T00:52:00.067Z pid=79840 tid=2214 class=HelloJob jid=56a274fa9a9db51df8591674 elapsed=5.013 INFO: done
10:52:02 worker.1 | 2022-03-02T00:52:02.205Z pid=79840 tid=221o class=HelloJob jid=2672ef81124e468da977f789 INFO: start
10:52:02 worker.1 | "HelloJob started with args [\"perform_at job\"]"
10:52:07 worker.1 | "HelloJob 2022-03-02 - 10:52:07.217"
10:52:07 worker.1 | 2022-03-02T00:52:07.217Z pid=79840 tid=221o class=HelloJob jid=2672ef81124e468da977f789 elapsed=5.012 INFO: done
copy

In our logs now we can see the 3 different jobs run at the expected intervals over the period of the 20 seconds that we expected them to be scheduled for.

The expected schedule is something you should keep in mind when writing your tests.

Summary

Today's post demonstrated how to run jobs in a Rails application with Sidekiq.

This was a simple overview to see a first job in action.

Moving forward, you should look into testing, distributed Redis configuration and more complex jobs that are not so contrived for the purpose of demonstration!

Jobs can be great for background processing such as sending emails, compressing images, or performing long-running tasks.

Resources and further reading

Photo credit: sir_moon