Running Sidekiq On Heroku Free Dyno With Puma

In my opinion, Heroku is the nearly perfect place to deploy a project. It provides support for a lot of technologies out of the box with minimal or no configuration.

Similarly, Sidekiq is what I strongly prefer whenever there is a need for background job processing. It ships with all the right set of APIs to deal with background processing and more importantly a great monitoring dashboard.

So, Last weekend I decide to hack on a side-project which has been on my mind forever (I will write about this project in another blog post). I rolled up a new rails project, added Sidekiq with Redis and pieced together a basic functional prototype. When I moved to deployment process I realised - I have to run another dyno to keep the Sidekiq process running.

Now let me be honest with you here, I have an itch for creating side-projects. Once I get hit by a problem I take some time out to hack on it & later I struggle to find time to work on the project again. That's why I don't go with any paid plan in project's infant days.

Coming back to our problem at hand, Heroku already provides a free tier of Redis as an addon – which does

heroku addons:create heroku-redis:hobby-dev

The other half is to figure out how to spawn a Sidekiq process within the same dyno. After some digging I found a great blog post Sidekiq on Heroku with Redis To Go Nano describing how to achieve the same in Unicorn. From here is just a matter of figuring out how to replicate this behaviour in Puma.

...some experiments later...

I got the puma.rb inside config to mimic the same behaviour for me – keeping Heroku Redis' client connection limit in check. Here is the config:

threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }

threads threads_count, threads_count

port        ENV.fetch("PORT") { 3000 }

environment ENV.fetch("RAILS_ENV") { "development" }

workers ENV.fetch("WEB_CONCURRENCY") { 2 }

preload_app!

// Update by (Stefan)
before_fork do 
    @sidekiq_pid ||= spawn('bundle exec sidekiq -t 25')
end

on_worker_boot do
  ActiveRecord::Base.establish_connection if defined?(ActiveRecord)
end

on_restart do
  Sidekiq.redis.shutdown { |conn| conn.close }
end

plugin :tmp_restart

Copy this to your Puma config and deploy this with your Heroku app.

git push heroku master

Be sure to point Procfile to right puma config

bundle exec puma -C config/puma.rb

P.S: I would not recommend this setup at all for your production application. This is strickly for experimental purpose.

UPDATE:
Mike Perham (creator of Sidekiq) make some points on Reddit thread which you must checkout before moving ahead with this approach.

I have also updated the code snippet above based the findings (mentioned in the comments below) of Stefan Wrobel.