Announcing BullMQ for Elixir

BullMQ is now available for Elixir. Version 1.0.0 is published on Hex as bullmq, and it is a native implementation built with GenServers and Supervisors, not a wrapper around the Node.js library.

Add it to your dependencies in mix.exs:

def deps do
  [
    {:bullmq, "~> 1.0"}
  ]
end

Queues and workers in your supervision tree

Adding a job takes one call, and a worker is a process that you start under your application's supervisor like any other:

children = [
  {Redix, name: :my_redis, host: "localhost", port: 6379},
  {BullMQ.Worker,
   queue: "emails",
   connection: :my_redis,
   processor: &MyApp.EmailWorker.process/1,
   concurrency: 5}
]

Supervisor.start_link(children, strategy: :one_for_one)

{:ok, job} =
  BullMQ.Queue.add("emails", "send-welcome", %{to: "user@example.com"},
    connection: :my_redis
  )

The processor is a function that receives a %BullMQ.Job{} and returns {:ok, result} or {:error, reason}.

What is included

This first release covers what an application needs to run background jobs in production:

• BullMQ.Queue adds jobs one by one or in bulk, with priorities, delays, custom job IDs and retries with fixed, exponential or custom backoff.
• BullMQ.Worker processes jobs with configurable concurrency, automatic lock renewal, stalled job recovery and graceful shutdown.
• BullMQ.FlowProducer creates parent and child jobs, including nested flows.
• BullMQ.JobScheduler runs recurring jobs from cron expressions or fixed intervals.
• Rate limiting is a worker option that caps how many jobs a queue processes in a given period.
• BullMQ.QueueEvents streams the lifecycle events of the jobs in real time.
• Telemetry events are emitted for jobs and workers, so you can plug BullMQ into the observability tools you already use.

It requires Elixir 1.15 or newer, Erlang/OTP 26 or newer and Redis 6.0 or newer.

The same queues as Node.js and Python

BullMQ for Elixir runs the same Lua scripts as BullMQ for Node.js. Those scripts handle the critical state changes of a job, such as claiming it, completing it, retrying it and moving it between states, and they have been used in production for years. The Elixir library is new, but the logic that decides what happens to your jobs is not.

It also means that the queues are compatible with BullMQ 5 for Node.js and with BullMQ for Python. A job added from a Node.js service can be processed by an Elixir worker, and the other way around, with no adapter in between. You can add Elixir to a system that already uses BullMQ one service at a time.

Try it

The package is on Hex, with the API reference on HexDocs and a guide in the BullMQ documentation. The source lives next to the other implementations in the BullMQ repository, where you can also open an issue if you run into anything.

Update: BullMQ for Elixir is now at version 2 and can also store its queues in PostgreSQL. See BullMQ for Elixir for what it supports today.