BullMQ on PostgreSQL
Since BullMQ v6, the same Queue, Worker, QueueEvents
and FlowProducer API runs on PostgreSQL instead of Redis, so you don't need to
run Redis just for your jobs.
PostgreSQL is a storage backend, not another library: you keep using BullMQ for your language and choose where your jobs are stored.
What you need
PostgreSQL 13 or newer
BullMQ works with PostgreSQL 13 and newer. PostgreSQL 14 or newer is recommended.
Its own schema
All BullMQ tables live in their own schema, bullmq by default, separate
from the rest of your database.
Your first queue on PostgreSQL
PostgreSQL is supported in Node.js, Python, Elixir and .NET today.
Node.js
npm install bullmq pgPass createPostgresBackend as the last constructor argument. In Node.js the schema is not migrated automatically: pass migrate: true on the connection, as below, or call runMigrations() from a deployment step.
import { Queue, Worker, createPostgresBackend } from "bullmq";
const opts = {
connection: {
connectionString: "postgres://localhost:5432/mydb",
migrate: true,
},
};
const queue = new Queue("Paint", opts, createPostgresBackend);
await queue.add("cars", { color: "blue" });
const worker = new Worker(
"Paint",
async (job) => paintCar(job.data.color),
opts,
createPostgresBackend,
);Python
pip install "bullmq[postgres]"The postgres extra installs psycopg. Set "backend": "postgres" in the options. The schema is migrated automatically when BullMQ connects.
import asyncio
from bullmq import Queue, Worker
opts = {"backend": "postgres", "connection": "postgres://localhost:5432/mydb"}
async def main():
queue = Queue("Paint", opts)
await queue.add("cars", {"color": "blue"})
async def process(job, token):
return await paint_car(job.data["color"])
worker = Worker("Paint", process, {**opts, "autorun": False})
await worker.run()
asyncio.run(main())Elixir
{:postgrex, "~> 0.22"}Add postgrex next to bullmq, start a connection and pass it with backend: BullMQ.Backends.Postgres. You can also set the backend for the whole application with config :bullmq, :backend, BullMQ.Backends.Postgres. The schema is migrated automatically when the connection starts.
{:ok, _conn} =
BullMQ.Backends.Postgres.Connection.start_link(
name: :my_pg,
url: "postgres://localhost:5432/mydb"
)
{:ok, _job} =
BullMQ.Queue.add("Paint", "cars", %{color: "blue"},
connection: :my_pg,
backend: BullMQ.Backends.Postgres
)
{:ok, _worker} =
BullMQ.Worker.start_link(
queue: "Paint",
connection: :my_pg,
backend: BullMQ.Backends.Postgres,
processor: &MyApp.Painter.process/1
).NET
dotnet add package BullMQSet Postgres on the queue and worker options. The schema is migrated automatically on first use.
using System.Text.Json;
using BullMQ;
using BullMQ.Postgres;
var pg = new PostgresOptions
{
ConnectionString = "Host=localhost;Database=mydb;Username=postgres"
};
await using var queue = new Queue("Paint", new QueueOptions { Postgres = pg });
await queue.AddAsync("cars", new { color = "blue" });
await using var worker = new Worker("Paint", async (job, token) =>
{
var data = (JsonElement)job.Data!;
await PaintCar(data.GetProperty("color").GetString());
return null;
}, new WorkerOptions { Postgres = pg });Redis or PostgreSQL?
PostgreSQL
One service less to run, transactional durability, and your jobs live next to your relational data.
Redis
Higher throughput and lower latency.
The API is the same on both backends, so you can start with one and switch later without changing your queue and worker code. Jobs that are already queued stay in the backend where they were added. The PostgreSQL guide covers connection options, migrations, performance and tuning.
Questions about the PostgreSQL backend
Can I use BullMQ without Redis?
Yes. Since BullMQ v6 you can store your queues in PostgreSQL 13 or newer instead of Redis. The Queue, Worker, QueueEvents and FlowProducer API is the same, so you don't need to run Redis just for your jobs.
Is BullMQ on PostgreSQL a separate library?
No. PostgreSQL is a storage backend, not another library or runtime. You install the same BullMQ package for your language and choose the backend when you create your queues and workers. The PostgreSQL backend is available in Node.js, Python, Elixir and .NET, and is coming soon to Rust and PHP.
Which BullMQ features work on PostgreSQL?
Queues, workers, flows, job schedulers, rate limiting, priorities, delayed jobs, deduplication, metrics and events all work on PostgreSQL with the same code. Only things that are specific to Redis, such as access to the raw Redis client or Redis Cluster settings, do not apply.
Is BullMQ slower on PostgreSQL than on Redis?
Adding jobs one at a time and in bulk is close to Redis. Processing throughput is usually around 1.5 to 2 times lower, because PostgreSQL writes every state change durably and Redis works in memory. The numbers depend on your hardware and on how close the database is to your workers, so benchmark with your own workload.
Can I switch a queue from Redis to PostgreSQL?
Yes. The API is the same on both backends, so your queue and worker code stays the same and only the backend option changes. Jobs that are already queued are not moved: they stay in the backend where they were added.
Does BullMQ Pro run on PostgreSQL?
Not yet. BullMQ Pro currently requires Redis.