Background jobs for Python

The official BullMQ library for Python. Async background jobs on Redis, using the same queues as your Node.js services.

pip install bullmq

Python 3.10+ ยท Redis 6.2+ (or Valkey/Dragonfly)

Your first queue and worker

my_queue.py
from bullmq import Queue

queue = Queue("Paint")
await queue.add("cars", {
"color": "blue",
"delay": 30000
})
my_worker.py
from bullmq import Worker

async def process(job, token):
if job.name == "cars":
await paint_car(job.data["color"])

worker = Worker("Paint", process, {
"concurrency": 100
})

The same job handling as the Node.js library

All BullMQ libraries run the same Lua scripts on Redis and the same SQL on PostgreSQL, so jobs are handled the same way in every language. Only the API around those scripts differs from one library to the next.

More Python examples

Common BullMQ features, written in Python.

Global events

events.py
from bullmq import QueueEvents

queue_events = QueueEvents("Paint")

async def on_completed(job, result):
print(f"Job {job['jobId']} completed")

async def on_failed(job, err):
print(f"Job {job['jobId']} failed")

queue_events.on("completed", on_completed)
queue_events.on("failed", on_failed)

Repeatable jobs

repeatable.py
from bullmq import Queue

queue = Queue("Paint")

# Repeat job once every day at 3:15 (am)
await queue.add(
"submarine",
{"color": "yellow"},
{
"repeat": {
"pattern": "* 15 3 * * *"
}
}
)

Rate limiting

ratelimit.py
from bullmq import Worker

async def process(job, token):
await paint_car(job)

worker = Worker("Paint", process, {
"limiter": {
"max": 10,
"duration": 1000
}
})

Retries & backoff

retry.py
from bullmq import Queue

queue = Queue("Paint")

await queue.add(
"car",
{"color": "pink"},
{
"attempts": 3,
"backoff": {
"type": "exponential",
"delay": 1000
}
}
)

Flows

flow.py
from bullmq import FlowProducer

flow = FlowProducer()

await flow.add({
"name": "Renovate",
"queueName": "cars",
"children": [
{"name": "paint", "queueName": "steps"},
{"name": "engine", "queueName": "steps"},
{"name": "wheels", "queueName": "steps"},
]
})

Using BullMQ with Python

Async API

Built on asyncio, so you can call it from FastAPI, Starlette or any other async Python code.

Coming from Celery

You get Redis-backed queues, flows and job schedulers, and your Node.js and .NET services can use the same queues.

Flows and schedulers

FlowProducer creates parent/child job trees and JobScheduler runs cron-style repeatable jobs. Both work like they do in Node.js.

Node.js Python

One queue, any language

Add jobs from a Node.js or PHP application and process them in Python workers. This is a common setup for machine learning and data processing.

What's available in Python

  • Add jobs to queues Supported
  • Process jobs (workers) Supported
  • Delayed jobs Supported
  • Repeatable / scheduled jobs Supported
  • Job priorities Supported
  • Rate limiting Supported
  • Retries & backoff Supported
  • Flows (parent / child) Supported
  • Job events Supported
  • Redis backend Supported
  • PostgreSQL backend Coming soon
  • BullMQ Pro Coming soon

BullMQ Pro for Python Coming soon

BullMQ Pro adds groups, observables and batches to the open source library. It is available for Node.js and Bun today, and support for Python is on the way.

Groups
Observables
Batches

Get started with BullMQ for Python