Background jobs for Python
The official BullMQ library for Python. Async background jobs on Redis, using the same queues as your Node.js services.
pip install bullmq
Python 3.10+ ยท Redis 6.2+ (or Valkey/Dragonfly)
Your first queue and worker
from bullmq import Queue
queue = Queue("Paint")
await queue.add("cars", {
"color": "blue",
"delay": 30000
})from bullmq import Worker
async def process(job, token):
if job.name == "cars":
await paint_car(job.data["color"])
worker = Worker("Paint", process, {
"concurrency": 100
})More Python examples
Common BullMQ features, written in Python.
Global events
from bullmq import QueueEvents
queue_events = QueueEvents("Paint")
async def on_completed(job, result):
print(f"Job {job['jobId']} completed")
async def on_failed(job, err):
print(f"Job {job['jobId']} failed")
queue_events.on("completed", on_completed)
queue_events.on("failed", on_failed)Repeatable jobs
from bullmq import Queue
queue = Queue("Paint")
# Repeat job once every day at 3:15 (am)
await queue.add(
"submarine",
{"color": "yellow"},
{
"repeat": {
"pattern": "* 15 3 * * *"
}
}
)Rate limiting
from bullmq import Worker
async def process(job, token):
await paint_car(job)
worker = Worker("Paint", process, {
"limiter": {
"max": 10,
"duration": 1000
}
})Retries & backoff
from bullmq import Queue
queue = Queue("Paint")
await queue.add(
"car",
{"color": "pink"},
{
"attempts": 3,
"backoff": {
"type": "exponential",
"delay": 1000
}
}
)Flows
from bullmq import FlowProducer
flow = FlowProducer()
await flow.add({
"name": "Renovate",
"queueName": "cars",
"children": [
{"name": "paint", "queueName": "steps"},
{"name": "engine", "queueName": "steps"},
{"name": "wheels", "queueName": "steps"},
]
})Using BullMQ with Python
Async API
Built on asyncio, so you can call it from FastAPI, Starlette or any other async Python code.
Coming from Celery
You get Redis-backed queues, flows and job schedulers, and your Node.js and .NET services can use the same queues.
Flows and schedulers
FlowProducer creates parent/child job trees and JobScheduler runs cron-style repeatable jobs. Both work like they do in Node.js.
One queue, any language
Add jobs from a Node.js or PHP application and process them in Python workers. This is a common setup for machine learning and data processing.
What's available in Python
- Add jobs to queues Supported
- Process jobs (workers) Supported
- Delayed jobs Supported
- Repeatable / scheduled jobs Supported
- Job priorities Supported
- Rate limiting Supported
- Retries & backoff Supported
- Flows (parent / child) Supported
- Job events Supported
- Redis backend Supported
- PostgreSQL backend Coming soon
- BullMQ Pro Coming soon
BullMQ Pro for Python Coming soon
BullMQ Pro adds groups, observables and batches to the open source library. It is available for Node.js and Bun today, and support for Python is on the way.