Django + Celery (Beat) + RabbitMQ – Víctor Alexandrino

This is a personal tech note that might be useful for you too. It was useful during some PoCs that I’ve recently made building periodic tasks with Django.

Useful docs
https://docs.celeryq.dev/en/stable/
https://docs.celeryq.dev/en/stable/django/first-steps-with-django.html

Running RabbitMQ container

docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3.9-management

Configuring RabbitMQ

In production it should be mapped to a volume, otherwise every time when the container is recreated you would have to reconfigure that.

docker ps
docker exec -it <container_id> bash
rabbitmqctl add_user admin admin 
rabbitmqctl add_vhost <v_host_name>
rabbitmqctl set_user_tags admin <tag_name>
rabbitmqctl set_permissions -p <v_host_name> admin "." "." ".*"

Setting up Django

1	`pip` `install` `celery`

from celery.schedules import crontab
 
CELERY_BROKER_URL='amqp://<user>:<password>@localhost/<v_host_name>' CELERY_RESULT_BACKEND='rpc://' CELERY_TASK_TRACK_STARTED = True CELERY_BEAT_SCHEDULE = {
    "<task_name>": { 
        "task": "<app_name>.tasks.<method_name>", 
        "schedule": crontab(minute="*/1")
    }
}

Settings.js (Located in the main app folder)

import os 
from celery import Celery 
from django.conf import settings 
 
os.environ.setdefault('DJANGO_SETTINGS_MODULE', '.settings')
app = Celery('current_app_name')
app.config_from_object('django.conf:settings',
namespace="CELERY") app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

celery.py (Create it in the main app folder)

from .celery import app as celery_app
 
__all__ = ['celery_app']

__init__.py (Located in the main app folder)

from <project_name>.celery import app
 
@app.task(bind=True)
def your_recurring_method(self):
    ...your code...

tasks.py (Create it in the app of your choice)

Running Celery and Celery Beat

celery -A <project_name> worker -l debug -E
celery -A <project_name> beat -l DEBUG

Monitoring Celery Beat and RabbitMQ

1	`celery -A <project_name> flower`

Access Flower dashboard via http://localhost:5555
Access RabbitMQ dashboard via http://localhost:15672