This is a personal tech note that might be useful for you too. It was useful during some PoCs that I’ve recently made building periodic tasks with Django.
Useful docs
https://docs.celeryq.dev/en/stable/
https://docs.celeryq.dev/en/stable/django/first-steps-with-django.html
Running RabbitMQ container
docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3.9-management
Configuring RabbitMQ
In production it should be mapped to a volume, otherwise every time when the container is recreated you would have to reconfigure that.
docker ps
docker exec -it <container_id> bash
rabbitmqctl add_user admin admin
rabbitmqctl add_vhost <v_host_name>
rabbitmqctl set_user_tags admin <tag_name>
rabbitmqctl set_permissions -p <v_host_name> admin "." "." ".*"
Setting up Django
pip install celery
from celery.schedules import crontab
CELERY_BROKER_URL='amqp://<user>:<password>@localhost/<v_host_name>' CELERY_RESULT_BACKEND='rpc://' CELERY_TASK_TRACK_STARTED = True CELERY_BEAT_SCHEDULE = {
"<task_name>": {
"task": "<app_name>.tasks.<method_name>",
"schedule": crontab(minute="*/1")
}
}
Settings.js (Located in the main app folder)
import os
from celery import Celery
from django.conf import settings
os.environ.setdefault('DJANGO_SETTINGS_MODULE', '.settings')
app = Celery('current_app_name')
app.config_from_object('django.conf:settings',
namespace="CELERY") app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
celery.py (Create it in the main app folder)
from .celery import app as celery_app
__all__ = ['celery_app']
__init__.py (Located in the main app folder)
from <project_name>.celery import app
@app.task(bind=True)
def your_recurring_method(self):
...your code...
tasks.py (Create it in the app of your choice)
Running Celery and Celery Beat
celery -A <project_name> worker -l debug -E
celery -A <project_name> beat -l DEBUG
Monitoring Celery Beat and RabbitMQ
celery -A <project_name> flower
Access Flower dashboard via http://localhost:5555
Access RabbitMQ dashboard via http://localhost:15672