Executing Asynchronous Scheduled Tasks with Database-Based Distributed Locking in FastAPI
Introduction When implementing scheduled tasks in FastAPI applications, Celery is often the go-to choice. However, Celery is relatively heavyweight and requires dependencies like Redis or RabbitMQ as message brokers. For smaller-scale services that don’t already utilize Redis or RabbitMQ, introducing these dependencies solely for scheduled task functionality adds unnecessary operational overhead. Another popular Python scheduling framework is APScheduler, but it presents two significant challenges in practice: Duplicate Execution in Multi-Process Environments: When running with Uvicorn’s multi-process mode, each worker process independently executes scheduled tasks, leading to duplicate executions. Poor Asynchronous Function Compatibility: Although APScheduler provides AsyncIOScheduler, its support for asynchronous functions is incomplete. The official documentation explicitly states: “If you’re running an asynchronous web framework like aiohttp, you probably want to use a different scheduler in order to take some advantage of the asynchronous nature of the framework.” For APScheduler users facing these issues, potential solutions include: ...