To improve the solution I explored in the previous entry, we need to go deeper into our knowledge of Celery.
Our goal is to change our tasks’ behavior so that, if the same type of task is currently being executed, the second task is marked as aborted (or similar) and not executed. To achieve this behavior we need to use Celery signals.
Continuing with our previous entry about how to daemonize with Celery. We left the solution in a state where our buffer could eventually collapse due to our producer generating tasks faster that our consumer could execute them.
To solve this I propose another question: “Is foo_action_postsave task currently being executed?”
We’re used to working with python so the examples I’ll use will be either pure python or python-based pseudocode. Python has a really simple and elegant syntax so it’s almost understandable for people who don’t know the language.
I’m going to explore a common problem in the backend of all kinds of applications that have heavy processing requirements, such as those that reprocess all links of each object from a class due to an update or recalculate an attribute that depends on huge functions, and walk through asynchronous solutions.