celery

EDINBURGH
Thanks to Ebury’s learning program, Héctor Álvarez and Jesús Gutiérrez were selected to attend EuroPython in Edinburgh.

EuroPython is a yearly conference in Europe that focuses on Python programming language and its ecosystem . This year’s sessions were held at the Edinburgh International Conference Center; an amazing building in the core of the city, just a stone’s throw away from the historical city of Edinburgh.

More than 1200 programmers and python lovers from 51 countries attended the event. With over 150 sessions in 7 tracks we were prepared to take away as much new information as possible.

From the outset, it was clear that the hottest topic in EuroPython was asynchronous programming. First integrated into 3.6 Python, and in various iterations since, it is still entirely possible to use Python without needing or even knowing about the asynchronous paradigm. However, if you are interested in the nuts and bolts of the tech involved, read on.

For the beginners out there, your central processing unit (CPU) follows a synchronous programming model, which means that things happen one by one. For example, when you create a function that performs a long-running action, it returns only when the action is finalised and it can return the result. Even when different programs that are run by your Operating System (OS), which is programmed synchronously, the OS manages them asynchronously. This is why multitask Operating Systems have been used for a long time.

The pitfall of asynchronous programming is that it’s difficult to know which of the coroutines has the execution time, and which coroutine spawned which; the event loop obviously knows but the programmer doesn’t.

The recently launched Python3.7 tries to solve that problem with inheritance of tags.

Under the asynchronous topic:

Asyncio in python 3.7 and 3.8 (Yury Selivanov)

Asyncio in production (Hrafn Eiriksson)

Asyncio in practice: We did it wrong (Lynn Root)

Here, I’ll briefly explain how the asynchronous programming works since Python 3.5.

As mentioned before, classic program code is executed in a single event line.In asynchronous programming, however, code is executed in a single loop. That loop is part of the code that orchestrates what’s executed when, inside the loop, there are a group of tasks—coroutines. Coroutines are defined with the reserved word async.

When a coroutine has been executed, it reports to the loop when it’s waiting for an external resource using the reserved word await.

When the loop detects that a coroutine is awaiting, it gives the execution time to the next coroutine, the loop then stores the memory state and where it’s waiting. When the external resource finally gives the response it fires a callback so knows that the coroutine is ready to keep working.

That’s  the theory, but now it’s time to look at the code:

import asyncio
import logging


logging.basicConfig(format='%(asctime)s %(message)s', datefmt='[%H:%M:%S]')
log = logging.getLogger()
log.setLevel(logging.INFO)


# define a coroutine
async def sleeper(name, delay):
    """This coroutine will wait for 2 seconds and then keep working."""
    log.info(f"{name}: START (wait for {delay}s)")
    await asyncio.sleep(delay)
    log.info(f"{name}: END (wait for {delay}s)")
    return name

if __name__ == '__main__':
    # create the loop
    loop = asyncio.get_event_loop()

    coroutine1 = sleeper('first coroutine', 2)
    coroutine2 = sleeper('second coroutine', 5)
    task1 = loop.create_task(coroutine1)
    task2 = loop.create_task(coroutine2)

    log.info("main: START run_until_complete")
    loop.run_until_complete(asyncio.wait([task1, task2]))
    log.info("main: END   run_until_complete")

Here at  Ebury, we don’t currently use asynchronous programming because Django (our framework) is not asynchronous. However, there are parts of the code that are slow (like sending an email that will delay for a second), and so in those cases we use a workaround—task executor named celery. If you want to know more about celery, follow this link: https://labs.ebury.rocks/?s=celery

Miscellaneous

Domain Driven Design, Robert SmallShire

Domain Driven Design is an approach to software development that emphasises high-fidelity modelling of the problem domain, which uses a software implementation of the domain model as a foundation for system design.

PEP 557 (data classes) versus the world (Guillaume Gelin)

Data classes are a very controversial feature, yet here it’s explained why they’re useful for us.

Getting started with mypy and type checking (Jukka Lehtosalo)

Mypy , is defined as: a static type checker for Python that aims to combine the benefits of dynamic (or “duck”) typing and static typing.

Static typing can help you find bugs faster with less testing and debugging. In large and complex projects, this can be a major time-saver.

Python decorators: Gift or poison? (Anastasiia Tymoshchuk)

A taxonomy of decorators: A-E (Andy Fundinger)

Python 2 is dead! Drag your old code into the modern age (Becky Smit)

What’s new in Python 3.7 (Stephane Wirtel)

Pythonic code vs. Performance (Łukasz Kąkol)