Tag: Kafka

Architecture

Managing Avro schemas lifecycle

In a previous article, we explained how we took care of data modelling and exchange between Ebury services using Apache Avro, whilst addressing the most important things to take into consideration when building robust and maintainable data models. As business grows, our internal services ecosystem evolves with it. Keeping track...

March 3, 2022
Architecture

Serialising data with Avro

When designing any new service to be included within Ebury‘s event driven architecture, we need to define the way producers and consumers will behave. This includes describing the structure of the events being produced, in order to enable downstream consumers to read them. Kafka is used to communicate between services....

February 18, 2022
Engineering

Optimizing Kafka producers for latency

This post was first published in aivarsk.com The code I am working with uses Confluent Kafka Python library that calls librdkafka C++ library underneath. The code does synchronous messaging: it produces events one-by-one and ensures event is persisted in Kafka topic by waiting for acknowledgments from all replicas. This code is sensitive to latency...

January 28, 2022
Engineering

4 uses of Kafka you’ll see inside Ebury

As Ebury fully embraces new architectures supporting our business growth, we’ve selected Kafka to be a backbone of system integrations. Here you will find 4 cases where it has been implemented successfully: 1 – Data Migration (a.k.a. Database Consolidation Service) Ebury has one system that has been leased to a...

January 27, 2022