Blogs

Many moons ago I was working on an online eCommerce platform. The platform used to undergo massive traffic spikes periodically. I was trying to implement a distributed counter. I was using a distributed counter because I wanted to be able to increment the counter on multiple servers. CAP Theorem The CAP theorem was proposed by Eric Brewer. CRDT CRDT stands for conflict-free replicated datatype. Conflict-free replicated datatype describe data-types that can be replicated across multiple computation units or nodes, they can be updated concurrently without any coordination, and then merged to get a consistent state. It doesn’t matter in which order you execute operations on the data type or if you repeat operations the result is eventually correct. Each node in a distributed system has its own replica of the CRDT. Each replica can resolve queries in isolation and can also process commands that immediately alter its state. CRDTs they can be concurrently updated across nodes and any conflicts can be resolved sensibly. CRDTs always have a merge function that can take many data entries living on different nodes and merge these automatically into one consistent view of the data, without any coordination between the nodes. CRDTs allow two conflicting updates to be merged. All replicas will converge to the same state when all updates have been delivered. The most important properties of the merge function are that it is symmetric and monotonic. The issue that CRDTs address is conflict resolution when different versions of the structure appear due to network partitions and their eventual repair. For a general data structure, if there are two conflicting versions, the solution is either to choose one (according to some general rules, like take the random one or the latest one, or application-specific logic) or to keep both versions and defer conflict resolution to the client code. CRDTs are conflict-free, that is, the structures are devised so that any conflict is resolved automatically in a way that doesn’t bring any data loss or corruption. ...

Introduction “Chaos Engineering is the discipline of experimenting on a system in order to build confidence in the system’s capability to withstand turbulent conditions in production.” Priciples of Chaos Engineering Introduction In modern, rapidly evolving distributed systems, components fail all the time. These failures can be complex as they can cascade across systems. System weaknesses such as latency, race conditions, byzantine failures etc can be exacerbated in the face of large traffic volumes. Chaos engineering is key to discovering how these complex failures may affect the system and then validating over time that the weaknesses have been overcome. ...

Infrastructure as code (IaC) is the process of provisioning and managing your infrastructure resources through definition files or code.

Imagine a green, sustainable city, meticulously designed for environmental harmony and efficiency. The city has many distinct localities such as neighborhoods, districts, and even villages with their own identity and cultures. This city boasts an intricate public transport system, with buses, trams, and subways efficiently transporting citizens to their destinations from its various localities. Multiple such cities are connected together in a thriving, fast-paced ecosystem. The cities are also similarly connected in an efficient and sustainable design. Each locality in a city represents a microservice, and each city in this system, a domain operating within the larger ecosystem - the application. In an ideal world, this system not only ensures smooth transit but also maintains each city’s eco-friendly ethos; balancing efficiency with sustainability. But how does this ecosystem manage to keep its vast and varied transport network running so smoothly and eco-consciously, avoiding traffic jams, pollution, and inefficiencies? ...

Sidecar pattern is a single-node pattern made up of two containers. It involves co-locating another container in a node/pod along with the main application container. The application container contains the core logic for the application. The role of the sidecar is to augment and improve the application container, often without the application container’s knowledge. In its simplest form, a sidecar container can be used to add functionality to a container that might otherwise be difficult to add. In addition to being co-located on the same machine, the application container and sidecar container share several resources, including parts of the filesystem, hostname and network, and many other namespaces. The sidecar can be plugged and unplugged from the main application, since it is isolated and cannot impact the application in case it starts misbehaving. ...

Transactional Outbox is a pattern to reliably publish messages without the use of distributed transactions. It uses an Outbox and a message dispatcher to reliably persist state and publish messages.

The Idempotent Consumer pattern provides the necessary safeguards to allows logic wrapped by it to be executed only once. It implements an unique identifier for each message and an idempotent repository.

Event driven architecture (EDA) is an architectural paradigm where behavior is composed by reacting to events. In this paradigm events imply a significant change in state.

A single container provides application isolation and mobility. However, a container by itself doesn’t improve the quality of your service—for example, in terms of load balancing or failover. This is where multi-container solutions come into play. However managing a handful of containers is completely different from managing production-scale containers, which may number from hundreds to thousands. To support container management, we need an easy way of deploying and handling these containers at scale. This is where container orchestration. comes into play. ...

An API Gateway provides a single and unified API entry point across one or more internal APIs. It mediates, routes, and invokes a respective endpoint after request verification, content filtering, authentication, and authorization.