Distributed Cache

Caching is one of the most powerful techniques for improving system performance. When done right, it can dramatically reduce latency and lighten the load on your databases. In a distributed system, caching becomes even more critical—and more complex.

In this section, we’ll explore:

Why caching matters: Understanding the performance benefits and use cases
Writing policies: How to keep your cache and database in sync
Eviction strategies: Managing limited cache space effectively
Distributed cache architecture: Sharding, replication, and consistency
Cache invalidation: Keeping your data fresh and avoiding stale reads

Whether you’re designing a high-traffic web application or optimising database queries, understanding distributed caching is essential for modern system design.

Foundational learning https://www.youtube.com/watch?v=U3RkDLtS7uY Writing policies The order in which data is written to the cache and the persistent data store will affect the performance of the cache. What happens if data is written into the cache first? Writing the data in the cache first, means you can then choose to either: write synchronously to the persistent datastore and then respond to the client write to cache and respond to the client but allow writing to the persistent storage asynchronously The first approach is called Write-through cache. The implications of this approach is that writes will take longer as both the cache and the database are written to and ensured it is successful before a response is sent to the client. ...

High level design of the Distributed Cache Functional requirements insert data into the distributed cache retrieve data from the distributed cache NFR Fast retrieval of data - insert and retrieve should be fast scalability - distributed cache must be scalable horizontally without bottlenecks availability - ensure that there is sufficient redundancy to keep the system highly available. Consistency - The same data retrieved from the cache concurrently should be the same. Affordability - shouldn’t cost the planet API put (key, value) get (key) Design considerations Storage hardware Make use of commodity hardware where possible to keep costs relatively low. Large data will demand more shards, hence keeping commodity hardware will enable us to scale efficiently. ...

Evaluating the design High performance Consistent Hashing results in O(log(N)) complexity where N represents the number of shards On a hash server, it takes constant time to retrieve a value using the key O(1) for eviction in the linked list Let’s assess the effective access time considering there could be cache hits and misses. EAT = Ratio (hit) x Time (hit) + Ratio(miss) x Time (miss) if MFU, we have 10% cache miss and for LRU, 5% cache miss rate ...