Replacing a cache service with a database

Here's a summary of the themes from the Hacker News discussion, with direct quotations:

Caching as a Symptom of Architectural Problems

A significant portion of the discussion revolves around the idea that caching is often implemented as a "band-aid" to fix underlying architectural deficiencies rather than a primary design choice. Many users expressed the sentiment that if a system requires extensive caching to function, there's likely a more fundamental issue that should be addressed first.

simonw stated, "A friend of mine once argued that adding a cache to a system is almost always an indication that you have an architectural problem further down the stack, and you should try to address that instead. The more software development experience I gain the more I agree with him on that!"
jmull echoed this sentiment: "That's true in my experience. Caches have perfectly valid uses, but they are so often used in fundamentally poor ways, especially with databases."
zeras elaborated, "I think a fundamental mistake I see many developers make is they use caching trying to solve problems rather than improve efficiency. It's the equivalent of adding more RAM to fix poor memory management or adding more CPUs/servers to compensate for resource heavy and slow requests and complex queries. If your application requires caching to function effectively then you have a core issue that needs to be resolved, and if you don't address that issue then caching will become the problem eventually as your application grows more complex and active."
barrkel offered a stark personal reflection: "I once 'solved' a huge performance problem with a couple of caches. The stain of it lies on my conscience. It was actually admitting defeat in reorganizing the logic to eliminate the need for the cache. I know that the invalidation logic will have caused bugs for years. I'm sure an engineer will curse my name for as long as that code lives."
chamomeal, while also defending legitimate caching uses, acknowledged, "However I agree that caching is often an easy bandaid for a bad architecture."

The Nature and Definition of Caching vs. Databases

There was disagreement and discussion about the fundamental distinction between a cache and a database, with some arguing that a cache is essentially a specialized type of database, while others emphasized unique characteristics like invalidation.

hoppp initiated this line of thought: "The cache service is a database of sorts that usually stores key value pairs. The difference is in persistence and scaling and read/write permissions"
Supermancho simplified this to: "ie A cache is a database. The difference is features and usage."
barrkel strongly dissented from this view, stating, "No, what makes a cache a cache is invalidation. A cache is stale data. It's a latent out of date calculation. It's misinformation that risks surviving until it lies to the user."
IgorPartola offered a more nuanced perspective: "If you think of it as a cache, yes. If you think of it as another data layer then no." He further suggested that thinking of a cache as a "data layer" rather than a temporary store can lead to better architectural patterns, avoiding invalidation issues.
chamomeal also joined the "database as a cache" analogy: "You can even think about databases as a kind of cache: the 'real' data is the stream of every event that ever updated data in the database! (Yes this stretching the meaning of cache lol)"

The Cost and Complexity of Cache Invalidation

A recurring concern was the inherent difficulty and potential for introducing bugs through cache invalidation logic. This complexity is seen as a major drawback of caching solutions.

barrkel highlighted this issue: "Caches suck because invalidation needs to be sprinkled all over the place in what is often an abstraction-violating way."
He continued with his personal anecdote about potential long-term negative consequences: "I know that the invalidation logic will have caused bugs for years. I'm sure an engineer will curse my name for as long as that code lives."
IgorPartola proposed a method to mitigate this: "I prefer this style of caching to on demand caching because it means you avoid cache invalidation issues AND the thundering herd problem." This refers to pre-generating cached content at "generation time."

Valid Use Cases for Caching

Despite the strong criticisms, several users defended caching, arguing it has essential roles in performance optimization, especially for read-heavy workloads and data that doesn't change frequently.

AtheistOfFail argued for its benefits: "I disagree. For large search pages where you're building payloads from multiple records that don't change often, it could be beneficial to use a cache. Your cache ends up helping the most common results to be fetched less often and return data faster."
chamomeal also defended its necessity: "Idk I think caching is a crucial part of many well-designed systems. There’s a lot of very cache-able data out there. If invalidating events are well defined or the data is fine being stale (week/month level dashboards, for example), that’s a fantastic reason to use a cache. I’d much rather just stuff those values in a cache than figure out any other more complicated solution."
lemmsjid connected caching issues to business/engineering agreements on data consistency: "When you think of cache as another derived dataset then you start to realize that the issues caches bring to architectures are often the result of not having an agreement between the business and engineering on acceptable data consistency tolerances."

The Role of Data Consistency and User Expectations

A nuanced point raised was the importance of understanding and communicating data consistency levels to users. Caching introduces potential staleness, and this needs to be managed in conjunction with user expectations.

lemmsjid elaborated: "In many cases this is fine, even preferred. Sometimes not, and instead you link the user to a realtime dashboard instead. Pretty much every view the user sees of data should include an understanding as to how consistent that data is with the source of truth. Issues with caching (besides basic bugs) often come up when a performance issue comes up and people slap in a cache without renegotiating how the end user would expect the data to look relative to its upstream state."

Alternative Data Management and Database Solutions

The discussion touched on alternative approaches to data storage and retrieval that might circumvent the perceived complexities of traditional caching.

DrBazza questioned the automatic reliance on databases: "The two questions no one seems to ask are 'do I even need a database?', and 'where do I need my database?' There are alternate data storage 'patterns' that aren't databases."
The concept of "Incrementally View Maintenance" (IVM) systems was introduced by avinassh as a way to get precomputed data directly from the database, similar to a cache but managed differently.
In response to the need for high-read performance, jamesblonde promoted RonDB as an open-source database designed for millions of concurrent reads, offering features like pushdown projections and partition-pruned index scans.
phoronixrly and xixixao pointed to specific tools and concepts like Rails' solid_cache and Convex's approach to default caching as examples of different philosophies in managing derived data and application complexity.