I have seen several projects where the developers had implemented caching all over the place. Caches were causing a large increase of heap usage, and users were always complaining that they were not seeing the latest data. My opinion on this is that a decision to add caching should not be taken lightly. Adding a cache means adding a lot of additional (or so-called accidental) complexity and also has a functional impact on the users. Adding a cache raises a lot of questions that need to be answered:
- What if cached data is updated, should the cached record be updated or evicted too?
- What should we do in a distributed environment, use a distributed cache? Is this distributed cache scalable?
- Do we get the performance improvements we're expecting?
- What is an acceptable delay for users to see the updated data?
- How many elements should we store in the cache?
- What eviction policy do we need when not all data fits in the cache?