Context
- I m building a dashboard application gathering data from a Rest API
- This Rest API can be requested by client applications others than the dashboard to get data
- That data is then aggregated and displayed in the dashboard for analysis reasons
Scenario
A client application running in the browser is requesting an API endpoint with a dynamic param being a b64 string containing a "projectId" data in it, and some other data that are custom per user.
It means that:
- multiple users requesting the API will not pass the same b64 string
- all these users can call the api with the b64 string containing the same
projectId - the API response might differ per user
The requested API has to do a bunch of work in the background, making computation intensive calculation. In order to speed up the response, I can cache the result on redis with a key that may look like: "{b64}:computation" => "API response"
The problem
On the dashboard, I can make a modification on the entity represented by the projectId field. When a modification happens on this specific entity, the cached entries containing the specific projectId should be invalidated and recomputed on the next call.
First thoughts
I could:
- Create a set in Redis
- When invalidating, requesting all the entries from the set and invalidating them one by one.
=> It seems slow
Question
How to make that effectively and working at scale when there are more than 200M (astonishing number) potential entries to invalidate?
