Query Cache

[Veeupup]

Query Cache will ba a universal tier in query node, cache metadata, block data and so on.
And when databend developing, cache layer will evolve over time( single node to distributed ). here are some discussions and thoughts about it.

Now databend has common/cache crate to have memory-based and disk-based LRU cache.

There maybe some stages about cache.

1. single node cache
   as comment here, Query cache #3433 (comment)

The idealised two layer cache is:
Memory LRU is the first level, Disk LRU is the secondary level
If the Memory LRU cache entry is evited(When the capacity reached), spill it to the Disk LRU
This will have three levels with different speeds：

read from s3 -- speed is slowest, capacity is largest
read from disk -- speed is slower, capacity is larger
read from memory -- speed is fast, capacity is small
The io path looks like : memory <-- disk <-- s3.

2. Distributed Cache
   to be, or not to be...

here are some issues and prs about it:
#1784
#3433

[flaneur2020]

imo we would get into a distributed cache at last.

but it's a pretty big project, we can try split the work in a series of PRs, in the first step, we might define a abstract trait which can both adapts a local cache or a distributed cache? in the client side's view, the interface of a local cache and a distributed cache can be the same.

[ZhiHanZ]

agreed. cache should be

1. correct
2. scalable
3. tenant isolated
4. utilize locality in memory, disk, networking etc
5. suitable for production deployment (kubernetes)

did not find some OSS distributed caches suitable for our cases, thus need more investigations with non-trivially number of production performance tests

[GrapeBaBa]

how about this paper https://vldb.org/pvldb/vol14/p2432-durner.pdf