Feature: 2 bit quantization

[rschu1ze]

Describe what you are looking for

Just as an idea (feel free to close this ticket if not feasible or useful).

Qdrant 1.15 added 2-bit (*) quantization: https://qdrant.tech/blog/qdrant-1.15.x/#15-bit-and-2-bit-quantization (besides 64, 32, 16, 1 bit quantization). 2-bit quantization seems like a good compromise between binary and 16 bit quantization (though 4-bit quantization seems an even a better compromise). Perhaps this could be implemented in usearch/SimSimd as well?

(*) ... as well as 1.5 bit quantization but it seems a bit obscure

Can you contribute to the implementation?

• I can contribute

Is your feature request specific to a certain interface?

It applies to everything

Contact Details

robert@clickhouse.com

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

[ashvardanian]

It's definitely a possibility, @rschu1ze! Thanks for recommending it.
4-bit has been considered and even drafted. 2 is a bit more obscure. Do you guys see a lot of demand?

[rschu1ze]

Nobody asked explicitly for 2-bit or 4-bit quantization yet. What we did see a couple of times were certain severely undersized systems (in terms of DRAM) where binary quantization was a prerequisite to build a HNSW index at all. Binary quantization unfortunately wreaked havoc to recall, and such use cases needed to compensate with rescoring (which is done in ClickHouse using the original and uncompressed vectors), slowing down the vector search.

One could argue that any N-bit quantization N = 1 ... 64 should be provided by the a vector search implementation but that would of course be a massive amount of work. So the question is rather what Ns are "interesting" enough on the Pareto front - N = 4 seems interesting in my view. That's why I brought it up. So if you guys are considering 4-bit, then we (ClickHouse) are going to cheer for it.