Cost vs. QPS ParetoQuery only · lower is better · USD / hour

Notes:All tests are conducted in AWS us-west-2. All costs shown here are based on each product's charges in this region.

InsertedSearchableFully IndexedWrite Cost

Notes:For exactly how we defined "inserted", "searchable", "fully indexed", please check VectorDBBench source code for each client. For basic intuition:

Vector Search Latency and QPSunfiltered · ids only · Max concurrency P99

Mode

Notes:

1. We note that while certain products may have a more dramatic cold/warm ratio at p99 percentile, this usually indicates a network shaking problem in later queries and cannot be fully reproduced. Thus we stick with the more faithful definition of cold/warm latency, i.e. the first query for each round.
2. The timing for when a product's collection becomes cold is rather ambiguous since most products don't offer public APIs to provide such info. In order to simulate real world production settings, for cold latency benchmarking, we ensure to wait at least 24 hours since the last operations on the products for the collections to become as cold as possible.