Qdrant has released a 10 billion document vector retrieval benchmark, 1,000 times larger than its prior public set.

The dataset, Qdrant-FineWeb-10B, holds 24.47 TB of vectors and 28.66 TB of source text and metadata. It ships with 120,000 ground truth queries, including 100,000 drawn from MS MARCO, a public set of real search queries.

The open-source vector search and database vendor published the benchmark on Hugging Face. It also released Supernova, a toolkit that computes ground truth on user corpora.

"Accurate benchmarking is one of the hardest things we get asked about, both in our open-source community and in customer discussions," said André Zayarni, CEO and Co-Founder of Qdrant. "Teams need real data at the size they actually run, and they need to know what the correct answers are. Assembling both yourself is expensive. We did it once and made it public."

What does the dataset contain?

Qdrant-FineWeb-10B embeds FineWeb, a corpus from Hugging Face that derives from Common Crawl. Qdrant ran every document through gte-multilingual-base, an open embedding model from Alibaba, on infrastructure provided by cloud provider Vultr to produce one dense vector and one sparse vector per document. Dense vectors encode meaning; sparse vectors encode terms.

Ground truth covers three query sets:

• 100,000 queries drawn from MS MARCO
• 10,000 sparse queries
• 10,000 filtered queries

Running all 120,000 queries against 10 billion documents requires over a quadrillion distance computations. Qdrant computed ground truth using Supernova with SkyPilot orchestration, distributing the exhaustive search across a compute fleet ranked by task size. Computing exact ground truth means comparing every query against every vector with no index and no approximation.

Why does the size matter?

Qdrant previously benchmarked vector search on dbpedia-openai-1M-1536-angular, a 1 million vector set. The vendor also used deep-image-96-angular with 10 million vectors. FineWeb-10B is 1,000 times larger than the first set.

That scale reveals recall behavior invisible at million-vector sizes. The benchmark covers dense vectors, sparse vectors, and metadata filters, supporting recall measurements up to k=1,000 on public data at 10 billion documents across a single unified dataset.

The ground truth runs to depth 1,000. Users can truncate the list to measure recall at any k up to 1,000. No recomputation is needed.

What else ships with the release?

Qdrant published two smaller companion datasets alongside FineWeb-10B:

• PubMed-BGE-M3 covers scientific literature with dense, sparse, and multi-vector representations from one model on one text set, enabling hybrid retrieval comparisons with other variables held constant.
• Coyo-VE places text and images in a shared representation, carrying 5.81 TB of payload against only 117 GB of vectors.

What is the compute footprint?

Qdrant did not disclose the Vultr cluster configuration or total GPU hours. SkyPilot ranked tasks and distributed them across machines, with the exhaustive comparison work consuming the bulk of the compute budget and providing exact ground truth without any index or approximation.

The methodology emphasizes correctness over speed. Engineers can then run approximate methods against that exact ground truth. Supernova also generates embeddings, loads clusters, and runs queries, allowing developers to replicate the methodology on proprietary corpora.

What comes next?

Qdrant plans a follow-up blog titled Internet-scale Vector Search: Benchmarking Qdrant at 10B Vectors, which will publish additional recall measurements at 10 billion vectors using the dataset. FineWeb-10B now stands as the largest open-source vector database benchmark Qdrant has identified publicly.