Vector DB Deep Dive

A production AI system might index billions of documents. If you calculated the distance for every document manually, a single search would take minutes. In this chapter, we explore the algorithms that make sub-second search possible at massive scale — and the platforms you'll use to deploy them.

1. HNSW: The Graph that Scales 🕸️

HNSW (Hierarchical Navigable Small World) is the industry standard for fast vector retrieval. It organizes vectors into a multi-layer graph.

• Top Layers: Sparse graphs with few nodes. Fast "jumps" across the dataset.
• Bottom Layers: Dense graphs with many nodes. High-precision "local" searches.

How it works: You start at the top, find the closest "neighbor," move down a layer, and repeat until you find the exact vector. This is much faster than checking every point!

Key HNSW Parameters:

• ef_construction: Search breadth during index building. Higher = better recall but slower build.
• M: Number of connections per node. Higher = better recall, more memory.
• ef_search: Search breadth at query time. Controls the speed/accuracy tradeoff.

[!TIP] In production, start with M=16 and ef_construction=128. Then tune ef_search at query time to hit your latency SLA.

2. Compressing Data: Product Quantization (PQ) 📦

Vector embeddings are large (e.g., 1536 dimensions). Storing a billion of them would require Terabytes of expensive RAM. Product Quantization (PQ) shrinks them by 90%+.

• Split: Break a long vector into smaller "chunks" (sub-vectors).
• Cluster: For each chunk, run k-means to build a "codebook" of ~256 centroids.
• Encode: Replace each chunk with the index of its nearest centroid (just 1 byte!).
• Reconstruct: Reassemble using only the small index numbers.

[!NOTE] PQ slightly reduces "Recall" (accuracy) but allows you to fit 10x more data on the same hardware. IVF+PQ (clustering + quantization) is the standard for billion-scale datasets.

Quantization Variants Comparison

3. Filtering: Pre-filter vs. Post-filter 🔍

What if you want to find "Python experts (vector)" but only in "New York (metadata)"?

• Post-filtering: Search 100 vectors, then delete those not in NY. (Bad: You might end up with 0 results if the top 100 were all in London).
• Pre-filtering: Only search in the NY bucket. (Modern Vector DBs use Metadata Filtering which indexes both vectors and keywords).
• ACORN Filter (Qdrant/Weaviate): A newer approach that interleaves filtering with graph traversal, achieving near-100% recall even with highly selective filters.

4. Algorithm Performance Comparison

5. The Vector Database Ecosystem 🌐

Understanding the algorithms is step one. Now let's look at the actual platforms you'll deploy in production.

📚 Official Documentation

• Pinecone Documentation — Production-ready managed vector DB
• Qdrant Documentation — Open-source with powerful filtering
• Weaviate Documentation — GraphQL-based vector search
• Milvus Documentation — Billion-scale vector search
• pgvector GitHub — Vectors inside PostgreSQL
• ChromaDB Documentation — Simplest for prototyping
• FAISS GitHub — Meta's lower-level library
• ANN Benchmarks — Objective speed/recall comparisons

6. Platform API Code: Working Examples 💻

Pinecone — Managed Cloud Vector DB

Qdrant — Open-Source with Rich Filtering

pgvector — Vectors Inside PostgreSQL

7. Choosing the Right Vector Database 🎯

Key Decision Factors:

• Team has zero ops experience → Pinecone (fully managed, no servers to maintain)
• Complex metadata filters → Qdrant (most powerful filter system)
• Already using PostgreSQL → pgvector (no new infra, ACID compliance)
• Billion-scale + GPU acceleration → Milvus / Zilliz
• Multi-modal (text + images) → Weaviate (built-in vectorizers for multiple modalities)
• Local development / prototyping → ChromaDB (zero config)
• Custom algorithms → FAISS (maximum flexibility, requires your own serving layer)

8. Multi-Tenancy: One Index for Many Customers 🏢

For SaaS applications, you need to isolate customer data within a shared index.

Interactive Challenge: Vector Compression (PQ)

A simple look at how quantization saves space by mapping to "clusters".

Quiz

Key Takeaways

✅ HNSW is the most efficient way to search large-scale vector datasets.
✅ Product Quantization is required if you have limited RAM and billion-scale data.
✅ Metadata Filtering must be pre-filtered for correctness in production.
✅ pgvector is the right choice if you already run PostgreSQL.
✅ Pinecone is the fastest path to zero-ops production vector search.
✅ Qdrant offers the most powerful filtering capabilities for complex queries.
✅ Multi-tenancy isolation via metadata filtering is essential for SaaS applications.

What's Next?

Data is retrieved. Now let's orchestrate a team of agents to use it.
Next Chapter: Multi-Agent Orchestration: Graphs, Handoffs, and State.