Index Algorithms

PistaDB ships 8 interchangeable index algorithms. One index type is active per database; pick the one that matches your scale, recall target, and memory budget.

Index	Algorithm	Best For
LINEAR	Brute-force exact scan	Ground truth, small embedding sets
HNSW	Hierarchical Navigable Small World	Recommended for RAG — best speed / recall tradeoff
IVF	Inverted File Index (k-means)	Large knowledge bases with a training budget
IVF_PQ	IVF + Product Quantization	Memory-constrained deployments
DISKANN	Vamana graph (DiskANN)	Billion-scale embedding collections
LSH	Locality-Sensitive Hashing	Ultra-low memory footprint
SCANN	Anisotropic Vector Quantization (Google ScaNN)	Maximum recall on MIPS / cosine workloads
SQ	Scalar Quantization (uint8)	4× memory & storage savings, no training needed

Picking an index

        ┌─ <10k vectors? ─────────────────────→ LINEAR (exact, trivial)
        │
        ├─ Need best speed/recall on text?  ──→ HNSW
        │
embedding set
        │
        ├─ Memory-constrained?               ──→ IVF_PQ  or  SQ
        │
        └─ Approaching billion scale?        ──→ DISKANN

• HNSW is the default recommendation for RAG. Set hnsw_M, hnsw_ef_construction, and hnsw_ef_search based on your recall target (see examples).
• IVF / IVF_PQ / SCANN require training. Train on a representative sample first via db.train().
• SQ is the easiest memory win: 4× smaller with negligible recall loss, no training step.

Distance metrics

All indices support all five distance metrics. For text embeddings (OpenAI, Cohere, BGE, GTE), use COSINE. For pre-normalised embeddings, IP is mathematically equivalent and slightly faster.

Full reference

The complete examples — including HNSW tuning, IVF training, SQ memory savings, and ScaNN two-phase search — live in docs/examples.md.