Schema & Collections (Milvus-style)

The base PistaDB API stores (id, label, vector) triples — perfect when your metadata fits in a 256-byte label. When you need multiple typed fields per row (section, key, language, line number, token count, …) on top of the embedding, the Collection layer adds a Milvus-compatible schema API.

What you get

• FieldSchema / CollectionSchema / DataType — declare INT64, VARCHAR, FLOAT, DOUBLE, BOOL, JSON, FLOAT_VECTOR fields with is_primary / auto_id / max_length / dim semantics that mirror pymilvus line for line.
• Collection.insert(rows) — accepts a list of dicts keyed by field name; validates types and lengths; auto-generates ids when auto_id=True.
• Collection.search(query, k, output_fields=…) — returns hits enriched with the projected scalar columns.
• JSON sidecar (<path>.meta.json) — vectors stay in the .pst file; scalar fields go to a sibling JSON file with a stable wire format, so a collection created from one language opens cleanly from any other.

Python example

import numpy as np
from pistadb import FieldSchema, DataType, create_collection, Metric, Index

fields = [
    FieldSchema("lc_id",      DataType.INT64,        is_primary=True, auto_id=True),
    FieldSchema("lc_section", DataType.VARCHAR,      max_length=100),
    FieldSchema("lc_key",     DataType.VARCHAR,      max_length=200),
    FieldSchema("lc_lang",    DataType.VARCHAR,      max_length=10),
    FieldSchema("lc_lineno",  DataType.INT64),
    FieldSchema("lc_tokens",  DataType.INT64),
    FieldSchema("lc_vector",  DataType.FLOAT_VECTOR, dim=1536),
]

coll = create_collection(
    "common_text", fields, "Common text search",
    metric=Metric.COSINE, index=Index.HNSW, base_dir="./db",
)

ids = coll.insert([
    {"lc_section": "common", "lc_key": "btn_ok",
     "lc_lang": "en", "lc_lineno": 12, "lc_tokens": 3,
     "lc_vector": np.random.rand(1536).astype("float32")},
])

hits = coll.search(query, limit=10, output_fields=["lc_key", "lc_lang"])[0]
for h in hits:
    print(h.id, h.distance, h["lc_key"], h["lc_lang"])

coll.flush()
coll.close()

A complete runnable port of the typical Milvus create_database() snippet lives at examples/example_schema.py.

Schema rules

• Exactly one is_primary field — must be INT64.
• Exactly one FLOAT_VECTOR field — must have a positive dim.
• Unique field names across the schema.

Validation happens at construction time. The same constraints apply in every wrapper (Python, Go, Rust, C#, C++, Java, Kotlin, Swift, WASM).

The same API, in every language

Go, Rust, C#, C++, Java, Kotlin, Swift, and WASM all expose the same FieldSchema / Collection surface. See Language Bindings for full per-language snippets.