Databricks · One-Pager · AI / ML

Mosaic AI Vector Search

A serverless similarity-search engine that turns Delta tables into governed vector indexes — the retrieval layer for RAG and semantic search on Databricks.

GA HNSW · ANN · L2 distance Unity Catalog governed Verified Jun 2026 docs

In one line

Vector Search is a managed vector database that indexes embeddings from a Unity Catalog Delta table and serves approximate-nearest-neighbor queries through an auto-scaling endpoint.

Why it matters

RAG and semantic search need fast similarity lookup over millions of embeddings. Vector Search gives you that without standing up a separate vector DB: it keeps the index in sync with your Delta source, computes embeddings for you if you want, and inherits Unity Catalog lineage and access control.

Core concepts

Endpoint

Auto-scaling compute that hosts indexes — Standard (low latency, ~320M vectors at 768-dim) or Storage-optimized (1B+ vectors, ~7× cheaper).

Delta Sync Index

Auto-syncs with a source Delta table; TRIGGERED (manual/batch) or CONTINUOUS (streaming, seconds latency). The default for changing data.

Direct Vector Access Index

No source table — you upsert and delete vectors yourself via API. Use for real-time writes or when there's no governing Delta table.

Managed vs self-provided embeddings

Either point at a text column + embedding model endpoint and Databricks computes vectors, or supply your own pre-computed embedding column.

Similarity search

HNSW approximate-nearest-neighbor over an L2 distance metric; normalize embeddings first for cosine. Supports metadata filters and HYBRID (vector + keyword) search.

UC + Delta integration

Indexes are Unity Catalog objects (governed, lineage-tracked). Standard endpoints require Change Data Feed on the source table.

How it works

Click a stage

1 · Delta table 2 · Embed 3 · Index 4 · Query 5 · LLM

Delta table — a Unity Catalog table with a primary key and the text (or pre-computed vector) column you want to search.

Minimal example

Python · databricks-vectorsearch SDKfrom databricks.vector_search.client import VectorSearchClient
vsc = VectorSearchClient()

# Managed embeddings: Databricks embeds the text column for you
index = vsc.create_delta_sync_index(
    endpoint_name="my-endpoint",
    index_name="main.rag.docs_index",
    source_table_name="main.rag.docs",
    primary_key="id",
    pipeline_type="TRIGGERED",
    embedding_source_column="content",
    embedding_model_endpoint_name="databricks-gte-large-en")

results = index.similarity_search(
    query_text="how do I tune a vector index?",
    columns=["id", "content"], num_results=5)

Gotchas & limits

Standard endpoints need Change Data Feed enabled on the source Delta table; the whole feature requires Unity Catalog + serverless.
Default metric is L2 distance, not cosine — normalize your embeddings before indexing if you want cosine similarity.
Only synced columns are queryable — include every field you'll need in results; you can't fetch un-synced columns later.
You can't convert a self-managed index to a managed one (or switch index type) in place — recreate it. Max embedding dimension is 4096.

Interview soundbites

Say this out loud

"Vector Search is a serverless, Unity-Catalog-governed vector database — it does ANN with HNSW and keeps the index in sync with a Delta table for you."
"Reach for a Delta Sync index when your data lives in a Delta table and changes over time; use Direct Vector Access when you push vectors in yourself with no source table."
"It's the retrieval half of RAG: managed embeddings, hybrid vector-plus-keyword search, and metadata filters, all behind one auto-scaling endpoint."

References

Verified against docs.databricks.com · June 2026