Databricks · One-Pager · AI / ML

Model Serving + MLflow

Track and register a model with MLflow, govern it in Unity Catalog, then deploy it behind one autoscaling REST endpoint — custom model, foundation model, or external provider.

GA Unified REST endpoint Scale-to-zero Verified June 2026 docs

In one line

Mosaic AI Model Serving is a unified interface to deploy, govern, and query AI models for real-time and batch inference — every model becomes a REST endpoint; MLflow is how you track, package, and register what you serve.

Why it matters

It collapses the ML lifecycle into one path: MLflow logs and versions the model, Unity Catalog governs it, and Model Serving exposes it as a low-latency API that autoscales (and can scale to zero) — no infra to manage, the same surface whether you serve your own model, a Databricks-hosted LLM, or OpenAI.

Core concepts

MLflow tracking

Experiments group runs; each run logs params, metrics, and artifacts. log_model packages the model in the MLflow format (flavor) so any tool can load it.

Models in Unity Catalog

The recommended registry: models live at catalog.schema.model with versions and aliases (e.g. @Champion). Replaces legacy Workspace Model Registry stages.

Custom models

Your Python model (scikit-learn, XGBoost, PyTorch, transformers) registered in MLflow format and served on CPU or GPU compute.

Foundation Model APIs

Databricks-hosted open LLMs (e.g. Llama) — pay-per-token to start, provisioned throughput for production SLAs.

External models

One endpoint in front of OpenAI, Anthropic, Cohere, Bedrock, Vertex AI — credentials encrypted and centrally governed.

Scale-to-zero & autoscaling

Endpoints scale up/down with demand and can drop to zero replicas when idle; workload_size (Small/Medium/Large) sets the concurrency band.

How it works

Click a stage

1 · Train + log (MLflow) 2 · Register (Unity Catalog) 3 · Deploy endpoint 4 · Query

Log model — an MLflow run captures params, metrics, and artifacts; log_model packages the model in the MLflow format.

Minimal example

Python · MLflow → Unity Catalog → Model Servingimport mlflow
from databricks.sdk import WorkspaceClient
from databricks.sdk.service.serving import EndpointCoreConfigInput, ServedEntityInput

mlflow.set_registry_uri("databricks-uc")            # register into Unity Catalog
mv = mlflow.register_model("runs://model", "main.ml.churn")

w = WorkspaceClient()
w.serving_endpoints.create(name="churn", config=EndpointCoreConfigInput(
    served_entities=[ServedEntityInput(entity_name="main.ml.churn",
        entity_version=mv.version, workload_size="Small",
        scale_to_zero_enabled=True)]))   # idle endpoint drops to 0 replicas
# then: POST /serving-endpoints/churn/invocations  (or w.serving_endpoints.query)

Gotchas & limits

Scale-to-zero adds a cold start on the first request after idle, and capacity is not guaranteed — Databricks does not recommend it for production endpoints.
Unity Catalog models use aliases, not stages; the old Workspace Model Registry stage transitions don't exist there. Express environment via the catalog/schema and promote with aliases.
Foundation Model APIs pay-per-token isn't optimized for high throughput; use provisioned throughput for production SLAs and HIPAA-style compliance.
GPU custom-model serving replica count = concurrency ÷ 4; size the workload accordingly and pick the right GPU_* workload type.

Interview soundbites

Say this out loud

"MLflow tracks and packages the model, Unity Catalog governs and versions it, and Model Serving turns it into one autoscaling REST endpoint — the whole lifecycle on one platform."
"Three flavors of endpoint behind the same API: custom models you trained, Databricks-hosted Foundation Model APIs, and external models like OpenAI or Anthropic with governed credentials."
"Real-time goes through serving endpoints; for batch you point ai_query at the same endpoint over a table — and scale-to-zero kills cost on idle endpoints at the price of a cold start."

References

Verified against docs.databricks.com · June 2026