Databricks · One-Pager · MLOps

LLMOps on Databricks

MLOps extended for generative AI: prompts, RAG and agents shipped through the same develop → evaluate → deploy → monitor loop, on MLflow, Model Serving, Agent Evaluation, and Unity Catalog.

MLflow 3 GenAI · GA Production monitoring · Beta Verified Jun 2026 docs

In one line

LLMOps is MLOps adapted for GenAI apps and agents — managing prompts, retrieval and tool-calling logic through tracing, LLM-judge evaluation, governed deployment, and continuous production monitoring on one Databricks lifecycle.

Why it matters

GenAI quality is non-deterministic and lives in prompts, retrieval, and tool calls rather than model weights, so classic accuracy metrics and "ship the pickled model" patterns break down. LLMOps gives you observability (traces), automated quality measurement (judges/scorers), and governance over the whole app so you can iterate safely and prove quality holds in production.

Core concepts

LLMOps vs MLOps

Same lifecycle discipline, but the artifact is an app (prompt + retriever + tools + model), eval is by LLM judges not just metrics, and you log code, not weights.

GenAI lifecycle

Develop with tracing → evaluate on datasets with judges/scorers → deploy to a serving endpoint → monitor production traffic → iterate.

The tool stack

MLflow 3 (tracing, prompt registry, evaluation) + Model Serving (Agent Serving) + Agent Evaluation + production monitoring on traces.

Unity Catalog governance

Models, agents, UC functions/tools, MCP servers and vector indexes are UC securables with grants and lineage; AI Gateway adds rate/cost limits and guardrails.

Deploy code, not model

MLflow "models from code" with the ResponsesAgent/ChatAgent interface logs the agent's source, so behavior is reproducible and framework-agnostic.

Tracing as the spine

MLflow Tracing captures retrievals, tool calls and intermediate steps — the same traces feed debugging, evaluation, and production monitoring.

How it works

Click a stage · the loop iterates back to Develop

1 · Develop 2 · Evaluate 3 · Deploy 4 · Monitor

Develop — build the agent in any framework, instrument with MLflow Tracing, and version prompts in the Prompt Registry.

Minimal example

Python · MLflow 3 GenAIimport mlflow
from databricks import agents

mlflow.set_registry_uri("databricks-uc")           # govern the agent in Unity Catalog

# 1 — evaluate the app against a dataset with LLM judges/scorers
results = mlflow.genai.evaluate(data=eval_dataset, predict_fn=my_agent, scorers=scorers)

# 2 — register the logged *code* model, then deploy to a serving endpoint
info = mlflow.register_model(model_uri, name="main.genai.support_agent")
agents.deploy("main.genai.support_agent", info.version)  # monitoring runs on prod traces

Gotchas & limits

LLMOps needs MLflow 3 (mlflow[databricks]>=3.1) — MLflow 2 GenAI patterns differ; pin the version.
Production monitoring (auto-running judges/scorers on prod traces) is Beta, as are Unity AI Gateway and service policies — confirm before promising it for a customer.
LLM judges are themselves models: they cost tokens, can be noisy, and should be calibrated against human-labeled examples — not a free oracle.
Log the agent as code (ResponsesAgent/ChatAgent), not a serialized object; serializing complex agents is fragile and not the recommended path.

Interview soundbites

Say this out loud

"LLMOps is MLOps for GenAI — the artifact moves from model weights to an app of prompts, retrieval and tools, so evaluation shifts to LLM judges and traces, but the develop-evaluate-deploy-monitor discipline is identical."
"On Databricks it's one stack: MLflow 3 for tracing and evaluation, Model Serving to deploy the agent, Agent Evaluation plus trace-based monitoring for quality, and Unity Catalog governing every asset."
"You deploy code, not a pickled model — MLflow models-from-code logs the agent's source with the ChatAgent/ResponsesAgent interface, so deploys are reproducible and framework-agnostic."

References

Verified against docs.databricks.com · June 2026