Databricks · One-Pager · DevOps / CI-CD

Databricks Asset Bundles

Infrastructure-as-code for the lakehouse: describe jobs, pipelines, and other resources in YAML and ship them across dev/staging/prod with one CLI.

GA databricks.yml + CLI ≥ 0.218.0 Verified Jun 2026 docs

In one line

Databricks Asset Bundles (DABs) are a Databricks CLI feature that packages your code plus a declarative YAML description of resources into a version-controlled "bundle" you validate, deploy, and run across environments.

Why it matters

DABs are the recommended approach to CI/CD on Databricks. Instead of clicking jobs and pipelines into existence per workspace, you define them once as source files and promote the same definition from dev to prod — bringing source control, code review, testing, and automated deployment to data and AI projects.

Core concepts

databricks.yml

The root config. Top-level mappings: bundle, include, variables, resources, targets, workspace, sync, artifacts.

Resources as code

Define Lakeflow Jobs, Lakeflow Spark Declarative Pipelines, dashboards, model serving endpoints, and MLflow experiments/models — typically split across resources/*.yml.

Targets / environments

Named deployment targets (dev, staging, prod) override workspace, variables, and resources. Exactly one is default: true; modes are development or production.

Variables & substitutions

Custom vars via ${var.x} (set by target, --var, or BUNDLE_VAR_*); built-ins like ${bundle.target} and lookups that resolve an object ID by name.

The CLI flow

bundle init → validate → deploy -t <target> → run. Validate checks syntax; deploy materializes resources; run executes a job or pipeline.

CI/CD

Wire the same commands into GitHub Actions, Azure DevOps, or Jenkins; authenticate as a service principal or via OAuth token federation (no stored secrets).

How it works

Click a stage

1 · Author 2 · Validate 3 · Deploy 4 · Run

databricks.yml — author the bundle: name, included resources/*.yml, variables, and per-environment targets, all in version control.

Minimal example

databricks.ymlbundle:
  name: hello_bundle

resources:
  jobs:
    nightly_etl:
      name: "[${bundle.target}] nightly_etl"  # target name flows into the job name
      tasks:
        - task_key: main
          notebook_task:
            notebook_path: ./src/etl.py

targets:
  dev:
    default: true
    mode: development   # prefixes "[dev you]", pauses schedules
  prod:
    mode: production

Gotchas & limits

Development mode is not isolation. It prefixes names with [dev <user>], pauses schedules/triggers, and tags resources dev — convenient, but it still writes to the target workspace.
Production mode enforces guardrails. It validates the Git branch, blocks per-user path/identity overrides, locks cluster definitions, and effectively expects explicit run_as and permissions.
Paths are relative to the file. In resources/*.yml use ../src/...; from databricks.yml use ./src/... — a frequent foot-gun.
DABs is a CLI feature, not a UI. Requires Databricks CLI v0.218.0+; bundle state lives in the workspace, so don't hand-edit deployed resources out of band.

Interview soundbites

Say this out loud

"Asset Bundles are infrastructure-as-code for Databricks — one databricks.yml describes your jobs and pipelines, and the CLI promotes the same definition from dev to prod."
"The loop is validate → deploy -t <target> → run; targets carry mode and variable overrides, so dev and prod share code but differ in config."
"For CI/CD you run those same commands in GitHub Actions or Azure DevOps as a service principal — DABs are Databricks' recommended CI/CD approach."

References

Verified against docs.databricks.com · June 2026