Git Folders (Repos)

The one-liner

The workflow — click a step

▸ clone → branch → commit → push → PR

How it works — deep dive

1 · Connect a Git provider — OAuth or PAT

Mechanism: Databricks needs credentials to act for you. Hosted GitHub → the Databricks GitHub app (OAuth 2.0) (auto-renews, repo-scoped); others → a PAT. Trade-off: PATs expire/rotate; the GitHub app needs a one-time admin install.

# Register a PAT credential (git-credentials group).
# Providers: gitHub, gitLab, azureDevOpsServices, bitbucketCloud, awsCodeCommit, …
databricks git-credentials create gitHub \
  --git-provider gitHub \
  --git-username my-user \
  --personal-access-token "$GIT_PAT"
# Hosted GitHub: prefer the Databricks GitHub app / OAuth over a PAT.

2 · Clone, branch, commit, push — in-workspace workflow

Mechanism: clone a remote into a Git folder, then branch/commit/pull/push & resolve conflicts in the Git dialog; the repos API/CLI does the same for automation. Trade-off: the UI client is for dev — prod deploys from a clean pinned ref via DABs.

# Programmatic clone (CI/automation) — databricks repos command group:
databricks repos create \
  --url https://github.com/acme/sales-platform \
  --provider gitHub \
  --path /Workspace/Repos/ci/sales-platform
# Day-to-day dev uses the workspace Git dialog (branch/commit/push).

3 · Branch-per-feature + PR — the team workflow

Mechanism: isolate each change on a feature branch, push, open a PR for review; merge to main triggers CI. Notebooks are stored as source files, so diffs are reviewable. Trade-off: notebook outputs aren't versioned (only source); mind notebook merge conflicts.

# git flow that maps to the UI buttons:
git checkout -b feature/add-returns-metric
git add src/ resources/ databricks.yml
git commit -m "Add net-returns metric to sales pipeline"
git push -u origin feature/add-returns-metric
# → open a PR; review; merge to main triggers CI.

4 · Git + Asset Bundles + CI/CD — wiring it together

Mechanism: the repo holds source + databricks.yml. On PR, CI runs bundle validate; on merge, CI authenticates as a service principal and runs bundle deploy -t prod. Trade-off: CI must auth without a human — use a service principal + OAuth, never a personal token.

# .github/workflows/deploy.yml — GitHub Action calling the Databricks CLI
name: deploy
on:
  push: { branches: [ main ] }
  pull_request: {}
jobs:
  bundle:
    runs-on: ubuntu-latest
    env:
      DATABRICKS_HOST: ${{ secrets.DATABRICKS_HOST }}
      DATABRICKS_CLIENT_ID: ${{ secrets.DBX_SP_CLIENT_ID }}      # service principal
      DATABRICKS_CLIENT_SECRET: ${{ secrets.DBX_SP_SECRET }}     # OAuth (M2M)
    steps:
      - uses: actions/checkout@v4
      - uses: databricks/setup-cli@main
      - run: databricks bundle validate                          # every PR
      - if: github.ref == 'refs/heads/main'
        run: databricks bundle deploy -t prod                    # on merge

5 · Sparse checkout & allow-lists — scaling monorepos

Mechanism: enable sparse checkout (cone patterns) at clone time so only chosen sub-paths sync; admins can allow-list Git URLs/providers workspace-wide. Trade-off: patterns are set at clone & must track repo layout changes.

# Cone sparse-checkout patterns entered in the "Create Git folder" dialog:
sales-platform/
docs/        # only these sub-trees sync into the workspace Git folder
# admin: also allow-list permitted Git URLs/providers for the workspace.

Git folders vs Asset Bundles — they pair up

Why it matters

• Workspace-only notebooks have no history, no review, no rollback. Git folders brings software discipline to DE.
• "Version-control & collaborate on Databricks code?" → Git folders + provider, feeding DABs.

Uses, edge cases & limitations

• Uses: source control for notebooks/code, branch-based dev, PR review, the repo CI/CD pulls from, automation via the repos/git-credentials APIs.
• Edge cases: production jobs should run from a clean, pinned ref (or a DABs deploy) — not a dev Git folder mid-edit; notebook outputs aren't versioned; CI must auth as a service principal.
• Limitation: Git folders versions workspace assets; resource deployment + cloud infra is DABs/Terraform's job; some file types/sizes have limits.

Common gotchas

• ❌ Editing prod notebooks directly with no Git → no history/rollback.
• ❌ Pointing a production job at a dev Git folder being edited live.
• ❌ Committing data/secrets — version code only; use secret scopes.
• ❌ Using a personal PAT in CI instead of a service-principal credential.
• ❌ Treating Git folders as the deployer — it version-controls; DABs deploy.