Architecture & Default Connectivity

2.1 · Control plane vs compute plane

Two sets of machines, two parties, two subscriptions. The control plane is the brain Databricks runs; the compute plane is where your data is processed. Switch the diagram between classic and serverless and click any box to see what it is and why it's there.

What an architect explains (classic vs serverless)

Illustrative config — the secure boundary at creation

Terraform (azurerm) — the one flag that defines the boundary. Full VNet-injection IaC is the Stage 3 hands-on artifact.

resource "azurerm_databricks_workspace" "this" {
  name                = "adb-arch-demo"
  resource_group_name = "adb-rg"
  location            = "eastus"
  sku                 = "premium"          # unlocks Private Link / CMK / IP ACLs later
  custom_parameters {
    no_public_ip = true                    # THE boundary: no public IPs, no inbound ports;
  }                                        # clusters dial OUT to the SCC relay on 443
}

Portal: Create → Azure Databricks → Networking → Secure Cluster Connectivity (No Public IP) = Yes. Verify: workspace → Managed Resource Group → a cluster VM → Properties → Networking → Public IP is empty.

2.2 · The three connectivity paths

Given the two planes, only three connections carry traffic, and almost every control protects one of them. Switch the diagram between the three paths — the highlighted (orange/green) arrow is the active path; click any box for the "what + why."

The three paths side by side

Illustrative config — where each path's controls live

Path ① lock the front door + IP allowlist · Path ③ default-deny storage firewall. Path ② is set at workspace creation (see 2.1); full VNet-injection/Private Link IaC is the Stage 3–4 hands-on artifact.

resource "azurerm_databricks_workspace" "this" {
  sku                           = "premium"      # IP access lists / Private Link need Premium
  public_network_access_enabled = false          # ① reject public access (front-end PL required)
}
# Path ③ — default-deny storage firewall + free Service Endpoint on the workspace subnet
resource "azurerm_storage_account_network_rules" "adls" {
  default_action             = "Deny"                        # ③ default-deny
  virtual_network_subnet_ids = [azurerm_subnet.container.id] # allow only the workspace subnet
}

2.3 · Deployment & workspace storage

Azure Databricks is a first-party Azure resource (Microsoft.Databricks/workspaces). One create lands resources in two resource groups, and the account → workspace → metastore spine is the governance backbone. Switch the diagram between the two resource groups and the governance spine.

Why NOT workspace storage / DBFS root for governed data

Illustrative config — first-party workspace

resource "azurerm_databricks_workspace" "this" {
  name                          = "adb-workspace"
  resource_group_name           = "adb-rg"
  location                      = "eastus"
  sku                           = "premium"        # Standard tier is EOL 2026-10-01
  managed_resource_group_name   = "adb-managed-rg" # workspace storage/identity land here (read-only to you)
  public_network_access_enabled = true             # set false + Private Link later (Stage 4)
  custom_parameters { no_public_ip = true }        # SCC / NPIP
}

Then Account Console (accounts.azuredatabricks.net) → create a region-matched metastore → assign workspaces. One metastore per region.

Decision guide — what an architect recommends

Customer Q&A — what the security team actually asks

Common asks & the answer

What breaks in the field — and the FIRST check

Three questions to ask before recommending an architecture

1. Is there a regulatory mandate for no public-IP hop, or is backbone-private enough?
2. Do you need full network-path control (custom firewall, UDR, exfiltration protection), or is speed/low-ops the priority?
3. Where will governed data live — and are you on Premium (Standard EOL 2026-10-01)?