Classic Compute Plane Networking

3.1 · Default (managed) VNet vs VNet injection

The first decision: let Databricks run a locked managed VNet, or inject the workspace into a VNet you own. The choice is permanent and it gates every advanced control. Switch the diagram and click any box for the "what + why."

Managed VNet vs injection

Illustrative config — the delegation + the one flag that declares injection

Terraform (azurerm) — the lines that teach the point. Full module is the Stage-3 hands-on artifact.

resource "azurerm_subnet" "host" {           # both subnets PRIVATE under SCC
  address_prefixes = ["10.179.0.0/18"]
  delegation {                               # REQUIRED: lets Databricks manage the subnet
    name = "databricks-del"
    service_delegation { name = "Microsoft.Databricks/workspaces" }
  }
}
resource "azurerm_databricks_workspace" "this" {
  sku = "premium"
  custom_parameters {
    virtual_network_id = azurerm_virtual_network.adb.id   # <-- THIS declares VNet injection
    no_public_ip       = true                             # SCC in either model
  }
}

Portal: Create → Azure Databricks → Networking → Deploy in your own VNet = Yes → pick the VNet + host/container subnets. Verify: VNet → Subnets → both show an NSG + Delegation = Microsoft.Databricks/workspaces.

3.2 · Subnets & address-space sizing

A VNet-injected workspace needs two equal subnets — host + container. Every node is a duplex that burns one IP in each, and you can never re-pave the streets. Switch the diagram between the node-IP view and the sizing-math view.

The numbers

VNet /16–/24; subnets up to /26 (min /28). Azure reserves 5 IPs/subnet. Max nodes ≈ 2^(32 − prefix) − 5, counted across all running clusters at once.

Illustrative config — sizing + delegation (the permanent part)

resource "azurerm_virtual_network" "adb" {
  address_space = ["10.179.0.0/16"]          # /16: room for big subnets + a future PL subnet
}
resource "azurerm_subnet" "host"      { address_prefixes = ["10.179.0.0/18"] }   # 16,379 usable
resource "azurerm_subnet" "container" { address_prefixes = ["10.179.64.0/18"] }  # MUST equal host, no overlap
# 10.179.128.0/18 left FREE -> headroom for a /27-/28 Private Link subnet (Stage 4).

Portal: VNet → IP addresses (set space a couple of bits larger than the subnets) → + Add subnet twice (equal, non-overlapping, leave a gap) → deploy the workspace into them.

3.3 · Secure Cluster Connectivity (SCC) / No Public IP

SCC reverses the call direction: the cluster dials out to the SCC relay and the control plane talks back down that tunnel — so no public IP on the VM, no inbound port. Step the diagram through the SCC handshake and the egress; click any box for the "what + why."

SCC disabled vs enabled vs + Private Link

Illustrative config — the entire SCC switch

resource "azurerm_databricks_workspace" "this" {
  sku = "premium"
  custom_parameters {
    no_public_ip       = true     # <-- THIS is Secure Cluster Connectivity / No Public IP
    virtual_network_id = var.vnet_id
  }
}
# ARM: parameters.enableNoPublicIp.value = true  (defaults to true in API 2024-05-01+)

Portal: Create → Networking → Secure Cluster Connectivity (No Public IP) = Yes (default). Verify: Managed RG → cluster VM → Properties → Networking → Public IP is empty.

3.4 · Egress — NSG rules, UDRs, NAT gateway

Once the VMs are private, three primitives control the outbound door: NSG = allowed?, UDR = which path?, NAT = what public IP?. Switch the diagram between plain NAT egress and the firewall-UDR (DEP) pattern; click any box for the "what + why."

What a cluster must reach (service tags, not raw IPs)

Illustrative config — NAT + firewall UDR (keep relay direct) + a neighbor deny

resource "azurerm_route" "to_firewall" {       # all other egress -> Azure Firewall
  address_prefix = "0.0.0.0/0"                 # NOTE: this UDR OVERRIDES the NAT gateway
  next_hop_type  = "VirtualAppliance"
  next_hop_in_ip_address = "10.10.0.4"
}
resource "azurerm_route" "databricks_direct" { # keep SCC relay DIRECT (skip firewall hop)
  address_prefix = "AzureDatabricks"           # service tag as route prefix (repeat: Storage, EventHub)
  next_hop_type  = "Internet"
}

Portal: NAT gateway → attach to both subnets; Route tables → 0.0.0.0/0 → Virtual appliance (firewall IP) + AzureDatabricks/Storage/EventHub → Internet → associate to both subnets. Never delete the managed NSG rules — add Deny rules on neighbor subnets instead.

3.5 · Compute → storage: public / service / private endpoints

Your cluster reads ADLS Gen2. By default that's a public endpoint; a storage firewall slams it shut and you re-open a controlled door via a Service Endpoint (free, backbone) or a Private Endpoint (paid, private IP). Switch the diagram between the three routes; click any box.

Public vs Service vs Private Endpoint

Illustrative config — Service Endpoint path (free)

resource "azurerm_subnet" "container" {
  service_endpoints = ["Microsoft.Storage"]            # <-- the Service Endpoint switch
  delegation { service_delegation { name = "Microsoft.Databricks/workspaces" } }  # still required
}
resource "azurerm_storage_account_network_rules" "lake" {
  default_action             = "Deny"                   # <-- closes the public door
  virtual_network_subnet_ids = [azurerm_subnet.host.id, azurerm_subnet.container.id]  # the VNet rule
}
# DNS unchanged. For Private Endpoint: a /28 PE subnet + privatelink.dfs.core.windows.net + public access Disabled.

Portal: Storage account → Networking → Enabled from selected VNets/IPs → add host + container subnets (auto-creates the SE) → Save. Add the VNet rule before setting Deny — the source-IP switch drops open connections.

Decision guide — what an architect recommends

Customer Q&A — what the security team actually asks

Common asks & the answer

What breaks in the field — and the FIRST check

Three questions to ask before recommending an architecture

1. Is there a regulatory mandate for no public-IP hop (control plane and/or storage), or is backbone-private enough?
2. What's the peak concurrent node count — and is there headroom in the VNet for a Private Link subnet later?
3. Does anything on-prem need to reach the same storage (forces Private Endpoint), and are you on Premium?