Azure Databricks · Networking & Security · Stage 12

Customer-Facing Collateral

The three artifacts an FDE/RSA puts on the table: the sizing & architecture decision guide (12.1), the six-pillar security overview (12.2), and the deployment & hardening checklist (12.3). This module doesn't teach new controls — it assembles Stages 1–11 into deliverables a customer's security team can sign off.
Azure-first 12.1 Sizing & topology 12.2 Six pillars 12.3 Hardening checklist Architect altitude

Design it, pitch it, sign it off — the same building

Stage 12 is the meta layer: it doesn't add a new connectivity path, it decides how hard to lock down all three at once and proves it. 12.1 pours the foundation (sizing you can't redo) and counts the doors (Managed → Hardened → Isolated). 12.2 is the guided tour of six trust pillars. 12.3 is the pre-flight checklist where every switch has a confirmation light (the verify step).

🧠 Mental model — hold this in your head

Design, pitch, and sign-off for one building. A fixed-size plot of land (the VNet — footings you can't widen), wrapped in a progression of gates you bolt on only when needed, explained through six pillars, and signed off with a 13-item inspection. Secure by default, hardened by configuration, sized once for peak — and we prove every control is on, not just configured.

The three artifacts mapped to the three paths

ArtifactWhat it deliversTouches
12.1 Sizing & decision guideCIDR/subnet worksheet + Managed/Hardened/Isolated treeShapes the VNet (paths ②③); chooses controls across all three
12.2 Security overviewSix pillars: isolation, network, encryption, identity, compliance, auditOverlays all three paths; default vs opt-in
12.3 Hardening checklist13 items, each with why / how / verifyOne control bolted onto each hop of ①②③

Static topology (zoom-crisp, reusable in slides)

Azure Databricks Stage 12 customer collateral: a peak-sized VNet-injection
      workspace with host/container subnets, a Private Link subnet and Private DNS, a transit/hub VNet with
      front-end and web-auth private endpoints, the control plane and serverless plane in the Databricks
      subscription, ADLS Gen2 and Key Vault in the customer subscription, with the three connectivity paths and
      the numbered hardening-checklist controls labelled.
The same map as the interactive diagrams — sized VNet (12.1), six pillars (12.2), numbered hardening controls ☐n (12.3) along paths ①②③.

The line a security team wants to hear

"We size the network once for your peak so you never rebuild, we add exactly the locked doors your regulator requires, and we hand you a checklist where every control has a confirmation light — not a marketing claim, a verifiable control with a doc behind it."

Terms used here (full deep-dive lives in the owning module)

VNet injection
Classic compute in your VNet so you own subnets/NSGs/routes. → M2.1
SCC / No Public IP
Clusters dial outbound :443; no inbound ports. → M2.3
Private Endpoint / Link
Private NIC+IP mapping a service to a private address; needs DNS. → M2.5/M3.1
Private DNS zone
privatelink.azuredatabricks.net — resolves the URL to the PE IP. → M3.2
NCC
Account-level, regional object giving serverless stable egress + private endpoints. → M4a.2
CMK
Your keys in Key Vault / Managed HSM; revoke to cut access. → M4b.4
CSP
Compliance Security Profile — CIS-L1 image, ESM, auto-update, TLS 1.2+. → M4b.6
IPAM
The customer's IP Address Management team — source of a non-overlapping CIDR block.

12.1 · Sizing & architecture decision guide

Two deliverables: a CIDR/subnet worksheet (size once for peak — subnet CIDR is immutable) and a topology tree (Managed → Hardened → Isolated). Switch the diagram between the sized VNet and the three reference architectures; click any box for the "what + why."

TipClick any component to see what it is and why it's there.

The two unfixable mistakes

Subnet CIDR is immutable after the workspace deploys — size for peak (+~30%), equal host/container, VNet 2+ bits larger, a /27 Private Link subnet reserved. And over-building the topology (Isolated + firewall) when an IP access list was the ask burns cost and weeks of DNS work. Match the architecture to the requirement.

Subnet size → node ceiling (usable = 2^(32−n) − 5)

Subnet CIDRUsable IPs≈ Max nodes (all clusters)Good for
/2659~59Small/dev
/24251~251Typical prod
/221,019~1,019Large prod / many jobs
/204,091~4,091Exceptional — usually split workspaces

The three reference architectures

DimensionManagedHardenedIsolated
Public endpointOpen / NSG-limitedIP access listsDisabled (front-end PL)
Compute → controlSCC (backbone)+ classic Private Linkclassic Private Link
Exfiltrationminimalegress control+ Azure Firewall (hub-spoke)
Cost / effort$$$$$$ (NAT+PE+firewall+DNS)
TierPE optionalPremiumPremium

SCC + VNet injection are in all three — the baseline, not an upgrade.

Illustrative config — sized, delegated subnets (immutable)

Terraform (azurerm) — the CIDRs that set the node ceiling. Full VNet/NSG/NAT module is in the M2/M3 hands-on artifacts.

resource "azurerm_virtual_network" "adb" {
  address_space = ["10.50.0.0/16"]          # /16: room for big subnets + a PL subnet
}
resource "azurerm_subnet" "container" {     # "private subnet" — MUST equal host size
  address_prefixes = ["10.50.1.0/24"]       # /24 = 251 usable -> ~251 node ceiling
  delegation { name = "databricks-del"
    service_delegation { name = "Microsoft.Databricks/workspaces" } }   # required
}
# 10.50.2.0/27 left free for the back-end Private Link endpoint subnet.

Portal: Virtual networks → + Create → set address space → + Add subnet ×2 (equal, non-overlapping) → Azure Databricks → Networking → Deploy to your VNet = Yes. ⚠️ CIDR can't change after Create.

2026 deadline to flag

After 2026-03-31, new Azure VNets default to no outbound internet — a new workspace needs an Azure NAT Gateway on both subnets (also a stable egress IP) or clusters can't reach the control plane.

12.2 · Customer security & networking overview

The "is it secure?" briefing — six pillars, each a named, verifiable control. Switch the diagram between the six pillars view (where each pillar attaches) and the default vs opt-in view; click a box for the "what + why."

TipClick any component to see the pillar it anchors and why.

The one sentence

"Secure by default, hardened by configuration." The baseline (isolation, SCC, TLS, Entra ID, Unity Catalog, audit) is on out of the box on Premium; Private Link, CMK, and CSP are opt-in controls you add only when a named requirement justifies the cost. Every claim must match the deployed config.

Default vs opt-in

ControlDefault (Premium)Opt-inAnswers
Plane separation / SCC✅—"Where does data run? Inbound exposure?"
Encryption at rest✅ platform-managedCMK"Can we hold the keys?"
Entra ID SSO / SCIM / UC✅Conditional Access · ABAC (Preview)"One identity? Row/column control?"
Front/back-end Private Link · NCC—✅ Premium"Off the internet? Trust serverless egress?"
Compliance Security Profile—✅ Add-on"PCI/HIPAA-ready?"
Audit (system.access.audit)✅ (Preview)SIEM export"Who did what?"

Illustrative config — prove the audit trail live

SQL — the query an FDE runs in front of the security team. Nothing lands better than showing the log.

SELECT event_time, user_identity.email AS who, service_name, action_name,
       source_ip_address, response.status_code
FROM system.access.audit
WHERE event_date >= current_date() - INTERVAL 7 DAYS
  AND response.status_code >= 400          -- 4xx/5xx = denied or errored
ORDER BY event_time DESC LIMIT 100;

Be precise — three over-claims that lose trust

CMK scope: covers managed services / DBFS root / managed disks — not serverless ephemeral disks. Serverless ≠ classic for CSP standard/region coverage. And don't say "fully isolated" unless front+back-end Private Link and public access Disabled are deployed.

12.3 · Deployment & hardening checklist

The pre-flight checklist — 13 items, each with why · how · verify. The whole list is the three-path scaffold with one control bolted onto each hop. Switch the diagram between the three connectivity paths to see which checklist items lock each; click a box for the proof.

TipPick a path, then click a box to see the checklist item, why it matters, and how to verify it.

"Configured" ≠ "effective" — the discipline

Half of field incidents are a control that was configured but silently not effective (an NSG rule overwritten, a Private DNS zone not linked, an audit schema never enabled). Every item has a one-command/one-click verify — read the light, don't assume the gear is down.

The 13 items (why / verify)

#ItemWhyVerify
1VNet injection + sizingOwn NSGs/UDRs/DNS; foundation for PL. CIDR immutable.Subnet shows NSG + delegation; cluster IPs in container range.
2SCC / No Public IPNo inbound attack surface.NICs have no public IP; enableNoPublicIp = true.
3Back-end Private LinkRemoves last public hop for classic.PE databricks_ui_api Approved; nslookup → private IP.
4Front-end PL + web-authPrivate user path; SSO over private. 1 web-auth/region.URL → transit PE IP; SSO completes.
5Private DNSPE useless if DNS returns public IP.nslookup from VNet → private IP.
6NCC (serverless)VNet injection doesn't cover serverless.NCC binding + PE rules Established; serverless query succeeds.
7Public access = DisabledRejects all public connections.Outside corp net: URL unreachable.
8CMKYou hold/revoke keys.Encryption blade shows Key Vault URI.
9IP access listsRestrict front-end to known IPs.Off-list blocked; egress IP in ALLOW.
10Entra ID SSO+SCIM+MFACentral identity; auto-deprovision.Test user syncs/de-syncs; MFA enforced.
11Cluster policies + access modesForbid legacy No Isolation Shared.Legacy mode rejected for non-admins.
12Audit loggingImmutable who-did-what. (Preview.)Query returns rows; schema enabled.
13Terraform IaCReproducible, drift-detectable.terraform plan = no diff.

Illustrative config — the SCC + VNet-injection workspace

resource "azurerm_databricks_workspace" "this" {
  sku = "premium"                            # Premium -> PL, CMK, IP ACLs, CSP
  custom_parameters {
    no_public_ip       = true                # SCC / NPIP (item 2)
    virtual_network_id = azurerm_virtual_network.adb.id   # VNet injection (item 1)
  }
  public_network_access_enabled = false      # item 7 (with front+back-end PL in place)
}

IP ACLs: databricks workspace-conf set-status --json '{"enableIpAccessLists":"true"}'. NCC: Account Console → Cloud resources → Network Connectivity Configurations.

Decision guide — what an architect recommends

Managed security

  • Startup / internal analytics, no regulatory driver
  • Baseline: VNet injection + SCC + UC + Entra ID + audit
  • Public endpoints OK; classic PE optional
  • Dev/sandbox: checklist items 1–2 + 10–13

Hardened connectivity

  • Must restrict who connects / control serverless egress
  • + IP access lists + serverless egress control
  • + classic & serverless Private Link (Premium)
  • Add CMK if "hold our own keys"

Isolated environment

  • Regulated (FSI / healthcare / gov) or "no public internet" mandate
  • Front+back-end PL + NCC + public access Disabled
  • Hub-spoke + Azure Firewall; CSP for the regime
  • Full checklist; Premium; accept NAT/PE/firewall cost

Rule of thumb

Start at the baseline, step up only when a named requirement justifies the cost — and prove every control is on, not just configured. Size once for peak (the cheapest insurance against a rebuild), and tie every opt-in control (Private Link, CMK, CSP — all Premium) to a regulatory or data-residency requirement, never "nice to have."

Customer Q&A — what the security team actually asks

The one-liner
"We size the network once for your peak so you never rebuild, we add exactly the locked doors your regulator requires — no more — and we hand you a 13-item checklist where every control has a confirmation light. Secure by default, hardened by configuration, and proven, not asserted."

Common asks & the answer

Customer asks…Architect answers…
"How big should subnets be — can we change them later?"Size for peak via the worksheet; no, subnet CIDR is immutable — a rebuild is the only redo.
"Which architecture do we need?"Run the tree: regulated/exfiltration → Isolated; ingress/egress control → Hardened; else Managed. Don't over-build.
"Does any traffic touch the public internet?"With SCC + front+back-end PL + public access Disabled + NCC for serverless — no public path.
"Can we hold and revoke our own keys?"Yes — CMK from Key Vault across managed services, workspace storage, and disks (not serverless disks).
"Are you PCI/HIPAA-ready, and what do we do?"Enable CSP + the right standard; it's shared responsibility — config + free-text fields are yours.
"Prove the audit trail."Run a live system.access.audit query in front of them.
"How do we prove each control is on?"The verify column — each is one command or Portal check.

What breaks in the field — and the FIRST check

"Cluster failed to start — insufficient IPs"

Container subnet too small for peak autoscale.First check: container subnet free-IP count vs peak nodes, minus the 5 reserved.

Clusters won't launch after enabling IP ACLs

The compute stable-egress IP isn't in the ALLOW list.First check: egress IP in ALLOW (or back-end PL configured) — not quota.

Workspace URL unreachable / SSO fails after Private Link

DNS still returns the public IP, or web-auth missing.First check: nslookup the host + the single regional browser_authentication endpoint.

Serverless can't reach firewalled ADLS

A classic NSG/UDR was applied; serverless needs NCC.First check: NCC binding + private-endpoint rule, not classic networking.

"Audit table is empty"

The system schema was never turned on.First check: system.access schema is enabled at the account.

"The overview doesn't match what we deployed"

PL/CMK/CSP are opt-in, not automatic.First check: workspace tier + which add-ons are actually enabled.

Three questions before recommending an architecture

  1. Is there a named regulatory mandate for no public-IP hop / data exfiltration protection — or is backbone-private enough?
  2. Do they need to restrict who connects and control serverless egress, or just secure defaults?
  3. What's the five-year peak node count (sizing), and are they on Premium (Private Link / CMK / IP ACLs / CSP are Premium-only)?