Azure Databricks · Networking & Security · Stage 11

FDE Interview Prep Capstone

The synthesis topic — for the FDE/RSA who has to draw the secure design, answer the rapid-fire probe, and defend the cost across the table from a customer's security and FinOps teams. Three doors, two compute planes, and every control traced to a driver and a dollar figure.
Azure-first 11.1 Whiteboard 11.2 Rapid-fire Q&A 11.3 Trade-offs & Cost Architect altitude

Draw it, answer it, defend it

This capstone turns everything from Stages 1–9 into three field skills: 11.1 design and trace a secure deployment hop-by-hop; 11.2 give crisp answers that survive the follow-up probe; 11.3 justify each control against a driver and put real Azure dollars on it. The map underneath all three is the same: three connectivity paths, two compute planes.

🧠 Mental model — hold this in your head

You are a security guard walking a VIP from the street to the vault — then the doctor who decides which locks are worth the money. Three doors: ① user → workspace, ② compute → control plane, ③ compute → storage/egress. Classic lives in your VNet (SCC + back-end PL + storage PE); serverless lives in Databricks' account (NCC; CP leg is always backbone TLS). Every control must trace to a driver — no driver, the cheaper default wins.

The three doors × two planes — the whole topic on one table

PathClassic (your VNet)Serverless (Databricks account)Strongest
① User → workspaceIP access list → front-end PLSame front door (shared CP)Front-end PL + public access Disabled
② Compute → control planeSCC relay → back-end PLBackbone TLS (managed)Back-end Private Link
③ Compute → storageService / Private EndpointNCC service-tag firewall / PEPrivate Endpoint + storage public access Disabled

Static topology (the "complete private isolation" end-state)

Secure Azure Databricks end-state: analyst reaches the workspace via a
      front-end private endpoint and web-auth PE in a transit VNet (path 1); the classic container subnet dials
      outbound to the SCC relay via a back-end private endpoint (path 2); classic reaches ADLS via a storage
      private/service endpoint and serverless reaches it via an NCC private endpoint (path 3); each control tagged
      with its driver and rough cost.
Front-end + back-end + web-auth Private Link + NCC + Public access Disabled = Microsoft's "complete private isolation." The optional Azure Firewall (DEP) is the ~$900/mo cliff.

The line a security team wants to hear

"We secure all three paths privately and set Public network access = Disabled. Classic uses SCC + back-end Private Link; serverless uses NCC; the data never traverses the control plane and you hold the encryption keys. And I'll tell you which of these you don't need to pay for."

Terms used here (full deep-dive lives in earlier stages)

SCC relay
Endpoint classic clusters dial outbound to (443) so nothing dials inbound. → Stage 2.3 / 4
Back-end / front-end PL
Private endpoints (databricks_ui_api) for compute→CP and user→workspace; web-auth = SSO callback. → Stage 6
NCC
Account-level, regional object giving serverless its private endpoints / service-tag egress. → Stage 7
Service / Private Endpoint
Free backbone route by subnet vs a private IP in your VNet (per-GB). → Stage 5 / 6
DEP
SCC + VNet injection + UDR → Azure Firewall FQDN allowlist. → Stage 8
CMK
Customer-managed at-rest keys in Azure Key Vault (Premium). → Stage 9

11.1 · Whiteboard scenarios — trace the packet, name the control

The classic senior-interview / security-workshop exercise: design a secure deployment and walk a query from the analyst's browser to data in ADLS, naming the control at every hop. Switch the diagram between the regulated bank (classic), healthcare (serverless-first), and the cost-conscious DEP walk-throughs; click any box for the "what + why."

TipPick a scenario, then click a box to see the hop and the control that protects it.

The end-state has a name

Securing all three paths privately and setting Public network access = Disabled is Microsoft's "Complete private isolation" — front-end + back-end + web-auth Private Link + NCC for any serverless. Prereqs: Premium + VNet injection + SCC.

The controls, hop by hop (the whiteboard narration)

HopControlWhat it does
FoundationVNet injectionWorkspace in your VNet (host + container subnets) — the only way to attach back-end PL, NSGs, UDRs, storage PE
② baselineSCC / No Public IPCluster dials outbound to the SCC relay; no public IP, no inbound ports
②Back-end PL (databricks_ui_api)Compute→CP on a private IP; NoAzureDatabricksRules; ports 443/6666/3306/8443-8451
①Front-end PL + web-authUser→workspace private; browser_authentication PE makes Entra ID SSO work (1 per region/DNS zone)
gluePrivate DNSprivatelink.azuredatabricks.net resolves the FQDN to the private IP — the #1 break
③ serverlessNCC private endpointServerless→ADLS privately (PENDING→ESTABLISHED); 10/region, 100 PE/region, 50 ws/NCC
③ classicService / Private EndpointStorage firewall by subnet (free) or a private IP (per-GB, on-prem reach)
at restCMK + Unity Catalog + auditCustomer key; grants/masks above the network; system.access.audit evidence

Illustrative config — the foundation (VNet injection + SCC + isolation)

Terraform (azurerm). Full multi-file IaC (PEs, DNS, firewall) is the Stage 3–8 hands-on artifact.

resource "azurerm_databricks_workspace" "bank" {
  sku                                   = "premium"               # PL/CMK/UC binding need Premium
  public_network_access_enabled         = false                   # "complete private isolation"
  network_security_group_rules_required = "NoAzureDatabricksRules" # back-end PL: managed public rules off
  custom_parameters {
    no_public_ip        = true                                    # SCC / NPIP
    virtual_network_id  = azurerm_virtual_network.spoke.id
    public_subnet_name  = azurerm_subnet.host.name                # "host" subnet
    private_subnet_name = azurerm_subnet.container.name           # "container" subnet
  }
}

Portal: Create → Networking → No Public IP = Yes, Deploy in your VNet = Yes, Required NSG rules = NoAzureDatabricksRules, Public network access = Disabled → add the back-end/front-end databricks_ui_api PEs + one browser_authentication PE → verify privatelink.azuredatabricks.net A-records.

The most common whiteboard slip

Mixing the two planes' mechanisms. Back-end PL secures classic compute→CP; NCC secures serverless→data. They are not interchangeable. And serverless has dynamic IPs — you can't peer or pin a static IP, which is the whole reason NCC exists.

11.2 · Rapid-fire Q&A — which control on which path, and what it costs

Crisp model answers that survive the follow-up probe. Switch the diagram between the three paths — the highlighted arrow is the active path; click a box for the one-line answer and the load-bearing fact (port, limit, direction).

TipPick a path, then click a box for the model answer and the fact interviewers grade on.

The probes everyone asks (one-liner + the fact that's graded)

QuestionThe answer (bold = load-bearing)
Trace the SCC callCluster dials outbound to the SCC relay over 443, holds it open; CP replies down that tunnel — never dials in. Default in ARM 2024-05-01+.
SCC vs Private LinkSCC removes inbound + node public IPs; the CP hop stays public-on-backbone until back-end PL makes it a private IP. PL lets you set public access = Disabled.
What's web-auth?browser_authentication PE for the Entra ID SSO callback — one per region per DNS zone; REST auth unaffected.
Ports on the PE subnet NSG443, 6666, 3306, 8443–8451 inbound.
Serverless → private ADLS?NCC private endpoint (PENDING→ESTABLISHED); can't whitelist serverless IPs — they're dynamic. Limits: 10/region, 100 PE/region, 50 ws/NCC.
Size the subnetsVNet /16–/24; two equal subnets ≥ /26; 2 IPs/node, 5 reserved; CIDR immutable after deploy.
SE vs PESE = free, subnet trust, storage keeps a public IP; PE = private IP/NIC, DNS change, per-GB both ways, on-prem reach.

The DNS-first reflex

Most front-end PL failures are DNS. First move on any "can't reach workspace / OAuth denied": nslookup the workspace FQDN from the client subnet and confirm it returns the private PE IP in privatelink.azuredatabricks.net, not the public one.

Illustrative config — an NCC private endpoint rule (serverless → ADLS)

# Account-level, regional. Storage owner APPROVES in the portal: PENDING -> ESTABLISHED.
databricks account network-connectivity create-private-endpoint-rule "$NCC_ID" --json '{
  "resource_id": "/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.Storage/storageAccounts/<acct>",
  "group_id": "dfs"
}'
# Then attach the NCC to the workspace, set storage Public network access = Disabled, restart serverless.

Don't say this

"Just whitelist the serverless IPs" (they're dynamic — it's NCC). "SCC makes traffic private" (it removes inbound; back-end PL privatizes the hop). "Route SCC through the firewall" (needless hop/latency). "Private Link is free because endpoints are cheap" (per-GB on hot paths bites).

11.3 · Defending trade-offs & cost — be a doctor, not a salesman

Every control must trace to a driver; no driver → over-build. Switch the diagram between the cheaper baseline and the hardened / mandated design — the labels carry the rough Azure cost on each control. Click any box for the driver and the price.

TipClick a box to see the driver that justifies the control — and the cheaper default if there's no driver.

Cost rule of thumb (East US list, region-varying — say "approximately")

Service Endpoint = free · NAT Gateway = ~$32/mo flat · Private Endpoint = ~$7/mo to exist + ~$0.01/GB both ways (a TB/day path ≈ ~$300/mo just for the PL premium) · Azure Firewall = ~$900/mo flat + $0.016/GB (double it for HA). The Firewall is the cliff — reserve it for a genuine exfiltration mandate.

Anchor each control to a driver

ControlDriver that justifies itNo driver → cheaper default
VNet injection + SCCAny prod workspace; own NSGs/UDRs/egress(baseline; managed VNet for POCs)
Back-end Private Link"No internal traffic on the public internet"SCC alone (outbound TLS on backbone)
Front-end Private Link"UI only reachable from our network"IP access lists + Conditional Access
Private Endpoint to ADLSPrivate-only storage / regulated dataService Endpoint + storage firewall (free)
Azure Firewall + UDR (DEP)Explicit FQDN egress allowlist / exfil mandateNSG egress + NAT Gateway
CMK (Key Vault)Customer must hold/rotate keysMicrosoft-managed keys (default, free)

The defense sentence template

"We're adding [control] because you told us [driver]. It costs [rough number] and adds [latency/ops]. If [driver] weren't a hard requirement, I'd recommend [cheaper default] instead."

Illustrative config — prove cost with the billing system table (SQL)

-- Show a customer their ACTUAL serverless networking spend, not a guess.
SELECT u.usage_date, u.sku_name,
       SUM(u.usage_quantity)                     AS qty,
       SUM(u.usage_quantity * p.pricing.default) AS approx_usd
FROM system.billing.usage u
JOIN system.billing.list_prices p
  ON u.sku_name = p.sku_name
 AND u.usage_end_time BETWEEN p.price_start_time AND COALESCE(p.price_end_time, current_timestamp())
WHERE u.sku_name ILIKE '%NETWORK%'                -- networking-related SKUs
  AND u.usage_date >= current_date() - INTERVAL 30 DAYS
GROUP BY u.usage_date, u.sku_name ORDER BY u.usage_date DESC;

Who pays: classic = you pay Azure directly; serverless = Databricks meters it and passes it through (billing system table). Serverless Private Connectivity /GB is waived until further notice; the /hour endpoint fee still bills.

Decision guide — what an architect recommends

Complete private isolation

  • Regulated bank / healthcare (PCI / HIPAA)
  • Documented "no public internet" mandate
  • VNet injection + SCC + back-end + front-end + web-auth PL
  • + NCC for any serverless, CMK, UC masks, audit
  • Public network access = Disabled (Premium)

Baseline (most enterprises)

  • Internal analytics, non-sensitive data
  • VNet injection + SCC + NAT (~$32/mo)
  • Service Endpoint to ADLS (free, backbone)
  • IP access lists if a cheap front-door win
  • Private-enough without per-GB / firewall spend

Rule of thumb

Baseline first, harden on a driver. VNet injection + SCC + NAT + service-endpoint storage firewall is cheap and private-enough for most. Add back-end PL + PE-to-ADLS only on a no-public-internet mandate; add the Azure Firewall (DEP) only on an explicit FQDN-allowlist requirement. Confirm Premium first; price in Azure's terms ("approximately — here's the live pricing page").

Customer Q&A — what the security & FinOps teams actually ask

The one-liner
"We secure three paths — who connects, how compute talks to the brain, how compute reaches your data — for both classic and serverless. Each gets a private channel and an identity/governance control on top; you hold the keys and every action is audited. My job is to match controls to your threat model and budget — and tell you the cheapest design that satisfies your drivers."

Common asks & the answer

Customer asks…Architect answers…
"Prove no traffic ever hits the public internet."In complete isolation — front-end + back-end PL + NCC + public access Disabled. Walk it hop-by-hop with the doc behind each claim.
"How does serverless reach our private ADLS?"NCC private endpoints; you can't peer to serverless (dynamic IPs) — that's why NCC exists.
"Who holds the encryption keys?"You do — CMK via your Azure Key Vault (Premium).
"This looks expensive — what's driving it?"Azure Firewall ~$900/mo flat is the cliff; Private Endpoint per-GB on hot data paths is the sleeper. Usually one line item, not "Databricks is expensive."
"Can we drop the firewall / PEs and stay secure?"Yes if there's no regulatory driver — pivot to service endpoints / NSG egress.
"What do we need Premium for?"All Private Link, IP access lists, CMK, CSP. Confirm the plan before scoping.

What breaks in the field — and the FIRST check

Front-end access fails after enabling PL

Workspace URL resolves to the public IP, not the private endpoint.First check: nslookup the FQDN from the client subnet — is the zone linked / forwarding correct?

Cluster won't start, "Control Plane Request Failure"

Back-end PL / SCC relay path misconfig.First check: back-end PE + NSG NoAzureDatabricksRules.

Web login broken region-wide

The single web-auth host workspace was deleted.First check: browser_authentication host workspace exists + PE healthy.

Serverless query can't reach storage

NCC PE still PENDING, or storage went Disabled before approval.First check: NCC PE = ESTABLISHED, attached to the workspace.

Cluster won't start, "insufficient IPs"

Container subnet exhausted at peak.First check: free IPs vs peak nodes minus 5 reserved (not a quota issue).

Surprise networking bill (FinOps)

One control is the driver.First check: system.billing.usage networking SKUs — usually a Firewall or high-volume PE path.

A control won't enable

Workspace is on Standard.First check: pricing tier — Private Link / CMK / IP ACLs are Premium-only.

Latency after hardening

SCC / storage egress forced through the firewall.First check: is the relay going direct via NAT, not the firewall hop?

Three questions to ask before recommending an architecture

  1. Is there a regulatory mandate for no public-IP hop, or is backbone-private enough?
  2. Full network-path control (custom firewall, UDR, DEP), or is speed/low-ops the priority?
  3. Where will governed data live — and are you on Premium?