Private Link, DNS & Topologies

4.1 · Private Link connection types

Three private extensions, one per direction. Switch the diagram between front-end, back-end, and web-auth — the highlighted arrow is the active path; click any box for the "what + why."

The three connection types side by side

Illustrative config — the three endpoints + the two flags

Terraform (azurerm). Full apply-ready module deferred; only DEP (4.4) ships a standalone main.tf.

resource "azurerm_private_endpoint" "backend" {        # BACK-END: in the WORKSPACE VNet
  subnet_id = var.workspace_pe_subnet_id                # dedicated /27
  private_service_connection {
    private_connection_resource_id = var.workspace_id
    subresource_names = ["databricks_ui_api"]           # SAME groupId as front-end; location decides the role
    is_manual_connection = false
  }
  private_dns_zone_group { private_dns_zone_ids = [azurerm_private_dns_zone.adb.id] }
}
# FRONT-END: same groupId, placed in the TRANSIT VNet.
# WEB-AUTH:  subresource_names = ["browser_authentication"] on the LOCKED web-auth workspace (1 per region).
# Flags: back-end/full = public_network_access_enabled=false + NSG "NoAzureDatabricksRules";
#        front-end hybrid = public_network_access_enabled=true + NSG "AllRules".

Portal: workspace → Settings → Networking → Private endpoint connections → + Private endpoint → Target sub-resource → transit (front-end/web-auth) or workspace (back-end) VNet → Integrate with private DNS zone = Yes.

4.2 · Private DNS for Private Link

Private Link gives you the private IP; DNS is the switch that decides whether anyone lands on it. Switch the diagram between the workspace name and the SSO name to see the CNAME→A override chain; click any box for the "what + why."

Automatic vs custom DNS

Illustrative config — zone, two records, VNet link

resource "azurerm_private_dns_zone" "adb" {
  name = "privatelink.azuredatabricks.net"               # FIXED zone name — do not change
  resource_group_name = "dns-rg"
}
resource "azurerm_private_dns_a_record" "workspace" {    # adb-<id>.<n> -> databricks_ui_api PE IP
  name = "adb-1234567890123456.7"                        # host LABEL only (zone supplies the suffix)
  zone_name = azurerm_private_dns_zone.adb.name ; resource_group_name = "dns-rg" ; ttl = 10
  records = [azurerm_private_endpoint.ui_api.private_service_connection[0].private_ip_address]
}
# webauth A record: name = "westus.pl-auth" -> browser_authentication PE IP (required for front-end SSO)
# Link the zone to EVERY VNet that must resolve privately (workspace + transit + spokes).

Custom DNS: conditional-forward *.azuredatabricks.net, *.privatelink.azuredatabricks.net, *.databricksapps.com to 168.63.129.16.

4.3 · Transit / hub-and-spoke

Build the private front door, DNS, and egress firewall once in a transit VNet, then peer every workspace spoke to it. Switch between the Standard and Simplified deployments; click any box for the "what + why."

Standard vs Simplified — blast radius vs cost/effort

Illustrative config — peering + PE placement + DNS link

resource "azurerm_virtual_network_peering" "ws_to_hub" {
  virtual_network_name      = azurerm_virtual_network.ws.name      # spoke
  remote_virtual_network_id = azurerm_virtual_network.transit.id   # hub
  allow_forwarded_traffic   = true                                 # let hub firewall forward spoke egress
  resource_group_name = "ws-rg" ; name = "ws-to-hub"
}
# FRONT-END PE in transit VNet, BACK-END PE in workspace VNet — same groupId, location decides the role.
# Link the ONE privatelink.azuredatabricks.net zone to BOTH VNets.

Subnet sizing: transit PE subnet /26–/25 · web-auth ws host+container /28 or /27 · classic back-end PE /27. Stop all compute before NSG/Private Link changes (15+ min).

4.4 · Data exfiltration protection (DEP)

Force every internet-bound packet through one inspected, default-deny chokepoint — but keep the control plane and your storage on private paths that bypass it. Switch the diagram between the four flows; click any box for the "what + why."

Egress controls for classic compute

Illustrative config — firewall allowlist + the forcing UDR

A focused, apply-ready main.tf ships alongside the lesson — DEP is the deliverable the network team applies.

resource "azurerm_firewall" "hub" {
  sku_name = "AZFW_VNet" ; sku_tier = "Standard"          # Standard = FQDN application rules
  ip_configuration {
    subnet_id = azurerm_subnet.fw.id                      # MUST be named "AzureFirewallSubnet"
    public_ip_address_id = var.fw_pip                     # stable egress IP for downstream allowlists
    name = "fw-ipconfig"
  }
  name = "adb-hub-fw" ; resource_group_name = "adb-hub-rg" ; location = var.region
  firewall_policy_id = azurerm_firewall_policy.dep.id
}
resource "azurerm_route" "to_firewall" {                  # the forcing function (BOTH subnets)
  address_prefix = "0.0.0.0/0" ; next_hop_type = "VirtualAppliance"
  next_hop_in_ip_address = azurerm_firewall.hub.ip_configuration[0].private_ip_address
  route_table_name = azurerm_route_table.spoke.name ; resource_group_name = "adb-rg" ; name = "to-firewall"
}
# With back-end Private Link on, NO AzureDatabricks service-tag route is needed.
# Allowlist (East US ex.): dbartifactsprodeastus.blob… , ucstprdeastus.dfs… , *.pypi.org , login.microsoftonline.com.
# Network rules: metastore 3306/TCP, Event Hubs 9093/TCP.

Decision guide — what an architect recommends

Customer Q&A — what the security team actually asks

Common asks & the answer

What breaks in the field — and the FIRST check

Three questions to ask before recommending an architecture

1. Is there a regulatory mandate for no public-IP hop / a provable internet allowlist, or is backbone-private enough?
2. Is the customer on Premium with VNet injection + SCC (prerequisites for back-end / DEP)?
3. Do they run their own DNS (custom DNS + conditional forwarding) — and is it classic or serverless (DEP vs NCC)?