Security, Identity & Access

6.1 · Identity & Authentication

Microsoft Entra ID is the single source of truth: identities are defined once in the account and assigned to identity-federated workspaces. Switch the diagram between the provisioning model (how identities get in) and the authentication flow (how humans vs robots prove it), and click any box for the "what + why."

Provisioning — Automatic identity management vs SCIM

PAT vs OAuth — and token lifetime (the part reviewers probe)

Illustrative config — service principal + token policy

Databricks CLI — create a robot identity for prod and cap PAT lifetime. Full IaC is the hands-on artifact.

# Create a Databricks-managed service principal + OAuth M2M secret (account-admin auth).
databricks account service-principals create --json '{"displayName":"prod-job-runner"}'
databricks account service-principal-secrets create <SP_ID>   # store secret in Azure Key Vault

# Cap NEW PATs at 90 days (Databricks advises < 90). "0" reverts to the 730-day default.
databricks workspace-conf set-status --json '{ "maxTokenLifetimeDays": "90" }'
# Disable PATs entirely (warning: Partner Connect needs PATs ON):
databricks workspace-conf set-status --json '{ "enableTokensConfig": "false" }'

Portal: Account console → Security → User provisioning → Automatic identity management = Enabled; Account console → User management → Service principals → Add; Workspace → Settings → Advanced → Personal Access Tokens.

6.2 · Network access controls for users

Three layers gate the workspace front door, weakest to strongest. Switch the diagram between the gated public door (Conditional Access → context ingress + IP list) and the private tunnel (front-end Private Link), and click any box for the "what + why."

The three controls side by side

Illustrative config — IP list + front-end Private Endpoint

Workspace IP list (CLI) + the user-facing databricks_ui_api PE (Terraform). Full runbook is the Stage 3 hands-on.

# Enable first — lists are silently ignored until the flag is true. MUST allow the SCC/NAT egress IP.
databricks workspace-conf set-status --json '{ "enableIpAccessLists": "true" }'
databricks ip-access-lists create --json '{
  "label": "office-and-vpn", "list_type": "ALLOW",
  "ip_addresses": ["1.1.1.0/24", "203.0.113.7/32"]   # include the SCC/NAT egress IP!
}'

resource "azurerm_private_endpoint" "ui_api" {
  name                = "adb-fe-ui-api-pe"
  subnet_id           = azurerm_subnet.transit_pe.id        # PE subnet in the TRANSIT VNet
  private_service_connection {
    private_connection_resource_id = azurerm_databricks_workspace.this.id
    subresource_names              = ["databricks_ui_api"]  # exact sub-resource name
    is_manual_connection           = false
  }
  private_dns_zone_group {                                  # resolve URL to the PRIVATE IP
    private_dns_zone_ids = [azurerm_private_dns_zone.adb.id] # privatelink.azuredatabricks.net (fixed name)
  }
}

Portal: Conditional Access → target app 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d (AzureDatabricks), Report-only first; Private Link → add databricks_ui_api + browser_authentication PEs, delete-lock the web-auth workspace, verify nslookup returns the private IP.

Decision guide — what an architect recommends

Customer Q&A — what the security team actually asks

Common asks & the answer

What breaks in the field — and the FIRST check

Three questions to ask before recommending a posture

1. Is Entra ID the single source of truth, and are production jobs running as service principals (not humans)?
2. Is the mandate "users must be on our network" (IP lists suffice) or "no public internet path" (front-end Private Link)?
3. Are PATs capped/disabled — and does the customer understand they bypass Conditional Access?