Databricks · One-Pager · SQL & BI

Databricks SQL & Photon

Serverless-grade SQL warehouses running queries on Photon, the vectorized C++ engine that accelerates SQL and DataFrame work with no code change.

GA Photon on by default Serverless start 2–6s Verified Jun 2026 docs

In one line

Databricks SQL serves BI and analytics through SQL warehouses (classic / pro / serverless), each powered by Photon — Databricks' native, vectorized C++ query engine.

Why it matters

It is the SQL/BI front door to the lakehouse: dashboards, ad-hoc SQL, and federation hit a warehouse instead of a general cluster. Serverless gives second-scale startup and elastic autoscaling, and Photon makes the same SQL run materially faster with zero query changes.

Core concepts

Three warehouse types

Classic, Pro, and Serverless — all run Photon. Pro adds Predictive I/O; Serverless adds Intelligent Workload Management.

Serverless startup

Compute runs in the Databricks account and starts in 2–6 seconds, vs. ~4 minutes for classic/pro running in your cloud account.

Photon engine

Native C++ engine that processes data in columnar batches (SIMD, no JVM GC); a drop-in accelerator, compatible with Spark APIs.

Autoscaling

Classic/pro: ~1 cluster per 10 concurrent queries, scale-out by queued load. Serverless: IWM predicts demand and scales clusters dynamically.

Sizing & auto-stop

T-shirt sizes 2X-Small→5X-Large. Default 1 cluster (min=max=1). Auto-stop default 10 min serverless / 45 min pro & classic.

Result caching

Local (in-memory, cluster lifetime) + serverless-only remote result cache; both 24h TTL and invalidated when source tables change.

How it works

Click a stage — query path through a Photon warehouse

1 · BI / SQL client 2 · SQL warehouse 3 · Photon engine 4 · Result + cache

BI / SQL client — a dashboard, notebook, or JDBC/ODBC tool submits SQL to a warehouse endpoint.

Minimal example

SQL · Databricks CLI / API-- Create a serverless warehouse: starts in seconds, Photon on by default
databricks warehouses create --json '{
  "name": "analytics-wh",
  "warehouse_type": "PRO",          -- PRO + serverless flag = Serverless
  "enable_serverless_compute": true,
  "cluster_size": "Small",
  "auto_stop_mins": 10,             -- serverless default; stop when idle
  "min_num_clusters": 1,
  "max_num_clusters": 4
}'

Gotchas & limits

Photon falls back to standard Spark for unsupported work — Python/Scala UDFs, RDD/Dataset APIs, and stateful streaming run on the JVM, not C++.
Photon gives little benefit on tiny queries (under ~2s); it shines on large scans, joins, and aggregations.
Serverless is cheap to start but bills DBUs the moment it runs — set a tight auto-stop (default 10 min) and watch always-on warehouses.
Photon instances consume DBUs at a higher rate than the non-Photon runtime; faster wall-clock, different per-hour cost.

Interview soundbites

Say this out loud

"Photon is a vectorized C++ engine that's a drop-in accelerator for SQL and DataFrame workloads — no code change, you just see it in the query plan; unsupported ops like UDFs fall back to Spark."
"Pick serverless when available: 2–6 second startup vs ~4 minutes, compute in the Databricks account, plus Intelligent Workload Management for elastic autoscaling."
"Classic, pro, and serverless all run Photon; the difference is startup, where compute runs, and pro/serverless-only features like Predictive I/O and IWM."

References

Verified against docs.databricks.com · June 2026