Databricks DE · Topic 6.3

Genie Code — NL Transformations

Describe a data task in plain language; Genie Code generates or updates governed code, pipelines, and dashboards — UC-aware and permission-bound. This deep dive shows prompt → code, and the review discipline.

NL → codeUC permission-awareagent modereview before ship

The one-liner

Genie Code turns plain-language prompts into governed transformations/pipelines, using Unity Catalog metadata and respecting your permissions. In Designer it builds/edits transforms from prompts instead of manual operators.

👩‍💻 Analogy: a junior data engineer who pair-programs with you — say "monthly revenue per region," it writes the transform; keep it, tweak it, or say "now add last year for comparison."

1 · Prompt → transformation — click a prompt

▸ watch the generated (governed) code update conversationally

"total revenue by region"

"join orders to customers, last 90 days"

"...and exclude refunded orders"

💡 The output is production-ready, UC-governed code — review it like any PR before shipping.

2 · The generated code (SQL or PySpark)

Genie Code emits SQL or the PySpark equivalent (e.g. for a Designer Python operator). Prompts can reference your real schema because it reads Unity Catalog metadata.

# Prompt: "join customers, last 90 days, exclude refunds, revenue by region" → PySpark
from pyspark.sql import functions as F
(orders.join(customers, "customer_id")
       .where("order_date >= current_date() - INTERVAL 90 DAYS AND status <> 'refunded'")
       .groupBy("region").agg(F.sum("amount").alias("revenue")))

3 · The review discipline (enterprise-critical)

Treat generated code like a junior engineer's PR. AI can produce plausible-but-wrong logic — review before shipping, especially:

Joins — wrong key/grain → row explosion or silent duplicates.
Aggregations — double-counting after a fan-out join.
Filters / nulls / time windows — off-by-one day, timezone slips.
Deletes/overwrites in generated write steps.

⚠️ Because the output is real governed code, you can diff it, review it in a PR, and version it in Git — make that part of the workflow, not optional.

4 · The Genie family — click each

Genie Code▸

AI-native coding assistant: NL → code, pipelines, dashboards across notebooks, SQL editor, Lakeflow Pipelines, MLflow. Has an agent mode in Lakeflow Pipelines for autonomous ETL/SDP building. This is the data-engineering side.

Genie (AI/BI)▸

For business users — natural-language analytics / Q&A over governed data.

Genie Spaces▸

For data teams — configure trusted metrics and governance that power the Genie experiences.

Member	For	Does
Genie Code	engineers / analysts	NL → code/pipelines/dashboards
Genie (AI/BI)	business users	NL Q&A over data
Genie Spaces	data teams	trusted metrics / governance

Why it matters

Collapses "what I want" → "working transform" without hand-writing every step, while staying inside the governed lakehouse.
The AI-native NL-to-pipeline story is a hot 2026 interview theme — knowing Genie Code (and that it's permission-aware) shows current fluency.

🧑‍💼 Real-world use case: an analyst in Designer types "join orders to customers, keep last 90 days, total revenue by region"; Genie Code generates it, they refine with "exclude refunded orders," review it, and ship a governed pipeline.

Uses, edge cases & limitations

Uses: drafting/editing Designer transforms by prompt; scaffolding SDP pipelines; quick dashboard/notebook code; assisted debugging.
Edge case: AI output must be reviewed — verify logic, especially joins/aggregations and edge cases.
Limitation: it's a "Designated Service" with regional data-residency rules — features vary by region; ⚠️ verify availability. Permission-bound, so it can't surface data you can't already see.

Common gotchas

❌ Shipping Genie-generated transforms unreviewed.
❌ Assuming it bypasses governance — it's UC permission-aware, not a back door.
❌ Expecting identical features everywhere — availability varies by region.
❌ Confusing Genie Code (build code/pipelines) with Genie AI/BI (business Q&A).