Latest Exam Updates

The one-liner

Rename flashcards — click to flip old → current

▸ know both: old names still appear in docs & dumps

What changed — with current code

1 · Lakeflow naming — the rebrand the exam expects

Mechanism: tools unified under Lakeflow (Connect · SDP · Designer · Jobs); the pipeline API moved to pyspark.pipelines (as dp). Trade-off: old names persist in docs — recognize both, answer current.

# Current SDP Python — NOT legacy "import dlt / @dlt.table / dlt.apply_changes":
from pyspark import pipelines as dp

@dp.table()                                   # was @dlt.table
def orders_clean():
    return spark.readStream.table("bronze.orders")

dp.create_auto_cdc_flow(                       # was dlt.apply_changes
    target="silver.customers", source="cdc_feed",
    keys=["id"], sequence_by="ts", stored_as_scd_type=2)

2 · Data-level security — row filters & column masks

Mechanism: a SQL UDF attached as a row filter (limits rows) or column mask (transforms a column) — Stage 8.3. Why: Governance is 15% and expanded. Trade-off: runs per query — keep UDF logic cheap.

-- Row filter: a UDF returning BOOLEAN, attached to the table.
CREATE FUNCTION sec.region_filter(region STRING)
  RETURN is_account_group_member('admins') OR region = current_user();
ALTER TABLE prod.sales.orders SET ROW FILTER sec.region_filter ON (region);

-- Column mask: a UDF returning the (possibly masked) value.
CREATE FUNCTION sec.mask_email(e STRING)
  RETURN CASE WHEN is_account_group_member('pii_readers') THEN e ELSE '***' END;
ALTER TABLE prod.sales.customers ALTER COLUMN email SET MASK sec.mask_email;

3 · ABAC — attribute-based access with governed tags

Mechanism: a policy applies masks/filters by tag (e.g. all pii-tagged columns) instead of one-by-one — Stage 8.4. Trade-off: ⚠️ verify — ABAC GRANT/CREATE POLICY has been Beta; confirm GA against docs.

-- ABAC: one policy masks every column carrying the 'pii' tag value 'email'.
CREATE POLICY mask_pii_email
  ON SCHEMA prod.sales
  COLUMN MASK sec.mask_email
  TO `analysts`
  FOR TABLES
  MATCH COLUMNS hasTagValue('pii', 'email') AS c
  ON COLUMN c;   -- ⚠️ verify exact syntax/GA against current docs

4 · Liquid Clustering — added to the syllabus

Mechanism: CLUSTER BY replaces hive partitioning + ZORDER, and is changeable later — Stage 3. Why: now the recommended layout; appears instead of partitioning/ZORDER. Trade-off: still compact via OPTIMIZE / Predictive Optimization.

-- Liquid Clustering — the modern layout (not PARTITIONED BY / ZORDER):
ALTER TABLE prod.silver.events CLUSTER BY (event_date, region);
OPTIMIZE prod.silver.events;        -- compaction; clustering applied here

5 · PySpark / Spark-SQL ETL — still core

Mechanism: read → transform → write to Delta with the 3-level UC namespace. Why: the Transformation domain (22%) assumes DataFrame fluency. Trade-off: Spark core internals are out of this plan's scope — review separately if rusty.

# Bread-and-butter ETL the exam assumes you can write:
df = spark.read.table("main.bronze.orders").filter("amount > 0")
(df.groupBy("region").sum("amount")
   .write.mode("overwrite").saveAsTable("main.silver.revenue_by_region"))

What's emphasized — click each

Why it matters

• Exam questions & interviewers use current names; old dumps use old ones — a right concept with a stale name can still cost the question.
• Data-level security (15%) and optimization (10%) are heavily weighted and recently expanded — exactly Stages 3 & 8.

Uses, edge cases & limitations

• Uses: a final naming & recent-emphasis pass; sanity-checking old notes use current terms.
• Edge cases: DLT still appears in docs/UI/questions — know it = Lakeflow SDP; ABAC GRANT/CREATE POLICY has been Beta (⚠️ verify GA).
• Limitation: emphasis/weights shift between versions — this reflects the current guide; re-verify on the official exam guide near your date.

Common gotchas

• ❌ Answering with old names (DLT/Workflows/Repos/apply_changes).
• ❌ Under-studying governance (15%) and optimization (10%).
• ❌ Treating Liquid Clustering as optional — it's the recommended layout.
• ❌ Assuming every new feature is GA (ABAC policies have been Beta — verify).