Databricks DE · Topic 7.3

Triggers, Schedules & Monitoring

How a job starts (Quartz CRON / file arrival / table update / continuous), recovers (retries, partial re-run), and reports (monitoring + notifications + health SLAs). This deep dive pairs each with bundle YAML.

Quartz CRONfile-arrivalretriesnotificationshealth SLA

The one-liner

Triggers start a run (Quartz CRON, file arrival, table update, continuous, periodic, manual); retries & timeouts recover from blips (re-run only failed tasks); monitoring + notifications + health rules tell you what happened (email / webhook / Slack / SLA breach).
⏰ Analogy: triggers are the alarm clock (time) or doorbell (an event); retries are "try the key again if the door sticks"; notifications are the text message telling you it worked or broke.

Trigger types — click each

▸ what starts the run?

⏰
Scheduled (CRON)
📁
File arrival
🗃️
Table update
♾️
Continuous
🔁
Periodic
▶️
Manual
Scheduled (Quartz CRON) — time-based, e.g. 0 0 2 * * ? = 02:00 daily. Runs in the job's timezone_id.
File arrival — fires when new files reach a monitored Unity Catalog location (trigger.file_arrival.url). Pairs well with Auto Loader downstream.
Table update — fires when a source table is modified.
Continuous — auto-restarts on completion/failure for always-on jobs (compute stays up → cost).
Periodic — fixed interval (trigger.periodic.interval/unit), an alternative to cron.
Manual — "Run now" or via external orchestration / API.

Quartz CRON decoder — click a field

▸ 0 0 2 * * ? — Databricks uses 6-field Quartz (not 5-field Unix)

0
seconds
0
minutes
2
hours
*
day of month
*
month
?
day of week

1 · Schedule & triggers as code

schedule:
  quartz_cron_expression: "0 0 2 * * ?"   # 02:00 daily (sec min hr dom mon dow)
  timezone_id: "America/New_York"       # DST-aware; "UTC" avoids DST drift
  pause_status: UNPAUSED                  # or PAUSED
# event-driven instead of cron:
trigger:
  file_arrival: { url: "/Volumes/main/raw/orders/" }
# or: trigger: { periodic: { interval: 6, unit: HOURS } }

2 · Reliability — retries, timeouts, concurrency

timeout_seconds: 7200            # job kill-switch (2h)
max_concurrent_runs: 1           # default 1; raise only if overlap is safe
tasks:
  - task_key: ingest
    max_retries: 2                       # retry transient failures twice
    min_retry_interval_millis: 60000     # wait 60s between retries
    retry_on_timeout: true               # also retry on task timeout
    notebook_task: { notebook_path: ../src/ingest.py }
💡 Retries fix transient failures (flaky API, lost node), not deterministic bugs. On a fix, re-run only the failed tasks (repair run), not the whole job.

3 · Notifications & health (SLA)

email_notifications:
  on_failure: ["oncall@acme.com"]
  on_duration_warning_threshold_exceeded: ["oncall@acme.com"]
webhook_notifications:
  on_failure: [{ id: "${var.slack_webhook_id}" }]   # Slack / PagerDuty
health:
  rules:
    - metric: RUN_DURATION_SECONDS      # SLA: warn if a run exceeds 1h
      op: GREATER_THAN
      value: 3600

Health rules turn "it's slow" into an alert, not a silent miss. Monitor runs in the Runs tab and trend health/cost via system.lakeflow.* system tables.

Why it matters

🌙 Real-world use case: a gold pipeline runs on Quartz CRON 0 0 2 * * ?, retries transient failures twice, re-runs only failed tasks on manual fix, and emails + Slack-webhooks the on-call if the run fails or breaches its duration SLA.

Uses, edge cases & limitations

Common gotchas