Databricks Certified Data Engineer Professional

Not this time. SCD type 2 keeps the whole history: AUTO CDC adds a new row for each version of a record, with __START_AT and __END_AT columns for when it was in force, and sequence_by puts out-of-order changes in the right order. The default, SCD type 1, updates the row in place, so only the current address is kept. Time travel is not a history model: VACUUM removes old files after the retention threshold, 7 days by default. A newest-row view and an overwriting MERGE both keep only the current address.

Not this time. Databricks says joins in streaming tables do not recompute when a dimension changes, which is fine when fast matters more than exact, and that materialized views are always correct because they recompute joins when dimensions change. Processing each row once is why streaming tables suit ingestion, and it is the reason old orders would keep the old category here. A longer watermark only changes how late an order may arrive. A temporary view is not stored, so it is not a table for analysts to query.

Not this time. When foreachBatch writes to Delta tables, Databricks says you must use the txnAppId and txnVersion write options, binding txnVersion to the batch ID, so a batch that is retried is not written again. It also says not to catch and retry exceptions locally, but to let errors propagate. A new checkpoint loses the record of which batches were done and reprocesses data. A different trigger changes how often batches run, not whether a retried batch is written twice.

Not this time. Lakeflow Connect's database connectors ingest from relational databases, MySQL, PostgreSQL and SQL Server among them, using change data capture, so each change is carried over without code of your own. Lakehouse Federation queries an external database without migrating the data, which is the opposite of a copy in Unity Catalog. Nightly exports and full JDBC reloads are custom pipelines that reread everything and miss the order of changes in between.

Not this time. On VARIANT data, : selects a top-level field and :: casts the result, the shorthand for variant_get(raw, '$.device', 'string'). A dot selects nested fields further down the path, not the first field. Field names are case sensitive, so Device does not match device. Casting the whole value to a string and searching the text throws away the structure VARIANT keeps.

Not this time. system.billing.usage records all billable usage in the account, and its usage_metadata.job_id is filled for serverless jobs and jobs on job compute (not for jobs on all-purpose compute). system.billing.list_prices is the history of SKU prices, so joining the two turns usage into dollars. Run duration times cluster size ignores the SKU and its price. system.query.history holds queries on SQL warehouses and serverless compute, not a job's whole bill. Event logs describe what a cluster did, not what it cost.

Not this time. Liquid clustering replaces partitioning and ZORDER. Databricks lists tables with poor data skipping or too many partitions, and tables whose query patterns change, as the ones that gain from converting, and CLUSTER BY AUTO picks and changes the clustering keys as the queries change, without rewriting the data each time. Another partition column makes the small-partition problem worse. More ZORDER columns weaken it for every column and must be rerun as data arrives. Fewer files fix the file count but not the layout for the new filters.

Not this time. With deletion vectors, a DELETE marks rows as soft-deleted and leaves them in the Parquet files. REORG TABLE ... APPLY (PURGE) rewrites the files that hold soft-deleted rows, the old files can still hold the data, and VACUUM deletes those once they are older than the retention threshold, 7 days by default. VACUUM straight after the DELETE removes nothing that matters, because the files with the customer's rows are still part of the table. Restoring an old version loses every other change since then.

Not this time. Attribute-based access control uses governed tags and policies to apply permissions by an object's attributes instead of per-object grants, so one column mask policy covers every column with the tag, including columns in tables created later. Per-table masks and per-table views work, but each new table needs its own, and a missed one is a leak. Hashed copies double the storage and leave the originals to be secured anyway.

Not this time. Repair run reruns the unsuccessful tasks and the tasks that depend on them, so tasks 9 to 12 run and tasks 1 to 8 keep their results. Parameters entered in the Repair job run dialog override the existing values. Run now starts every task again. A cloned job loses the run's history and settings, and running the notebook by hand skips tasks 10 to 12 and leaves no record in the job.

Not this time. Privileges granted on a catalog apply to its current and future objects, so SELECT on the catalog covers tables added later. Reading a table also needs USE CATALOG on the parent catalog and USE SCHEMA on the parent schema, which can be granted on the catalog too. SELECT on its own is not enough without them. Grants table by table need upkeep, and ownership hands over the right to manage and drop everything in the catalog.