Free CERTIFIED-DATA-ENGINEER-PROFESSIONAL – Databricks CERTIFIED-DATA-ENGINEER-PROFESSIONAL Practice Test Question

Select an option, then click Submit answer.

• ○ Create a separate history table for each pk_id resolve the current state of the table by running a union all filtering the history tables for the most recent state.
• ○ Use MERGE INTO to insert, update, or delete the most recent entry for each pk_id into a bronze table, then propagate all changes throughout the system.
• ○ Iterate through an ordered set of changes to the table, applying each in turn; rely on Delta Lake's versioning ability to create an audit log.
• ○ Use Delta Lake's change data feed to automatically process CDC data from an external system, propagating all changes to all dependent tables in the Lakehouse.
• ○ Ingest all log information into a bronze table; use MERGE INTO to insert, update, or delete the most recent entry for each pk_id into a silver table to recreate the current table state.

Select an option, then click Submit answer.

• ○ Task queueing resulting from improper thread pool assignment.
• ○ Spill resulting from attached volume storage being too small.
• ○ Network latency due to some cluster nodes being in different regions from the source data
• ○ Skew caused by more data being assigned to a subset of spark-partitions.
• ○ Credential validation errors while pulling data from an external system.

Select an option, then click Submit answer.

• ○ Isolating tables in separate databases based on data quality tiers allows for easy permissions management through database ACLs and allows physical separation of default storage locations for managed tables.
• ○ Because databases on Databricks are merely a logical construct, choices around database organization do not impact security or discoverability in the Lakehouse.
• ○ Storing all production tables in a single database provides a unified view of all data assets available throughout the Lakehouse, simplifying discoverability by granting all users view privileges on this database.
• ○ Working in the default Databricks database provides the greatest security when working with managed tables, as these will be created in the DBFS root.
• ○ Because all tables must live in the same storage containers used for the database they're created in, organizations should be prepared to create between dozens and thousands of databases depending on their data isolation requirements.

Select an option, then click Submit answer.

• ○ The customers table is implemented as a Type 3 table; old values are maintained as a new column alongside the current value.
• ○ The customers table is implemented as a Type 2 table; old values are maintained but marked as no longer current and new values are inserted.
• ○ The customers table is implemented as a Type 0 table; all writes are append only with no changes to existing values.
• ○ The customers table is implemented as a Type 1 table; old values are overwritten by new values and no history is maintained.
• ○ The customers table is implemented as a Type 2 table; old values are overwritten and new customers are appended.

Select an option, then click Submit answer.

• ○ Yes; Delta Lake ACID guarantees provide assurance that the DELETE command succeeded fully and permanently purged these records.
• ○ No; the Delta cache may return records from previous versions of the table until the cluster is restarted.
• ○ Yes; the Delta cache immediately updates to reflect the latest data files recorded to disk.
• ○ No; the Delta Lake DELETE command only provides ACID guarantees when combined with the MERGE INTO command.
• ○ No; files containing deleted records may still be accessible with time travel until a VACUUM command is used to remove invalidated data files.

Select an option, then click Submit answer.

• ○ The code executed was PySpark but was executed in a Scala notebook.
• ○ There is no column in the table named heartrateheartrateheartrate
• ○ There is a type error because a column object cannot be multiplied.
• ○ There is a type error because a DataFrame object cannot be multiplied.
• ○ There is a syntax error because the heartrate column is not correctly identified as a column.

Select an option, then click Submit answer.

• ○ Implement the appropriate aggregate logic as a batch read against the daily_store_sales table and overwrite the store_sales_summary table with each Update.
• ○ Implement the appropriate aggregate logic as a batch read against the daily_store_sales table and append new rows nightly to the store_sales_summary table.
• ○ Implement the appropriate aggregate logic as a batch read against the daily_store_sales table and use upsert logic to update results in the store_sales_summary table.
• ○ Implement the appropriate aggregate logic as a Structured Streaming read against the daily_store_sales table and use upsert logic to update results in the store_sales_summary table.
• ○ Use Structured Streaming to subscribe to the change data feed for daily_store_sales and apply changes to the aggregates in the store_sales_summary table with each update.

Select an option, then click Submit answer.

• ○ Specify a new checkpointLocation
• ○ Increase the shuffle partitions to account for additional aggregates
• ○ Run REFRESH TABLE delta.'/item_agg'
• ○ Register the data in the "/item_agg" directory to the Hive metastore
• ○ Remove .option(‘mergeSchema’, ‘true’) from the streaming write

Select an option, then click Submit answer.

• ○ New fields will not be computed for historic records.
• ○ Spark cannot capture the topic and partition fields from a Kafka source.
• ○ New fields cannot be added to a production Delta table.
• ○ Updating the table schema will invalidate the Delta transaction log metadata.
• ○ Updating the table schema requires a default value provided for each field added.

Select an option, then click Submit answer.

• ○ Because all tasks are managed as a dependency graph, no changes will be committed to the Lakehouse until all tasks have successfully been completed.
• ○ Tasks B and C will attempt to run as configured; any changes made in task A will be rolled back due to task failure.
• ○ Unless all tasks complete successfully, no changes will be committed to the Lakehouse; because task A failed, all commits will be rolled back automatically.
• ○ Tasks B and C will be skipped; some logic expressed in task A may have been committed before task failure.
• ○ Tasks B and C will be skipped; task A will not commit any changes because of stage failure.

Select an option, then click Submit answer.

• ○ The Databricks REST API can be used to modify the Spark configuration properties for an interactive cluster without interrupting jobs currently running on the cluster.
• ○ Spark configurations set within a notebook will affect all SparkSessions attached to the same interactive cluster.
• ○ Spark configuration properties can only be set for an interactive cluster by creating a global init script.
• ○ Spark configuration properties set for an interactive cluster with the Clusters UI will impact all notebooks attached to that cluster.
• ○ When the same Spark configuration property is set for an interactive cluster and a notebook attached to that cluster, the notebook setting will always be ignored.

Select an option, then click Submit answer.

• ○ Size on Disk is < Size in Memory
• ○ The RDD Block Name includes the “*” annotation signaling a failure to cache
• ○ Size on Disk is > 0
• ○ The number of Cached Partitions > the number of Spark Partitions
• ○ On Heap Memory Usage is within 75% of Off Heap Memory Usage

Select an option, then click Submit answer.

• ○ The job_id and number of times the job has been run are concatenated and returned.
• ○ The total number of jobs that have been run in the workspace.
• ○ The number of times the job definition has been run in this workspace.
• ○ The job_id is returned in this field.
• ○ The globally unique ID of the newly triggered run.

Select an option, then click Submit answer.

• ○ • Total VMs: 8 • 50 GB per Executor • 20 Cores / Executor
• ○ • Total VMs: 16 • 25 GB per Executor • 10 Cores / Executor
• ○ • Total VMs: 1 • 400 GB per Executor • 160 Cores/Executor
• ○ • Total VMs: 4 • 100 GB per Executor • 40 Cores / Executor
• ○ • Total VMs: 2 • 200 GB per Executor • 80 Cores / Executor

Select an option, then click Submit answer.

• ○ Populate the dashboard by configuring a nightly batch job to save the required values as a table overwritten with each update.
• ○ Use Structured Streaming to configure a live dashboard against the products_per_order table within a Databricks notebook.
• ○ Define a view against the products_per_order table and define the dashboard against this view.
• ○ Use the Delta Cache to persist the products_per_order table in memory to quickly update the dashboard with each query.

Select an option, then click Submit answer.

• ○ to_interval("event_time", "5 minutes").alias("time")
• ○ window("event_time", "5 minutes").alias("time")
• ○ "event_time"
• ○ lag("event_time", "10 minutes").alias("time")

Select an option, then click Submit answer.

• ○ Before a Jobs cluster terminates, OPTIMIZE is executed on all tables modified during the most recent job.
• ○ An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an OPTIMIZE job is executed toward a default of 1 GB.
• ○ A shuffle occurs prior to writing to try to group similar data together resulting in fewer files instead of each executor writing multiple files based on directory partitions.
• ○ Optimized writes use logical partitions instead of directory partitions; because partition boundaries are only represented in metadata, fewer small files are written.

Select an option, then click Submit answer.

• ○ The five Minute Load Average remains consistent/flat
• ○ CPU Utilization is around 75%
• ○ Network I/O never spikes
• ○ Total Disk Space remains constant

Select an option, then click Submit answer.

• ○ It validates an application use case.
• ○ It validates behavior of individual elements of an application,
• ○ It requires an automated testing framework.
• ○ It validates interactions between subsystems of your application.

Select an option, then click Submit answer.

• ○ The silver_customer_sales table will be overwritten by aggregated values calculated from all records in the gold_customer_lifetime_sales_summary table as a batch job.
• ○ A batch job will update the gold_customer_lifetime_sales_summary table, replacing only those rows that have different values than the current version of the table, using customer_id as the primary key.
• ○ The gold_customer_lifetime_sales_summary table will be overwritten by aggregated values calculated from all records in the silver_customer_sales table as a batch job.
• ○ An incremental job will detect if new rows have been written to the silver_customer_sales table; if new rows are detected, all aggregates will be recalculated and used to overwrite the gold_customer_lifetime_sales_summary table.

Select an option, then click Submit answer.

• ○ Send all users notice that the schema for the table will be changing; include in the communication the logic necessary to revert the new table schema to match historic queries.
• ○ Configure a new table with all the requisite fields and new names and use this as the source for the customer-facing application; create a view that maintains the original data schema and table name by aliasing select fields from the new table.
• ○ Create a new table with the required schema and new fields and use Delta Lake's deep clone functionality to sync up changes committed to one table to the corresponding table.
• ○ Replace the current table definition with a logical view defined with the query logic currently writing the aggregate table; create a new table to power the customer-facing application.

Select an option, then click Submit answer.

• ○ In the Executor’s log file, by grepping for "predicate push-down"
• ○ In the Stage’s Detail screen, in the Completed Stages table, by noting the size of data read from the Input column
• ○ In the Query Detail screen, by interpreting the Physical Plan
• ○ In the Delta Lake transaction log. by noting the column statistics

Select an option, then click Submit answer.

• ○ It makes it easier to automate your test suite
• ○ It pinpoints errors in the building blocks of your application
• ○ It provides testing coverage for all code paths and branches
• ○ It closely simulates real world usage of your application

Select an option, then click Submit answer.

• ○ CAN VIEW
• ○ CAN RESTART
• ○ CAN ATTACH TO
• ○ CAN MANAGE

Select an option, then click Submit answer.

• ○ Use Hive-style partitioning, as it provides efficient data skipping and is easy to change partition columns at any time.
• ○ Combine partitioning and Z-ordering to maximize flexibility and minimize maintenance as query patterns change.
• ○ Enable liquid clustering, as it efficiently handles data skew, allows clustering keys to be changed without rewriting existing data, and adapts to evolving query patterns.
• ○ Apply Z-ordering, since it allows flexible reorganization of data layout without rewriting existing and adapts easily to new filter columns.

Select an option, then click Submit answer.

• ○ Define a streaming table to ingest and store the raw telemetry data, and create a streaming table to compute the daily aggregated distance and fuel efficiency per truck_id reporting. Create a materialized view to compute the latest location, speed, and fuel_level per truck_id for real-time monitoring.
• ○ Define a streaming table to ingest and store the raw telemetry data, and create a materialized view to compute the latest location, speed, and fuel_level per truck_id for real-time monitoring. Create another materialized view to compute the daily aggregated distance and fuel efficiency per truck_id for reporting.
• ○ Define a streaming table to ingest and store the raw telemetry data, and create a streaming table to incrementally compute the latest location, speed, and fuel_level per truck_id for real-time monitoring. Create a materialized view to compute the daily aggregated distance and fuel efficiency per truck_id for reporting.
• ○ Define a materialized view to ingest and store the raw telemetry data, and create a streaming table to compute the latest location, speed, and fuel_level per truck_id for real-time monitoring. Create another materialized view to compute the daily aggregated distance and fuel efficiency per truck_id for reporting.