最新的高品質 Databricks-Certified-Data-Engineer-Professional 考試練習題庫

A Delta Lake table in the Lakehouse named customer_parsams is used in churn prediction by the machine learning team. The table contains information about customers derived from a number of upstream sources. Currently, the data engineering team populates this table nightly by overwriting the table with the current valid values derived from upstream data sources.
Immediately after each update succeeds, the data engineer team would like to determine the difference between the new version and the previous of the table. Given the current implementation, which method can be used?

• A. Parse the Spark event logs to identify those rows that were updated, inserted, or deleted.
• B. Execute DESCRIBE HISTORY customer_churn_params to obtain the full operation metrics for the update, including a log of all records that have been added or modified.
• C. Parse the Delta Lake transaction log to identify all newly written data files.
• D. Execute a query to calculate the difference between the new version and the previous version using Delta Lake's built-in versioning and time travel functionality.

A data engineering team has a time-consuming data ingestion job with three data sources. Each notebook takes about one hour to load new data. One day, the job fails because a notebook update introduced a new required configuration parameter. The team must quickly fix the issue and load the latest data from the failing source. Which action should the team take?

• A. Share the analysis with the failing notebook owner so that they can fix it quickly.
• B. Update the task by adding the missing task parameter, and manually run the job.
• C. Repair the run with the new parameter, and update the task by adding the missing task parameter.
• D. Repair the run with the new parameter.

Where in the Spark UI can one diagnose a performance problem induced by not leveraging predicate push-down?

• A. In the Delta Lake transaction log. by noting the column statistics
• B. In the Query Detail screen, by interpreting the Physical Plan
• C. In the Storage Detail screen, by noting which RDDs are not stored on disk
• D. In the Executor's log file, by gripping for "predicate push-down"
• E. In the Stage's Detail screen, in the Completed Stages table, by noting the size of data read from the Input column

A data engineer is analyzing a large, partitioned retail dataset in Databricks, where each row represents a sale made by a salesperson. The dataset contains millions of records with the following schema:
sales_df: [salesperson_id: string, region: string, sale_amount: double, sale_date: date] The data engineer needs to generate a DataFrame that ranks salespeople within each region based on their total cumulative sales, with the highest seller ranked as 1. If multiple salespeople have the same total sales, they should share the same rank.
The data engineer wants to implement this logic using a PySpark window function and the dense_rank () function.
Which code snippet will perform this ranking?

• A.
  
• B.
  
• C.
  
• D.
  

Assuming that the Databricks CLI has been installed and configured correctly, which Databricks CLI command can be used to upload a custom Python Wheel to object storage mounted with the DBFS for use with a production job?

• A. jobs
• B. configure
• C. libraries
• D. fs
• E. workspace