最新的高品質 Certified-Data-Engineer-Professional 考試練習題庫

Which statement describes Delta Lake Auto Compaction?

• A. Optimized writes use logical partitions instead of directory partitions; because partition boundaries are only represented in metadata, fewer small files are written.
• B. Data is queued in a messaging bus instead of committing data directly to memory; all data is committed from the messaging bus in one batch once the job is complete.
• C. An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 128 MB.
• D. An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 1 GB.
• E. Before a Jobs cluster terminates, optimize is executed on all tables modified during the most recent job.

A data engineer is optimizing a MERGE operation on an 800GB UC-managed table that experiences frequent updates and deletions. Which two actions should the engineer prioritize to improve MERGE performance? (Choose two.)

• A. Apply liquid clustering using the merge join keys.
• B. Partition the table by date.
• C. Use ZORDER on high-cardinality columns.
• D. Overwrite the table instead of Merge.
• E. Enable deletion vectors on the table if not already enabled.

A data engineering team is migrating off its legacy Hadoop platform. As part of the process, they are evaluating storage formats for performance comparison. The legacy platform uses ORC and RCFile formats. After converting a subset of data to Delta Lake, they noticed significantly better query performance. Upon investigation, they discovered that queries reading from Delta tables leveraged a Shuffle Hash Join, whereas queries on legacy formats used Sort Merge Joins. The queries reading Delta Lake data also scanned less data. Which reason could be attributed to the difference in query performance?

• A. Shuffle Hash Joins are always more efficient than Sort Merge Joins.
• B. Delta Lake enables data skipping and file pruning using a vectorized Parquet reader.
• C. The queries against the ORC tables leveraged the dynamic data skipping optimization but not the dynamic file pruning optimization.
• D. The queries against the Delta Lake tables were able to leverage the dynamic file pruning optimization.

A data engineer is analyzing a large, partitioned retail dataset in Databricks, where each row represents a sale made by a salesperson. The dataset contains millions of records with the following schema:
sales_df: [salesperson_id: string, region: string, sale_amount: double, sale_date: date] The data engineer needs to generate a DataFrame that ranks salespeople within each region based on their total cumulative sales, with the highest seller ranked as 1. If multiple salespeople have the same total sales, they should share the same rank.
The data engineer wants to implement this logic using a PySpark window function and the dense_rank () function.
Which code snippet will perform this ranking?

• A.
  
• B.
  
• C.
  
• D.
  

A data engineer is reviewing the PySpark code to copy a part of the production dataset to the sandbox environment, and needs to be sure that no PII(Personally Identifiable Information) data is being copied. After checking the sales table, the data engineer notices that it has user emails as the only PII data included as well as being the only column to identify the user.
from pyspark.sql import functions as F

Which anonymised code should be used to achieve the required outcome?

• A. df.withColumn ("hashed_email", sha2 ("user_email"))
• B. df.withColumn ("user_email", F.sha2 ("user_email"))
• C. df.withColumn ("user_email", F.regexp_replace ("user_eamail", "@*", "@anonymized.com"))
• D. df.withColumn ("user_emai", F.expr("uuid()"))