Databricks Databricks-Certified-Professional-Data-Engineer Exam Dumps

All records from an Apache Kafka producer are being ingested into a single Delta Lake table with the following schema: key BINARY, value BINARY, topic STRING, partition LONG, offset LONG, timestamp LONG There are 5 unique topics being ingested. Only the "registration" topic contains Personal Identifiable Information (PII). The company wishes to restrict access to PII. The company also wishes to only retain records containing PII in this table for 14 days after initial ingestion. However, for non-PII information, it would like to retain these records indefinitely. Which of the following solutions meets the requirements? 

A. All data should be deleted biweekly; Delta Lake's time travel functionality should be leveraged to maintain a history of non-PII information. 
B. Data should be partitioned by the registration field, allowing ACLs and delete statements to be set for the PII directory. 
C. Because the value field is stored as binary data, this information is not considered PII and no special precautions should be taken. 
D. Separate object storage containers should be specified based on the partition field, allowing isolation at the storage level. 
E. Data should be partitioned by the topic field, allowing ACLs and delete statements to leverage partition boundaries. 

Answer: B

Each configuration below is identical to the extent that each cluster has 400 GB total of RAM, 160 total cores and only one Executor per VM. Given a job with at least one wide transformation, which of the following cluster configurations will result in maximum performance? 

A. • Total VMs; 1 • 400 GB per Executor • 160 Cores / Executor 
B. • Total VMs: 8 • 50 GB per Executor • 20 Cores / Executor 
C. • Total VMs: 4 • 100 GB per Executor • 40 Cores/Executor 
D. • Total VMs:2 • 200 GB per Executor • 80 Cores / Executor 

Answer: B

A new data engineer notices that a critical field was omitted from an application that writes its Kafka source to Delta Lake. This happened even though the critical field was in the Kafka source. That field was further missing from data written to dependent, long-term storage. The retention threshold on the Kafka service is seven days. The pipeline has been in production for three months. Which describes how Delta Lake can help to avoid data loss of this nature in the future?

A. The Delta log and Structured Streaming checkpoints record the full history of the Kafka  producer. 
B. Delta Lake schema evolution can retroactively calculate the correct value for newly added fields, as long as the data was in the original source. 
C. Delta Lake automatically checks that all fields present in the source data are included in the ingestion layer. 
D. Data can never be permanently dropped or deleted from Delta Lake, so data loss is not possible under any circumstance. 
E. Ingestine all raw data and metadata from Kafka to a bronze Delta table creates a permanent, replayable history of the data state. 

Answer: E

Which statement describes Delta Lake Auto Compaction?

 A. An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 1 GB. 
B. Before a Jobs cluster terminates, optimize is executed on all tables modified during the most recent job. 
C. Optimized writes use logical partitions instead of directory partitions; because partition boundaries are only represented in metadata, fewer small files are written. 
D. Data is queued in a messaging bus instead of committing data directly to memory; all data is committed from the messaging bus in one batch once the job is complete. 
E. An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an optimize job is executed toward a default of 128 MB. 

Answer: E

The view updates represents an incremental batch of all newly ingested data to be inserted or updated in the customers table. The following logic is used to process these records. MERGE INTO customers USING ( SELECT updates.customer_id as merge_ey, updates .* FROM updates UNION ALL SELECT NULL as merge_key, updates .* FROM updates JOIN customers ON updates.customer_id = customers.customer_id WHERE customers.current = true AND updates.address <> customers.address ) staged_updates ON customers.customer_id = mergekey WHEN MATCHED AND customers. current = true AND customers.address <> staged_updates.address THEN UPDATE SET current = false, end_date = staged_updates.effective_date WHEN NOT MATCHED THEN INSERT (customer_id, address, current, effective_date, end_date) VALUES (staged_updates.customer_id, staged_updates.address, true, staged_updates.effective_date, null) Which statement describes this implementation? 

A. The customers table is implemented as a Type 2 table; old values are overwritten and new customers are appended. 
B. The customers table is implemented as a Type 1 table; old values are overwritten by new values and no history is maintained. 
C. The customers table is implemented as a Type 2 table; old values are maintained but marked as no longer current and new values are inserted. 
D. The customers table is implemented as a Type 0 table; all writes are append only with no changes to existing values. 

Answer: C

An external object storage container has been mounted to the location /mnt/finance_eda_bucket. The following logic was executed to create a database for the finance team: After the database was successfully created and permissions configured, a member of the finance team runs the following code: If all users on the finance team are members of the finance group, which statement describes how the tx_sales table will be created?

A. A logical table will persist the query plan to the Hive Metastore in the Databricks control plane. 
B. An external table will be created in the storage container mounted to /mnt/finance eda bucket. 
C. A logical table will persist the physical plan to the Hive Metastore in the Databricks control plane. 
D. An managed table will be created in the storage container mounted to /mnt/finance eda bucket.
 E. A managed table will be created in the DBFS root storage container. 

Answer: A

A small company based in the United States has recently contracted a consulting firm in India to implement several new data engineering pipelines to power artificial intelligence applications. All the company's data is stored in regional cloud storage in the United States. The workspace administrator at the company is uncertain about where the Databricks workspace used by the contractors should be deployed. Assuming that all data governance considerations are accounted for, which statement accurately informs this decision? 

A. Databricks runs HDFS on cloud volume storage; as such, cloud virtual machines must be deployed in the region where the data is stored. 
B. Databricks workspaces do not rely on any regional infrastructure; as such, the decision should be made based upon what is most convenient for the workspace administrator. 
C. Cross-region reads and writes can incur significant costs and latency; whenever possible, compute should be deployed in the same region the data is stored. 
D. Databricks leverages user workstations as the driver during interactive development; as such, users should always use a workspace deployed in a region they are physically near. 
E. Databricks notebooks send all executable code from the user's browser to virtual machines over the open internet; whenever possible, choosing a workspace region near the end users is the most secure. 

Answer: C

Where in the Spark UI can one diagnose a performance problem induced by not leveraging predicate push-down? 

A. In the Executor's log file, by gripping for "predicate push-down" 
B. In the Stage's Detail screen, in the Completed Stages table, by noting the size of data read from the Input column 
C. In the Storage Detail screen, by noting which RDDs are not stored on disk 
D. In the Delta Lake transaction log. by noting the column statistics 
E. In the Query Detail screen, by interpreting the Physical Plan 

Answer: E

Which of the following is true of Delta Lake and the Lakehouse? 

A. Because Parquet compresses data row by row. strings will only be compressed when a character is repeated multiple times. 
B. Delta Lake automatically collects statistics on the first 32 columns of each table which are leveraged in data skipping based on query filters. 
C. Views in the Lakehouse maintain a valid cache of the most recent versions of source tables at all times.
 D. Primary and foreign key constraints can be leveraged to ensure duplicate values are never entered into a dimension table. E. Z-order can only be applied to numeric values stored in Delta Lake tables 

Answer: B

Which is a key benefit of an end-to-end test? 

A. It closely simulates real world usage of your application. 
B. It pinpoint errors in the building blocks of your application. 
C. It provides testing coverage for all code paths and branches. 
D. It makes it easier to automate your test suite 

Answer: A

Although the Databricks Utilities Secrets module provides tools to store sensitive credentials and avoid accidentally displaying them in plain text users should still be careful with which credentials are stored here and which users have access to using these secrets. Which statement describes a limitation of Databricks Secrets? 

A. Because the SHA256 hash is used to obfuscate stored secrets, reversing this hash will display the value in plain text. 
B. Account administrators can see all secrets in plain text by logging on to the Databricks Accounts console. 
C. Secrets are stored in an administrators-only table within the Hive Metastore; database administrators have permission to query this table by default. 
D. Iterating through a stored secret and printing each character will display secret contents in plain text. 
E. The Databricks REST API can be used to list secrets in plain text if the personal access token has proper credentials. 

Answer: E

A Databricks SQL dashboard has been configured to monitor the total number of records present in a collection of Delta Lake tables using the following query pattern: SELECT COUNT (*) FROM table - Which of the following describes how results are generated each time the dashboard is updated? 

A. The total count of rows is calculated by scanning all data files 
B. The total count of rows will be returned from cached results unless REFRESH is run 
C. The total count of records is calculated from the Delta transaction logs 
D. The total count of records is calculated from the parquet file metadata E. The total count of records is calculated from the Hive metastore 

Answer: C

Which distribution does Databricks support for installing custom Python code packages? 

A. sbt 
B. CRAN 
C. CRAM 
D. nom 
E. Wheels 
F. jars 

Answer: D

A data architect has heard about lake's built-in versioning and time travel capabilities. For auditing purposes they have a requirement to maintain a full of all valid street addresses as they appear in the customers table. The architect is interested in implementing a Type 1 table, overwriting existing records with new values and relying on Delta Lake time travel to support long-term auditing. A data engineer on the project feels that a Type 2 table will provide better performance and scalability Which piece of information is critical to this decision? 

A. Delta Lake time travel does not scale well in cost or latency to provide a long-term versioning solution. 
B. Delta Lake time travel cannot be used to query previous versions of these tables because Type 1 changes modify data files in place. 
C. Shallow clones can be combined with Type 1 tables to accelerate historic queries for long-term versioning. 
D. Data corruption can occur if a query fails in a partially completed state because Type 2 tables requires Setting multiple fields in a single update.

Answer: A