Free Certified Data Engineer Professional Exam braindumps download (Certified Data Engineer Professional exam dumps Free)

QUESTION: 145

A task orchestrator has been configured to run two hourly tasks. First, an outside system writes Parquet data to a directory mounted at /mnt/raw_orders/. After this data is written, a Databricks job containing the following code is executed:

Assume that the fields customer_id and order_id serve as a composite key to uniquely identify each order, and that the time field indicates when the record was queued in the source system.

If the upstream system is known to occasionally enqueue duplicate entries for a single order hours apart, which statement is correct?

Duplicate records enqueued more than 2 hours apart may be retained and the orders table may contain duplicate records with the same customer_id and order_id.
All records will be held in the state store for 2 hours before being deduplicated and committed to the orders table.
The orders table will contain only the most recent 2 hours of records and no duplicates will be present.
The orders table will not contain duplicates, but records arriving more than 2 hours late will be ignored and missing from the table.

Answer(s): A

Show Answer Next Question

QUESTION: 146

A data engineer is configuring a pipeline that will potentially see late-arriving, duplicate records.

In addition to de-duplicating records within the batch, which of the following approaches allows the data engineer to deduplicate data against previously processed records as it is inserted into a Delta table?

Rely on Delta Lake schema enforcement to prevent duplicate records.
VACUUM the Delta table after each batch completes.
Perform an insert-only merge with a matching condition on a unique key.
Perform a full outer join on a unique key and overwrite existing data.

Answer(s): C

Reference:

Show Answer Next Question

QUESTION: 147

A junior data engineer seeks to leverage Delta Lake's Change Data Feed functionality to create a Type 1 table representing all of the values that have ever been valid for all rows in a bronze table created with the property delta.enableChangeDataFeed = true. They plan to execute the following code as a daily job:

Which statement describes the execution and results of running the above query multiple times?

Each time the job is executed, newly updated records will be merged into the target table, overwriting previous values with the same primary keys.
Each time the job is executed, the entire available history of inserted or updated records will be appended to the target table, resulting in many duplicate entries.
Each time the job is executed, only those records that have been inserted or updated since the last execution will be appended to the target table, giving the desired result.
Each time the job is executed, the differences between the original and current versions are calculated; this may result in duplicate entries for some records.

Answer(s): B

Reference:

Show Answer Next Question

QUESTION: 148

A DLT pipeline includes the following streaming tables:

raw_iot ingests raw device measurement data from a heart rate tracking device.

bpm_stats incrementally computes user statistics based on BPM measurements from raw_iot.

How can the data engineer configure this pipeline to be able to retain manually deleted or updated records in the raw_iot table, while recomputing the downstream table bpm_stats table when a pipeline update is run?

Set the pipelines.reset.allowed property to false on raw_iot
Set the skipChangeCommits flag to true on raw_iot
Set the pipelines.reset.allowed property to false on bpm_stats
Set the skipChangeCommits flag to true on bpm_stats

Answer(s): A

Show Answer Next Question

QUESTION: 149

A data pipeline uses Structured Streaming to ingest data from Apache Kafka to Delta Lake. Data is being stored in a bronze table, and includes the Kafka-generated timestamp, key, and value. Three months after the pipeline is deployed, the data engineering team has noticed some latency issues during certain times of the day.

A senior data engineer updates the Delta Table's schema and ingestion logic to include the current timestamp (as recorded by Apache Spark) as well as the Kafka topic and partition. The team plans to use these additional metadata fields to diagnose the transient processing delays.

Which limitation will the team face while diagnosing this problem?

New fields will not be computed for historic records.
Spark cannot capture the topic and partition fields from a Kafka source.
Updating the table schema requires a default value provided for each field added.
Updating the table schema will invalidate the Delta transaction log metadata.

Answer(s): A

Reference:

Show Answer Next Question

QUESTION: 150

A nightly job ingests data into a Delta Lake table using the following code:

The next step in the pipeline requires a function that returns an object that can be used to manipulate new records that have not yet been processed to the next table in the pipeline.

Which code snippet completes this function definition?

def new_records():

return spark.readStream.table("bronze")
return spark.read.option("readChangeFeed", "true").table ("bronze")
C.
D.

Answer(s): A

Show Answer Next Question

QUESTION: 151

A junior data engineer is working to implement logic for a Lakehouse table named silver_device_recordings. The source data contains 100 unique fields in a highly nested JSON structure.

The silver_device_recordings table will be used downstream to power several production monitoring dashboards and a production model. At present, 45 of the 100 fields are being used in at least one of these applications.

The data engineer is trying to determine the best approach for dealing with schema declaration given the highly-nested structure of the data and the numerous fields.

Which of the following accurately presents information about Delta Lake and Databricks that may impact their decision-making process?

The Tungsten encoding used by Databricks is optimized for storing string data; newly-added native support for querying JSON strings means that string types are always most efficient.
Because Delta Lake uses Parquet for data storage, data types can be easily evolved by just modifying file footer information in place.
Schema inference and evolution on Databricks ensure that inferred types will always accurately match the data types used by downstream systems.
Because Databricks will infer schema using types that allow all observed data to be processed, setting types manually provides greater assurance of data quality enforcement.

Answer(s): D

Reference:

Show Answer Next Question

QUESTION: 152

The data engineering team maintains the following code:

Assuming that this code produces logically correct results and the data in the source tables has been de- duplicated and validated, which statement describes what will occur when this code is executed?

A batch job will update the enriched_itemized_orders_by_account table, replacing only those rows that have different values than the current version of the table, using accountID as the primary key.
The enriched_itemized_orders_by_account table will be overwritten using the current valid version of data in each of the three tables referenced in the join logic.
No computation will occur until enriched_itemized_orders_by_account is queried; upon query materialization, results will be calculated using the current valid version of data in each of the three tables referenced in the join logic.
An incremental job will detect if new rows have been written to any of the source tables; if new rows are detected, all results will be recalculated and used to overwrite the enriched_itemized_orders_by_account table.

Answer(s): B

Show Answer Next Question

Free Certified Data Engineer Professional Exam Braindumps Certified Data Engineer Professional Questions 145–152 (page: 20)

QUESTION: 145

QUESTION: 146

Reference:

QUESTION: 147

Reference:

QUESTION: 148

QUESTION: 149

Reference:

QUESTION: 150

QUESTION: 151

Reference:

QUESTION: 152

Free Certified Data Engineer Professional Exam Braindumps
Certified Data Engineer Professional Questions 145–152 (page: 20)