Free Certified Data Engineer Professional Exam braindumps download (Certified Data Engineer Professional exam dumps Free)

QUESTION: 105

The data science team has created and logged a production model using MLflow. The model accepts a list of column names and returns a new column of type DOUBLE.

The following code correctly imports the production model, loads the customers table containing the customer_id key column into a DataFrame, and defines the feature columns needed for the model.

Which code block will output a DataFrame with the schema "customer_id LONG, predictions DOUBLE"?

df.map(lambda x:model(x[columns])).select("customer_id, predictions")
df.select("customer_id", model(*columns).alias("predictions"))
model.predict(df, columns)
df.select("customer_id", pandas_udf(model, columns).alias("predictions"))
df.apply(model, columns).select("customer_id, predictions")

Answer(s): B

Show Answer Next Question

QUESTION: 106

A nightly batch job is configured to ingest all data files from a cloud object storage container where records are stored in a nested directory structure YYYY/MM/DD. The data for each date represents all records that were processed by the source system on that date, noting that some records may be delayed as they await moderator approval. Each entry represents a user review of a product and has the following schema:

user_id STRING, review_id BIGINT, product_id BIGINT, review_timestamp TIMESTAMP, review_text STRING

The ingestion job is configured to append all data for the previous date to a target table reviews_raw with an identical schema to the source system. The next step in the pipeline is a batch write to propagate all new records inserted into reviews_raw to a table where data is fully deduplicated, validated, and enriched.

Which solution minimizes the compute costs to propagate this batch of data?

Perform a batch read on the reviews_raw table and perform an insert-only merge using the natural composite key user_id, review_id, product_id, review_timestamp.
Configure a Structured Streaming read against the reviews_raw table using the trigger once execution mode to process new records as a batch job.
Use Delta Lake version history to get the difference between the latest version of reviews_raw and one version prior, then write these records to the next table.
Filter all records in the reviews_raw table based on the review_timestamp; batch append those records produced in the last 48 hours.
Reprocess all records in reviews_raw and overwrite the next table in the pipeline.

Answer(s): A

Show Answer Next Question

QUESTION: 107

Which statement describes Delta Lake optimized writes?

Before a Jobs cluster terminates, OPTIMIZE is executed on all tables modified during the most recent job.
An asynchronous job runs after the write completes to detect if files could be further compacted; if yes, an OPTIMIZE job is executed toward a default of 1 G
Data is queued in a messaging bus instead of committing data directly to memory; all data is committed from the messaging bus in one batch once the job is complete.
Optimized writes use logical partitions instead of directory partitions; because partition boundaries are only represented in metadata, fewer small files are written.
A shuffle occurs prior to writing to try to group similar data together resulting in fewer files instead of each executor writing multiple files based on directory partitions.

Answer(s): E

Show Answer Next Question

QUESTION: 108

Which statement describes the default execution mode for Databricks Auto Loader?

Cloud vendor-specific queue storage and notification services are configured to track newly arriving files; the target table is materialized by directly querying all valid files in the source directory.
New files are identified by listing the input directory; the target table is materialized by directly querying all valid files in the source directory.
Webhooks trigger a Databricks job to run anytime new data arrives in a source directory; new data are automatically merged into target tables using rules inferred from the data.
New files are identified by listing the input directory; new files are incrementally and idempotently loaded into the target Delta Lake table.
Cloud vendor-specific queue storage and notification services are configured to track newly arriving files; new files are incrementally and idempotently loaded into the target Delta Lake table.

Answer(s): D

Show Answer Next Question

QUESTION: 109

A Delta Lake table representing metadata about content posts from users has the following schema:

user_id LONG, post_text STRING, post_id STRING, longitude FLOAT, latitude FLOAT, post_time TIMESTAMP, date DATE

Based on the above schema, which column is a good candidate for partitioning the Delta Table?

post_time
latitude
post_id
user_id
date

Answer(s): E

Show Answer Next Question

QUESTION: 110

A large company seeks to implement a near real-time solution involving hundreds of pipelines with parallel updates of many tables with extremely high volume and high velocity data.

Which of the following solutions would you implement to achieve this requirement?

Use Databricks High Concurrency clusters, which leverage optimized cloud storage connections to maximize data throughput.
Partition ingestion tables by a small time duration to allow for many data files to be written in parallel.
Configure Databricks to save all data to attached SSD volumes instead of object storage, increasing file I/O significantly.
Isolate Delta Lake tables in their own storage containers to avoid API limits imposed by cloud vendors.
Store all tables in a single database to ensure that the Databricks Catalyst Metastore can load balance overall throughput.

Answer(s): A

Show Answer Next Question

QUESTION: 111

Which describes a method of installing a Python package scoped at the notebook level to all nodes in the currently active cluster?

Run source env/bin/activate in a notebook setup script
Use b in a notebook cell
Use %pip install in a notebook cell
Use %sh pip install in a notebook cell
Install libraries from PyPI using the cluster UI

Answer(s): C

Show Answer Next Question

QUESTION: 112

Each configuration below is identical to the extent that each cluster has 400 GB total of RAM, 160 total cores and only one Executor per VM.

Given an extremely long-running job for which completion must be guaranteed, which cluster configuration will be able to guarantee completion of the job in light of one or more VM failures?

· Total VMs: 8
· 50 GB per Executor
· 20 Cores / Executor
· Total VMs: 16
· 25 GB per Executor
· 10 Cores / Executor
· Total VMs: 1
· 400 GB per Executor
· 160 Cores/Executor
· Total VMs: 4
· 100 GB per Executor
· 40 Cores / Executor
· Total VMs: 2
· 200 GB per Executor
· 80 Cores / Executor

Answer(s): B

Show Answer Next Question

Free Certified Data Engineer Professional Exam Braindumps Certified Data Engineer Professional Questions 105–112 (page: 15)

QUESTION: 105

QUESTION: 106

QUESTION: 107

QUESTION: 108

QUESTION: 109

QUESTION: 110

QUESTION: 111

QUESTION: 112

Free Certified Data Engineer Professional Exam Braindumps
Certified Data Engineer Professional Questions 105–112 (page: 15)