A data engineer is setting up a new Databricks pipeline that ingests clickstream events from Kafka and daily product catalogs from cloud object storage. To ensure auditability and easy reprocessing, the engineer wants to land all source data. Later stages will handle cleaning, deduplication, and business modeling before the data is used in dashboards.Which approach aligns to Medallion Architecture principles?
Answer(s): C
The Medallion Architecture recommends landing all raw source data in the Bronze layer in an append-only, minimally transformed form to preserve source fidelity and support auditability and reprocessing, with cleaning,deduplication, and business modeling performed in downstream Silver and Gold layers.
A data engineer is building a PySpark ingestion pipeline in a local IDE and must execute heavy DataFrame transformations on a remote Databricks cluster for scale while stepping through business logic locally. The workspace uses Unity Catalog and a cluster running Databricks Runtime 15.4 LTS. The engineer also plans to register UDFs.What should the data engineer do to avoid runtime issues during development with Databricks Connect?
Answer(s): A
Databricks Connect requires tight compatibility between the local development environment and the remote cluster. When registering UDFs, the local Python minor version must match the cluster's Python minor version, and the Databricks Connect package must align with the Databricks Runtime version to prevent serialization and runtime incompatibilities.
A data engineer working in Databricks needs to control BI query latency and cost for a sales analytics domain. Silver tables already contain validated, normalized sales and product data. The team wants dashboard-friendly query performance and to align datasets with business concepts (e.g., weekly_sales and account_performance) without overloading analysts with complex joins.Which design approach should a data engineer choose for the Gold layer in a medallion architecture?
Answer(s): B
The Gold layer is designed for business-level consumption, so creating denormalized, domain-aligned tables and precomputed aggregates reduces query complexity and latency while optimizing cost for BI workloads and dashboards.
A developer has configured Databricks Connect and starts running PySpark code locally. They notice that Data Frame operations are executed on the remote cluster, but actions like .show() return results directly in their local console.What does this behaviour illustrate about how Databricks Connect is used?
Databricks Connect forwards DataFrame transformations and actions from the local environment to the remote cluster for execution, while returning results such as .show() back to the local client, demonstrating a proxy- based interaction model.
A data engineer must support self-serve BI dashboards for hundreds of business users who run ad hoc, high- concurrency SQL queries throughout the day against governed Unity Catalog tables. The team needs a near- instant start, autoscaling without manual tuning, and the best performance for SQL while minimizing operational overhead.Which Databricks compute should the data engineer use?
Answer(s): D
Serverless SQL Warehouses provide instant startup, automatic scaling, and optimized performance with Photon for high-concurrency BI workloads, while minimizing operational overhead and management effort.
A data engineer has realized that the data files associated with a Delta table are incredibly small. They want to compact the small files to form larger files to improve performance.Which of the following keywords can be used to compact the small files?
A data engineer has a Python notebook in Databricks, but they need to use SQL to accomplish a specific task within a cell. They still want all of the other cells to use Python without making any changes to those cells.Which of the following describes how the data engineer can use SQL within a cell of their Python notebook?
Which of the following SQL keywords can be used to convert a table from a long format to a wide format?
Post your Comments and Discuss Databricks Certified Data Engineer Associate exam prep with other Community members:
Our website is free, but we have to fight against AI bots and content theft. We're sorry for the inconvenience caused by these security measures. You can access the rest of the Certified Data Engineer Associate content, but please register or login to continue.