A data engineer wants to create a new table containing the names of customers that live in France.They have written the following command:A senior data engineer mentions that it is organization policy to include a table property indicating that the new table includes personally identifiable information (PII).Which line of code fills in the above blank to successfully complete the task?
Answer(s): D
A data engineer is attempting to drop a Spark SQL table my_table. The data engineer wants to delete all table metadata and data.They run the following command:DROP TABLE IF EXISTS my_tableWhile the object no longer appears when they run SHOW TABLES, the data files still exist.Which of the following describes why the data files still exist and the metadata files were deleted?
Answer(s): C
A data engineer wants to create a data entity from a couple of tables. The data entity must be used by other data engineers in other sessions. It also must be saved to a physical location.Which of the following data entities should the data engineer create?
Answer(s): E
A data engineer needs to compute metrics such as:rolling 7-day salespercent of total revenuetop-N most profitable categoriesWhich two approaches are appropriate for implementing these metrics with PySpark DataFrames? (Choose two.)
Answer(s): C,E
Window functions are the correct approach for rolling calculations and ranking tasks such as rolling 7-day sales and top-N categories. Joins and groupBy are appropriate for percent-of-total calculations by first computing total revenue at the needed level and then comparing each subgroup against that total.
A data engineering team is analyzing customer transactions using a PySpark DataFrame named transactions with columns customer_id, amount, and category.The team needs to compute, for each customer, the total transaction amount, the average transaction amount, and the maximum amount - all in a single operation for efficiency.Which code fragment should be used to achieve this in PySpark?
Answer(s): A
Grouping by customer_id and applying multiple aggregate functions inside a single agg() call is the correct PySpark pattern for computing total, average, and maximum values per customer in one operation.
Which of the following commands can be used to write data into a Delta table while avoiding the writing of duplicate records?
A data engineer runs a statement every day to copy the previous day's sales into the table transactions. Each day's sales are in their own file in the location "/transactions/raw".Today, the data engineer runs the following command to complete this task:After running the command today, the data engineer notices that the number of records in table transactions has not changed.Which of the following describes why the statement might not have copied any new records into the table?
A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table.The code block used by the data engineer is below:If the data engineer only wants the query to execute a micro-batch to process data every 5 seconds, which of the following lines of code should the data engineer use to fill in the blank?
Post your Comments and Discuss Databricks Certified Data Engineer Associate exam prep with other Community members:
Our website is free, but we have to fight against AI bots and content theft. We're sorry for the inconvenience caused by these security measures. You can access the rest of the Certified Data Engineer Associate content, but please register or login to continue.