Free Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 Exam Braindumps
Databricks Certified Associate Developer for Apache Spark 3.5 - Python Questions 26–30 (page: 7)

Updated On: 10-Feb-2026

What is the behavior for function date_sub(start, days) if a negative value is passed into the days parameter?

  1. The same start date will be returned
  2. An error message of an invalid parameter will be returned
  3. The number of days specified will be added to the start date
  4. The number of days specified will be removed from the start date

Answer(s): C

Explanation:

The function date_sub(start, days) subtracts the number of days from the start date. If a negative number is passed, the behavior becomes a date addition.

Example:

SELECT date_sub('2024-05-01', -5)

-- Returns: 2024-05-06

So, a negative value effectively adds the absolute number of days to the date.


Reference:

Spark SQL Functions date_sub()



A data engineer is building an Apache SparkTM Structured Streaming application to process a stream of JSON events in real time. The engineer wants the application to be fault-tolerant and resume processing from the last successfully processed record in case of a failure. To achieve this, the data engineer decides to implement checkpoints.

Which code snippet should the data engineer use?

  1. query = streaming_df.writeStream \
    .format("console") \
    .option("checkpoint", "/path/to/checkpoint") \
    .outputMode("append") \
    .start()
  2. query = streaming_df.writeStream \

    .format("console") \
    .outputMode("append") \
    .option("checkpointLocation", "/path/to/checkpoint") \ .start()
  3. query = streaming_df.writeStream \
    .format("console") \
    .outputMode("complete") \
    .start()
  4. query = streaming_df.writeStream \
    .format("console") \
    .outputMode("append") \
    .start()
  5. Option A
  6. Option B
  7. Option C
  8. Option D

Answer(s): B

Explanation:

To enable fault tolerance and ensure that Spark can resume from the last committed offset after failure, you must configure a checkpoint location using the correct option key: "checkpointLocation".

From the official Spark Structured Streaming guide:

"To make a streaming query fault-tolerant and recoverable, a checkpoint directory must be specified using .option("checkpointLocation", "/path/to/dir")."

Explanation of options:

Option A uses an invalid option name: "checkpoint" (should be "checkpointLocation")

Option B is correct: it sets checkpointLocation properly

Option C lacks checkpointing and won't resume after failure

Option D also lacks checkpointing configuration


Reference:

Apache Spark 3.5 Documentation Structured Streaming Fault Tolerance Semantics



A data engineer is running a batch processing job on a Spark cluster with the following configuration:

10 worker nodes

16 CPU cores per worker node

64 GB RAM per node

The data engineer wants to allocate four executors per node, each executor using four cores.
What is the total number of CPU cores used by the application?

  1. 160
  2. 64
  3. 80
  4. 40

Answer(s): C

Explanation:

If each of the 10 nodes runs 4 executors, and each executor is assigned 4 CPU cores:

Executors per node = 4

Cores per executor = 4

Total executors = 4 * 10 = 40

Total cores = 40 executors * 4 cores = 160 cores

However, Spark uses 1 core for overhead on each node when managing multiple executors. Thus, the practical allocation is:

Total usable executors = 4 executors/node × 10 nodes = 40

Total cores = 4 cores × 40 executors = 160

Answe r : A -- but the question asks specifically about "CPU cores used by the application," assuming all

allocated cores are usable (as Spark typically runs executors without internal core reservation unless explicitly configured).

However, if you are considering 4 executors/node × 4 cores = 16 cores per node, across 10 nodes, that's 160.

Final Answe r: A



A developer is running Spark SQL queries and notices underutilization of resources. Executors are idle, and the number of tasks per stage is low.

What should the developer do to improve cluster utilization?

  1. Increase the value of spark.sql.shuffle.partitions
  2. Reduce the value of spark.sql.shuffle.partitions
  3. Increase the size of the dataset to create more partitions
  4. Enable dynamic resource allocation to scale resources as needed

Answer(s): A

Explanation:

The number of tasks is controlled by the number of partitions. By default, spark.sql.shuffle.partitions is 200. If stages are showing very few tasks (less than total cores), you may not be leveraging full parallelism.

From the Spark tuning guide:

"To improve performance, especially for large clusters, increase spark.sql.shuffle.partitions to create more tasks and parallelism."

Thus:

A is correct: increasing shuffle partitions increases parallelism

B is wrong: it further reduces parallelism

C is invalid: increasing dataset size doesn't guarantee more partitions

D is irrelevant to task count per stage

Final

Answer(s): A



What is the risk associated with this operation when converting a large Pandas API on Spark DataFrame back to a Pandas DataFrame?

  1. The conversion will automatically distribute the data across worker nodes
  2. The operation will fail if the Pandas DataFrame exceeds 1000 rows
  3. Data will be lost during conversion
  4. The operation will load all data into the driver's memory, potentially causing memory overflow

Answer(s): D

Explanation:

When you convert a large pyspark.pandas (aka Pandas API on Spark) DataFrame to a local Pandas DataFrame using .toPandas(), Spark collects all partitions to the driver.

From the Spark documentation:

"Be careful when converting large datasets to Pandas. The entire dataset will be pulled into the driver's memory."

Thus, for large datasets, this can cause memory overflow or out-of-memory errors on the driver.

Final

Answer(s): D



Viewing page 7 of 28
Viewing questions 31 - 35 out of 135 questions


Post your Comments and Discuss Databricks Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 exam prep with other Community members:

AI Tutor AI Tutor 👋 I’m here to help!