Free Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 Exam Braindumps
Databricks Certified Associate Developer for Apache Spark 3.5 - Python Questions 36–40 (page: 9)

Updated On: 10-Feb-2026

A Spark application developer wants to identify which operations cause shuffling, leading to a new stage in the Spark execution plan.

Which operation results in a shuffle and a new stage?

  1. DataFrame.groupBy().agg()
  2. DataFrame.filter()
  3. DataFrame.withColumn()
  4. DataFrame.select()

Answer(s): A

Explanation:

Operations that trigger data movement across partitions (like groupBy, join, repartition) result in a shuffle and a new stage.

From Spark documentation:

"groupBy and aggregation cause data to be shuffled across partitions to combine rows with the same key."

Option A (groupBy + agg) causes shuffle.

Options B, C, and D (filter, withColumn, select) transformations that do not require shuffling; they are narrow dependencies.

Final

Answer(s): A



Which UDF implementation calculates the length of strings in a Spark DataFrame?

  1. df.withColumn("length", spark.udf("len", StringType()))
  2. df.select(length(col("stringColumn")).alias("length"))
  3. spark.udf.register("stringLength", lambda s: len(s))
  4. df.withColumn("length", udf(lambda s: len(s), StringType()))

Answer(s): B

Explanation:

Option B uses Spark's built-in SQL function length(), which is efficient and avoids the overhead of a Python UDF:

from pyspark.sql.functions import length, col df.select(length(col("stringColumn")).alias("length"))

Explanation of other options:

Option A is incorrect syntax; spark.udf is not called this way.

Option C registers a UDF but doesn't apply it in the DataFrame transformation.

Option D is syntactically valid but uses a Python UDF which is less efficient than built-in functions.

Final

Answer(s): B



A data engineer observes that an upstream streaming source sends duplicate records, where duplicates share the same key and have at most a 30-minute difference in event_timestamp. The engineer adds:

dropDuplicatesWithinWatermark("event_timestamp", "30 minutes")

What is the result?

  1. It is not able to handle deduplication in this scenario
  2. It removes duplicates that arrive within the 30-minute window specified by the watermark
  3. It removes all duplicates regardless of when they arrive
  4. It accepts watermarks in seconds and the code results in an error

Answer(s): B

Explanation:

The method dropDuplicatesWithinWatermark() in Structured Streaming drops duplicate records based on a specified column and watermark window. The watermark defines the threshold for how late data is considered valid.

From the Spark documentation:

"dropDuplicatesWithinWatermark removes duplicates that occur within the event-time watermark window."

In this case, Spark will retain the first occurrence and drop subsequent records within the 30-minute watermark window.

Final

Answer(s): B



A Data Analyst needs to retrieve employees with 5 or more years of tenure.

Which code snippet filters and shows the list?

  1. employees_df.filter(employees_df.tenure >= 5).show()
  2. employees_df.where(employees_df.tenure >= 5)
  3. filter(employees_df.tenure >= 5)
  4. employees_df.filter(employees_df.tenure >= 5).collect()

Answer(s): A

Explanation:

To filter rows based on a condition and display them in Spark, use filter(...).show():

employees_df.filter(employees_df.tenure >= 5).show()

Option A is correct and shows the results.

Option B filters but doesn't display them.

Option C uses Python's built-in filter, not Spark.

Option D collects the results to the driver, which is unnecessary if .show() is sufficient.

Final

Answer(s): A



Given a CSV file with the content:



And the following code:

from pyspark.sql.types import *

schema = StructType([

StructField("name", StringType()),

StructField("age", IntegerType())

])

spark.read.schema(schema).csv(path).collect()

What is the resulting output?

  1. [Row(name='bambi'), Row(name='alladin', age=20)]
  2. [Row(name='alladin', age=20)]
  3. [Row(name='bambi', age=None), Row(name='alladin', age=20)]
  4. The code throws an error due to a schema mismatch.

Answer(s): C

Explanation:

In Spark, when a CSV row does not match the provided schema, Spark does not raise an error by default. Instead, it returns null for fields that cannot be parsed correctly.

In the first row, "hello" cannot be cast to Integer for the age field Spark sets age=None

In the second row, "20" is a valid integer age=20

So the output will be:

[Row(name='bambi', age=None), Row(name='alladin', age=20)]

Final

Answer(s): C



Viewing page 9 of 28
Viewing questions 41 - 45 out of 135 questions


Post your Comments and Discuss Databricks Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 exam prep with other Community members:

AI Tutor AI Tutor 👋 I’m here to help!