Free Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 Exam Braindumps
Databricks Certified Associate Developer for Apache Spark 3.5 - Python Questions 31–35 (page: 8)

Updated On: 10-Feb-2026

A developer notices that all the post-shuffle partitions in a dataset are smaller than the value set for spark.sql.adaptive.maxShuffledHashJoinLocalMapThreshold.
Which type of join will Adaptive Query Execution (AQE) choose in this case?

  1. A Cartesian join
  2. A shuffled hash join
  3. A broadcast nested loop join
  4. A sort-merge join

Answer(s): B

Explanation:

Adaptive Query Execution (AQE) dynamically selects join strategies based on actual data sizes at runtime. If the size of post-shuffle partitions is below the threshold set by:

spark.sql.adaptive.maxShuffledHashJoinLocalMapThreshold then Spark prefers to use a shuffled hash join.

From the Spark documentation:

"AQE selects a shuffled hash join when the size of post-shuffle data is small enough to fit within the configured threshold, avoiding more expensive sort-merge joins."

Therefore:

A is wrong -- Cartesian joins are only used with no join condition.

B is correct -- this is the optimized join for small partitioned shuffle data under AQE.

C and D are used under other scenarios but not for this case.

Final

Answer(s): B



Which configuration can be enabled to optimize the conversion between Pandas and PySpark DataFrames using Apache Arrow?

  1. spark.conf.set("spark.pandas.arrow.enabled", "true")
  2. spark.conf.set("spark.sql.execution.arrow.pyspark.enabled", "true")
  3. spark.conf.set("spark.sql.execution.arrow.enabled", "true")
  4. spark.conf.set("spark.sql.arrow.pandas.enabled", "true")

Answer(s): B

Explanation:

Apache Arrow is used under the hood to optimize conversion between Pandas and PySpark DataFrames. The correct configuration setting is:

spark.conf.set("spark.sql.execution.arrow.pyspark.enabled", "true")

From the official documentation:

"This configuration must be enabled to allow for vectorized execution and efficient conversion between Pandas and PySpark using Arrow."

Option B is correct.

Options A, C, and D are invalid config keys and not recognized by Spark.

Final

Answer(s): B



A data engineer writes the following code to join two DataFrames df1 and df2:

df1 = spark.read.csv("sales_data.csv") # ~10 GB

df2 = spark.read.csv("product_data.csv") # ~8 MB

result = df1.join(df2, df1.product_id == df2.product_id)



Which join strategy will Spark use?

  1. Shuffle join, because AQE is not enabled, and Spark uses a static query plan
  2. Broadcast join, as df2 is smaller than the default broadcast threshold
  3. Shuffle join, as the size difference between df1 and df2 is too large for a broadcast join to work efficiently
  4. Shuffle join because no broadcast hints were provided

Answer(s): B

Explanation:

The default broadcast join threshold in Spark is:

spark.sql.autoBroadcastJoinThreshold = 10MB

Since df2 is only 8 MB (less than 10 MB), Spark will automatically apply a broadcast join without requiring explicit hints.

From the Spark documentation:

"If one side of the join is smaller than the broadcast threshold, Spark will automatically broadcast it to all executors."

A is incorrect because Spark does support auto broadcast even with static plans.

B is correct: Spark will automatically broadcast df2.

C and D are incorrect because Spark's default logic handles this optimization.

Final

Answer(s): B



A developer wants to test Spark Connect with an existing Spark application.
What are the two alternative ways the developer can start a local Spark Connect server without changing their existing application code? (Choose 2 answers)

  1. Execute their pyspark shell with the option --remote "https://localhost"
  2. Execute their pyspark shell with the option --remote "sc://localhost"
  3. Set the environment variable SPARK_REMOTE="sc://localhost" before starting the pyspark shell
  4. Add .remote("sc://localhost") to their SparkSession.builder calls in their Spark code
  5. Ensure the Spark property spark.connect.grpc.binding.port is set to 15002 in the application code

Answer(s): B,C

Explanation:

Spark Connect enables decoupling of the client and Spark driver processes, allowing remote access. Spark supports configuring the remote Spark Connect server in multiple ways:

From Databricks and Spark documentation:

Option B (--remote "sc://localhost") is a valid command-line argument for the pyspark shell to connect using Spark Connect.

Option C (setting SPARK_REMOTE environment variable) is also a supported method to configure the remote endpoint.

Option A is incorrect because Spark Connect uses the sc:// protocol, not https://. Option D requires modifying the code, which the question explicitly avoids. Option E configures the port on the server side but doesn't start a client connection.

Final Answers: B and C



A Spark developer is building an app to monitor task performance. They need to track the maximum task processing time per worker node and consolidate it on the driver for analysis.

Which technique should be used?

  1. Use an RDD action like reduce() to compute the maximum time
  2. Use an accumulator to record the maximum time on the driver
  3. Broadcast a variable to share the maximum time among workers
  4. Configure the Spark UI to automatically collect maximum times

Answer(s): A

Explanation:

The correct way to aggregate information (e.g., max value) from distributed workers back to the driver is using RDD actions such as reduce() or aggregate().

From the documentation:

"To perform global aggregations on distributed data, actions like reduce() are commonly used to collect summaries such as min/max/avg."

Accumulators (Option B) do not support max operations directly and are not intended for such analytics.
Broadcast (Option C) is used to send data to workers, not collect from them. Spark UI (Option D) is a monitoring tool -- not an analytics collection interface.

Final

Answer(s): A



Viewing page 8 of 28
Viewing questions 36 - 40 out of 135 questions


Post your Comments and Discuss Databricks Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 exam prep with other Community members:

AI Tutor AI Tutor 👋 I’m here to help!