Free Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 Exam Braindumps
Databricks Certified Associate Developer for Apache Spark 3.5 - Python Questions 21–25 (page: 6)

Updated On: 10-Feb-2026

Given:

python

CopyEdit spark.sparkContext.setLogLevel("<LOG_LEVEL>")

Which set contains the suitable configuration settings for Spark driver LOG_LEVELs?

  1. ALL, DEBUG, FAIL, INFO
  2. ERROR, WARN, TRACE, OFF
  3. WARN, NONE, ERROR, FATAL
  4. FATAL, NONE, INFO, DEBUG

Answer(s): B

Explanation:

The setLogLevel() method of SparkContext sets the logging level on the driver, which controls the verbosity of logs emitted during job execution. Supported levels are inherited from log4j and include the following:

ALL

DEBUG

ERROR

FATAL

INFO

OFF

TRACE

WARN

According to official Spark and Databricks documentation:

"Valid log levels include: ALL, DEBUG, ERROR, FATAL, INFO, OFF, TRACE, and WARN."

Among the choices provided, only option B (ERROR, WARN, TRACE, OFF) includes four valid log levels and excludes invalid ones like "FAIL" or "NONE".


Reference:

Apache Spark API docs SparkContext.setLogLevel



An engineer wants to join two DataFrames df1 and df2 on the respective employee_id and emp_id columns:

df1: employee_id INT, name STRING

df2: emp_id INT, department STRING

The engineer uses:

result = df1.join(df2, df1.employee_id == df2.emp_id, how='inner')

What is the behaviour of the code snippet?

  1. The code fails to execute because the column names employee_id and emp_id do not match automatically
  2. The code fails to execute because it must use on='employee_id' to specify the join column explicitly
  3. The code fails to execute because PySpark does not support joining DataFrames with a different structure
  4. The code works as expected because the join condition explicitly matches employee_id from df1 with emp_id from df2

Answer(s): D

Explanation:

In PySpark, when performing a join between two DataFrames, the columns do not have to share the same name. You can explicitly provide a join condition by comparing specific columns from each DataFrame.

This syntax is correct and fully supported:

df1.join(df2, df1.employee_id == df2.emp_id, how='inner')

This will perform an inner join between df1 and df2 using the employee_id from df1 and emp_id from df2.


Reference:

Databricks Spark 3.5 Documentation DataFrame API join()



A data engineer has been asked to produce a Parquet table which is overwritten every day with the latest data. The downstream consumer of this Parquet table has a hard requirement that the data in this table is produced with all records sorted by the market_time field.

Which line of Spark code will produce a Parquet table that meets these requirements?

  1. final_df \
    .sort("market_time") \
    .write \
    .format("parquet") \
    .mode("overwrite") \
    .saveAsTable("output.market_events")
  2. final_df \
    .orderBy("market_time") \
    .write \
    .format("parquet") \
    .mode("overwrite") \
    .saveAsTable("output.market_events")
  3. final_df \
    .sort("market_time") \
    .coalesce(1) \
    .write \
    .format("parquet") \
    .mode("overwrite") \
    .saveAsTable("output.market_events")
  4. final_df \
    .sortWithinPartitions("market_time") \

    .write \
    .format("parquet") \
    .mode("overwrite") \
    .saveAsTable("output.market_events")
  5. Option A
  6. Option B
  7. Option C
  8. Option D

Answer(s): D

Explanation:

To ensure that data written out to disk is sorted, it is important to consider how Spark writes data when saving to Parquet tables. The methods .sort() or .orderBy() apply a global sort but do not guarantee that the sorting will persist in the final output files unless certain conditions are met (e.g. a single partition via .coalesce(1) -- which is not scalable).

Instead, the proper method in distributed Spark processing to ensure rows are sorted within their respective partitions when written out is:

.sortWithinPartitions("column_name")

According to Apache Spark documentation:

"sortWithinPartitions() ensures each partition is sorted by the specified columns. This is useful for downstream systems that require sorted files."

This method works efficiently in distributed settings, avoids the performance bottleneck of global sorting (as in .orderBy() or .sort()), and guarantees each output partition has sorted records -- which meets the requirement of consistently sorted data.

Thus:

Option A and B do not guarantee the persisted file contents are sorted.

Option C introduces a bottleneck via .coalesce(1) (single partition).

Option D correctly applies sorting within partitions and is scalable.


Reference:

Databricks & Apache Spark 3.5 Documentation DataFrame API sortWithinPartitions()



In the code block below, aggDF contains aggregations on a streaming DataFrame:



Which output mode at line 3 ensures that the entire result table is written to the console during each trigger execution?

  1. complete
  2. append
  3. replace
  4. aggregate

Answer(s): A

Explanation:

The correct output mode for streaming aggregations that need to output the full updated results at each trigger is "complete".

From the official documentation:

"complete: The entire updated result table will be output to the sink every time there is a trigger."

This is ideal for aggregations, such as counts or averages grouped by a key, where the result table changes incrementally over time.

append: only outputs newly added rows replace and aggregate: invalid values for output mode


Reference:

Spark Structured Streaming Programming Guide Output Modes



A data engineer is running a Spark job to process a dataset of 1 TB stored in distributed storage. The cluster has 10 nodes, each with 16 CPUs. Spark UI shows:

Low number of Active Tasks

Many tasks complete in milliseconds

Fewer tasks than available CPUs

Which approach should be used to adjust the partitioning for optimal resource allocation?

  1. Set the number of partitions equal to the total number of CPUs in the cluster
  2. Set the number of partitions to a fixed value, such as 200
  3. Set the number of partitions equal to the number of nodes in the cluster
  4. Set the number of partitions by dividing the dataset size (1 TB) by a reasonable partition size, such as 128 MB

Answer(s): D

Explanation:

Spark's best practice is to estimate partition count based on data volume and a reasonable partition size -- typically 128 MB to 256 MB per partition.

With 1 TB of data: 1 TB / 128 MB ~8000 partitions

This ensures that tasks are distributed across available CPUs for parallelism and that each task processes an optimal volume of data.

Option A (equal to cores) may result in partitions that are too large. Option B (fixed 200) is arbitrary and may underutilize the cluster.

Option C (nodes) gives too few partitions (10), limiting parallelism.


Reference:

Databricks Spark Tuning Guide Partitioning Strategy



Viewing page 6 of 28
Viewing questions 26 - 30 out of 135 questions


Post your Comments and Discuss Databricks Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 exam prep with other Community members:

AI Tutor AI Tutor 👋 I’m here to help!