Free Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 Exam Braindumps
Databricks Certified Associate Developer for Apache Spark 3.5 - Python Questions 61–65 (page: 14)

Updated On: 10-Feb-2026

A data engineer needs to write a DataFrame df to a Parquet file, partitioned by the column country, and overwrite any existing data at the destination path.

Which code should the data engineer use to accomplish this task in Apache Spark?

  1. df.write.mode("overwrite").partitionBy("country").parquet("/data/output")
  2. df.write.mode("append").partitionBy("country").parquet("/data/output")
  3. df.write.mode("overwrite").parquet("/data/output")
  4. df.write.partitionBy("country").parquet("/data/output")

Answer(s): A

Explanation:

The .mode("overwrite") ensures that existing files at the path will be replaced.

.partitionBy("country") optimizes queries by writing data into partitioned folders.

Correct syntax:

df.write.mode("overwrite").partitionBy("country").parquet("/data/output")

-- Source: Spark SQL, DataFrames and Datasets Guide



A data engineer is working with a large JSON dataset containing order information. The dataset is stored in a distributed file system and needs to be loaded into a Spark DataFrame for analysis. The data engineer wants to ensure that the schema is correctly defined and that the data is read efficiently.

Which approach should the data scientist use to efficiently load the JSON data into a Spark DataFrame with a predefined schema?

  1. Use spark.read.json() to load the data, then use DataFrame.printSchema() to view the inferred schema, and finally use DataFrame.cast() to modify column types.
  2. Use spark.read.json() with the inferSchema option set to true
  3. Use spark.read.format("json").load() and then use DataFrame.withColumn() to cast each column to the desired data type.
  4. Define a StructType schema and use spark.read.schema(predefinedSchema).json() to load the data.

Answer(s): D

Explanation:

The most efficient and correct approach is to define a schema using StructType and pass it to spark.read.schema(...).

This avoids schema inference overhead and ensures proper data types are enforced during read.

Example:

from pyspark.sql.types import StructType, StructField, StringType, DoubleType schema = StructType([

StructField("order_id", StringType(), True),

StructField("amount", DoubleType(), True),

...

])

df = spark.read.schema(schema).json("path/to/json")

-- Source: Databricks Guide ­ Read JSON with predefined schema



A data engineer wants to create a Streaming DataFrame that reads from a Kafka topic called feed.



Which code fragment should be inserted in line 5 to meet the requirement?

Code context:

spark \

.readStream \

.format("kafka") \

.option("kafka.bootstrap.servers", "host1:port1,host2:port2") \

.[LINE 5] \

.load()

Options:

  1. .option("subscribe", "feed")
  2. .option("subscribe.topic", "feed")
  3. .option("kafka.topic", "feed")
  4. .option("topic", "feed")

Answer(s): A

Explanation:

To read from a specific Kafka topic using Structured Streaming, the correct syntax is:

python

CopyEdit

.option("subscribe", "feed")

This is explicitly defined in the Spark documentation:

"subscribe ­ The Kafka topic to subscribe to. Only one topic can be specified for this option." (Source: Apache Spark Structured Streaming + Kafka Integration Guide)

B . "subscribe.topic" is invalid.

C . "kafka.topic" is not a recognized option.

D . "topic" is not valid for Kafka source in Spark.



A data scientist is working on a project that requires processing large amounts of structured data, performing SQL queries, and applying machine learning algorithms. The data scientist is considering using Apache Spark for this task.

Which combination of Apache Spark modules should the data scientist use in this scenario?

Options:

  1. Spark DataFrames, Structured Streaming, and GraphX
  2. Spark SQL, Pandas API on Spark, and Structured Streaming
  3. Spark Streaming, GraphX, and Pandas API on Spark
  4. Spark DataFrames, Spark SQL, and MLlib

Answer(s): D

Explanation:

Comprehensive

To cover structured data processing, SQL querying, and machine learning in Apache Spark, the correct combination of components is:

Spark DataFrames: for structured data processing

Spark SQL: to execute SQL queries over structured data

MLlib: Spark's scalable machine learning library

This trio is designed for exactly this type of use case.

Why other options are incorrect:

A: GraphX is for graph processing -- not needed here.

B: Pandas API on Spark is useful, but MLlib is essential for ML, which this option omits.

C: Spark Streaming is legacy; GraphX is irrelevant here.


Reference:

Apache Spark Modules Overview



A data scientist is working with a Spark DataFrame called customerDF that contains customer information. The DataFrame has a column named email with customer email addresses. The data scientist needs to split this column into username and domain parts.

Which code snippet splits the email column into username and domain columns?

  1. customerDF.select(
    col("email").substr(0, 5).alias("username"),
    col("email").substr(-5).alias("domain")
    )
  2. customerDF.withColumn("username", split(col("email"), "@").getItem(0)) \ .withColumn("domain", split(col("email"), "@").getItem(1))
  3. customerDF.withColumn("username", substring_index(col("email"), "@", 1)) \ .withColumn("domain", substring_index(col("email"), "@", -1))
  4. customerDF.select(
    regexp_replace(col("email"), "@", "").alias("username"), regexp_replace(col("email"), "@", "").alias("domain")
    )
  5. Option A
  6. Option B

  7. Option C
  8. Option D

Answer(s): B

Explanation:

Option B is the correct and idiomatic approach in PySpark to split a string column (like email) based on a delimiter such as "@".

The split(col("email"), "@") function returns an array with two elements: username and domain.

getItem(0) retrieves the first part (username).

getItem(1) retrieves the second part (domain).

withColumn() is used to create new columns from the extracted values.

Example from official Databricks Spark documentation on splitting columns:

from pyspark.sql.functions import split, col df.withColumn("username", split(col("email"), "@").getItem(0)) \

.withColumn("domain", split(col("email"), "@").getItem(1))

Why other options are incorrect:

A uses fixed substring indices (substr(0, 5)), which won't correctly extract usernames and domains of varying lengths.

C uses substring_index, which is available but less idiomatic for splitting emails and is slightly less readable.

D removes "@" from the email entirely, losing the separation between username and domain, and ends up duplicating values in both fields.

Therefore, Option B is the most accurate and reliable solution according to Apache Spark 3.5 best practices.



Viewing page 14 of 28
Viewing questions 66 - 70 out of 135 questions


Post your Comments and Discuss Databricks Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 exam prep with other Community members:

AI Tutor AI Tutor 👋 I’m here to help!