Free Databricks Certified Associate Developer for Apache Spark 3.0 Exam braindumps download (Databricks Certified Associate Developer for Apache Spark 3.0 exam dumps Free)

QUESTION: 36

The code block displayed below contains an error. The code block is intended to write DataFrame transactionsDf to disk as a parquet file in location /FileStore/transactions_split, using column storeId as key for partitioning. Find the error. Code block:
transactionsDf.write.format("parquet").partitionOn("storeId").save("/FileStore/transactions_split") A.

The format("parquet") expression is inappropriate to use here, "parquet" should be passed as first argument to the save() operator and "/FileStore/transactions_split" as the second argument.
Partitioning data by storeId is possible with the partitionBy expression, so partitionOn should be replaced by partitionBy.
Partitioning data by storeId is possible with the bucketBy expression, so partitionOn should be replaced by bucketBy.
partitionOn("storeId") should be called before the write operation.
The format("parquet") expression should be removed and instead, the information should be added to the write expression like so: write("parquet").

Answer(s): B

Explanation:

Correct code block: transactionsDf.write.format("parquet").partitionBy("storeId").save("/FileStore/transactions_split")
More info: partition by - Reading files which are written using PartitionBy or BucketBy in Spark - Stack
Overflow
Static notebook | Dynamic notebook: See test 1, Question: 36 ( Databricks import instructions)

Show Answer Next Question

QUESTION: 37

The code block displayed below contains an error. The code block is intended to return all columns of DataFrame transactionsDf except for columns predError, productId, and value. Find the error.
Excerpt of DataFrame transactionsDf: transactionsDf.select(~col("predError"), ~col("productId"), ~col("value"))

The select operator should be replaced by the drop operator and the arguments to the drop operator should be column names predError, productId and value wrapped in the col operator so they should be expressed like drop(col(predError), col(productId), col(value)).
The select operator should be replaced with the deselect operator.
The column names in the select operator should not be strings and wrapped in the col operator, so they should be expressed like select(~col(predError), ~col(productId), ~col(value)).
The select operator should be replaced by the drop operator.
The select operator should be replaced by the drop operator and the arguments to the drop operator should be column names predError, productId and value as strings.

Answer(s): E

Explanation:

Correct code block:
transactionsDf.drop("predError", "productId", "value")
Static notebook | Dynamic notebook: See test 1, Question: 37 ( Databricks import instructions)

Show Answer Next Question

QUESTION: 38

The code block displayed below contains an error. The code block should return DataFrame transactionsDf, but with the column storeId renamed to storeNumber. Find the error.
Code block:
transactionsDf.withColumn("storeNumber", "storeId")

Instead of withColumn, the withColumnRenamed method should be used.
Arguments "storeNumber" and "storeId" each need to be wrapped in a col() operator.
Argument "storeId" should be the first and argument "storeNumber" should be the second argument to the withColumn method.
The withColumn operator should be replaced with the copyDataFrame operator.
Instead of withColumn, the withColumnRenamed method should be used and argument "storeId" should be the first and argument "storeNumber" should be the second argument to that method.

Answer(s): E

Explanation:

Correct code block: transactionsDf.withColumnRenamed("storeId", "storeNumber")
More info: pyspark.sql.DataFrame.withColumnRenamed — PySpark 3.1.1 documentation Static notebook | Dynamic notebook: See test 1, Question: 38 (
Databricks import instructions)

Show Answer Next Question

QUESTION: 39

Which of the following code blocks returns a DataFrame with an added column to DataFrame transactionsDf that shows the unix epoch timestamps in column transactionDate as strings in the format month/day/year in column transactionDateFormatted? Excerpt of DataFrame transactionsDf:

transactionsDf.withColumn("transactionDateFormatted", from_unixtime("transactionDate", format="dd/MM/yyyy"))
transactionsDf.withColumnRenamed("transactionDate", "transactionDateFormatted", from_unixtime("transactionDateFormatted", format="MM/dd/yyyy"))
transactionsDf.apply(from_unixtime(format="MM/dd/yyyy")).asColumn("transactionDateFormatted ")
transactionsDf.withColumn("transactionDateFormatted", from_unixtime("transactionDate", format="MM/dd/yyyy"))
transactionsDf.withColumn("transactionDateFormatted", from_unixtime("transactionDate"))

Answer(s): D

Explanation:

transactionsDf.withColumn("transactionDateFormatted", from_unixtime("transactionDate", format="MM/dd/yyyy"))
Correct. This code block adds a new column with the name transactionDateFormatted to DataFrame transactionsDf, using Spark's from_unixtime method to transform values in column
transactionDate into strings, following the format requested in the question.
transactionsDf.withColumn("transactionDateFormatted", from_unixtime("transactionDate", format="dd/MM/yyyy"))
No. Although almost correct, this uses the wrong format for the timestamp to date conversion: day/month/year instead of month/day/year.
transactionsDf.withColumnRenamed("transactionDate", "transactionDateFormatted", from_unixtime("transactionDateFormatted", format="MM/dd/yyyy"))
Incorrect. This answer uses wrong syntax. The command DataFrame.withColumnRenamed() is for renaming an existing column only has two string parameters, specifying the old and the new name of the column.
transactionsDf.apply(from_unixtime(format="MM/dd/yyyy")).asColumn("transactionDateFormatted ")
Wrong. Although this answer looks very tempting, it is actually incorrect Spark syntax. In Spark, there is no method DataFrame.apply(). Spark has an apply() method that can be used on grouped
data – but this is irrelevant for this question, since we do not deal with grouped data here. transactionsDf.withColumn("transactionDateFormatted", from_unixtime("transactionDate"))
No. Although this is valid Spark syntax, the strings in column transactionDateFormatted would look
like this: 2020-04-26 15:35:32, the default format specified in Spark for from_unixtime and not what is asked for in the question.
More info: pyspark.sql.functions.from_unixtime — PySpark 3.1.1 documentation and pyspark.sql.DataFrame.withColumnRenamed — PySpark 3.1.1 documentation
Static notebook | Dynamic notebook: See test 1, Question: 39 ( Databricks import instructions)

Show Answer Next Question

QUESTION: 40

The code block displayed below contains an error. When the code block below has executed, it should have divided DataFrame transactionsDf into 14 parts, based on columns storeId and transactionDate (in this order). Find the error. Code block:
transactionsDf.coalesce(14, ("storeId", "transactionDate"))

The parentheses around the column names need to be removed and .select() needs to be appended to the code block.
Operator coalesce needs to be replaced by repartition, the parentheses around the column names need to be removed, and .count() needs to be appended to the code block.
Operator coalesce needs to be replaced by repartition, the parentheses around the column names need to be removed, and .select() needs to be appended to the code block.
Operator coalesce needs to be replaced by repartition and the parentheses around the column names need to be replaced by square brackets.
Operator coalesce needs to be replaced by repartition.

Answer(s): B

Explanation:

Correct code block:
transactionsDf.repartition(14, "storeId", "transactionDate").count()
Since we do not know how many partitions DataFrame transactionsDf has, we cannot safely use coalesce, since it would not make any change if the current number of partitions is smaller than 14.

So, we need to use repartition.
In the Spark documentation, the call structure for repartition is shown like this: DataFrame.repartition(numPartitions, *cols). The * operator means that any argument after numPartitions will be interpreted as column. Therefore, the brackets need to be removed.
Finally, the Question: specifies that after the execution the DataFrame should be divided. So, indirectly this Question: is asking us to append an action to the code block. Since .select() is a transformation. the only possible choice here is .count().
More info: pyspark.sql.DataFrame.repartition — PySpark 3.1.1 documentation
Static notebook | Dynamic notebook: See test 1, Question: 40 (Databricks import instructions)

Show Answer Next Question

Free Databricks Certified Associate Developer for Apache Spark 3.0 Exam Braindumps Certified Associate Developer for Apache Spark Questions 36–40 (page: 9)

QUESTION: 36

Explanation:

QUESTION: 37

Explanation:

QUESTION: 38

Explanation:

QUESTION: 39

Explanation:

QUESTION: 40

Explanation:

Free Databricks Certified Associate Developer for Apache Spark 3.0 Exam Braindumps
Certified Associate Developer for Apache Spark Questions 36–40 (page: 9)