support@Free-Braindumps.com
Register Login
Cisco
CompTIA
ISC
EC-Council
EXIN
Fortinet
ITIL
Juniper
Microsoft
VMware
Avaya
SAP
Google
NVIDIA
Salesforce
Amazon
Palo Alto Networks
ISC2
Splunk
ServiceNow
All Vendors
  1. Home
  2. Exams
  3. Amazon
  4. Amazon-DEA-C01

Amazon Amazon-DEA-C01 Exam Prep
AWS Certified Data Engineer - Associate (Page 8 )

Updated On: 20-Aug-2026
Page 8 of 47
PDF with 366 Questions

A data engineer has a one-time task to read data from objects that are in Apache Parquet format in an Amazon S3 bucket. The data engineer needs to query only one column of the data.
Which solution will meet these requirements with the LEAST operational overhead?

  1. Configure an AWS Lambda function to load data from the S3 bucket into a pandas dataframe. Write a SQL SELECT statement on the dataframe to query the required column.
  2. Use S3 Select to write a SQL SELECT statement to retrieve the required column from the S3 objects.
  3. Prepare an AWS Glue DataBrew project to consume the S3 objects and to query the required column.
  4. Run an AWS Glue crawler on the S3 objects. Use a SQL SELECT statement in Amazon Athena to query the required column.

Answer(s): B

Explanation:

The best solution is
B: Use S3 Select to write a SQL SELECT statement to retrieve the required column from the S3 objects.
Here's why:
Operational Overhead: The core requirement is least operational overhead. S3 Select stands out because it directly queries data within S3, avoiding the need to move or transform the entire dataset. It requires minimal setup.
S3 Select Functionality: S3 Select uses simple SQL expressions to retrieve a subset of data from an object, such as a single column. Since Parquet is a supported format, S3 Select can efficiently parse and query the data.
Cost Efficiency: By only retrieving the necessary column, S3 Select reduces data transfer costs. You only pay for the data scanned and returned.
Alternatives' Drawbacks:
Option A (Lambda + Pandas): This approach involves loading the entire Parquet file into memory within a Lambda function, even though only one column is needed. Pandas provides querying functionality but it's not optimized for large datasets directly in S3. This increases both memory usage and Lambda execution time, leading to higher costs and potential memory limitations. It requires code development and deployment. Option C (Glue DataBrew): Glue DataBrew is suitable for data cleaning and transformation but it's overkill for a simple query, adds operational complexity because you are not transforming anything and you are working with very big data sets. It requires project setup and configuration. Option D (Glue Crawler + Athena): This is an acceptable method, however there is too much overhead. This involves crawling the S3 bucket to create metadata in the Glue Data Catalog and then using Athena to query.
While Athena is also a good option for querying data in S3, the Glue Crawler adds unnecessary operational overhead for a one-time, single-column retrieval.
Direct Querying: S3 Select aligns perfectly with the task's requirement of directly querying data in S3 without requiring additional services for data loading or transformation, ensuring the least operational overhead.
In summary, S3 Select is the simplest, fastest, and most cost-effective way to query a single column from Parquet files in S3 for a one-time task.
Authoritative Links:
Amazon S3 Select: https://docs.aws.amazon.com/AmazonS3/latest/userguide/selecting-content-from-objects.html



A company uses Amazon Redshift for its data warehouse. The company must automate refresh schedules for Amazon Redshift materialized views.
Which solution will meet this requirement with the LEAST effort?

  1. Use Apache Airflow to refresh the materialized views.
  2. Use an AWS Lambda user-defined function (UDF) within Amazon Redshift to refresh the materialized views.
  3. Use the query editor v2 in Amazon Redshift to refresh the materialized views.
  4. Use an AWS Glue workflow to refresh the materialized views.

Answer(s): C

Explanation:

The correct answer is C, using the query editor v2 in Amazon Redshift to refresh the materialized views. Here's why:
The primary goal is to automate refresh schedules for Redshift materialized views with the least effort. Query editor v2 offers a straightforward interface for scheduling queries, including REFRESH MATERIALIZED VIEW .
This requires minimal setup and is directly integrated within the Redshift console. You can simply create a scheduled query to execute the REFRESH MATERIALIZED VIEW command at desired intervals.
Option A, Apache Airflow, introduces significant overhead. Airflow is a powerful orchestration tool, but setting up and managing an Airflow cluster specifically for this single task is an overkill and requires considerably more effort than necessary. This involves defining DAGs, managing dependencies, and maintaining the Airflow infrastructure.
Option B, Lambda UDF, is also less efficient.
While Lambda can execute code, using it within a Redshift UDF to refresh materialized views is complex. It would require setting up proper IAM roles and managing network configurations to allow Redshift to invoke the Lambda function. Furthermore, UDFs have limitations on execution time, which could be problematic for large materialized views.
Option D, AWS Glue workflow, is designed for ETL (Extract, Transform, Load) processes and not specifically for refreshing materialized views.
While Glue could be used to achieve this, it would again involve unnecessary complexity of defining crawlers, transformations, and jobs when a simple scheduled query would suffice. Glue is better suited for more complex data processing tasks, rather than just refreshing a MV.
Therefore, leveraging the built-in scheduling capabilities of Redshift's query editor v2 provides the simplest and most direct approach to automating materialized view refreshes. It requires the least amount of effort, configuration, and management overhead.
Refer to the official Amazon Redshift documentation for more details on scheduled queries:
Amazon Redshift Scheduled Queries REFRESH MATERIALIZED VIEW



A data engineer must orchestrate a data pipeline that consists of one AWS Lambda function and one AWS Glue job. The solution must integrate with AWS services.
Which solution will meet these requirements with the LEAST management overhead?

  1. Use an AWS Step Functions workflow that includes a state machine. Configure the state machine to run the Lambda function and then the AWS Glue job.
  2. Use an Apache Airflow workflow that is deployed on an Amazon EC2 instance. Define a directed acyclic graph (DAG) in which the first task is to call the Lambda function and the second task is to call the AWS Glue job.
  3. Use an AWS Glue workflow to run the Lambda function and then the AWS Glue job.
  4. Use an Apache Airflow workflow that is deployed on Amazon Elastic Kubernetes Service (Amazon EKS). Define a directed acyclic graph (DAG) in which the first task is to call the Lambda function and the second task is to call the AWS Glue job.

Answer(s): A

Explanation:

The correct answer is A: Use an AWS Step Functions workflow that includes a state machine. Configure the state machine to run the Lambda function and then the AWS Glue job.
Here's a detailed justification:
AWS Step Functions is a fully managed, serverless orchestration service that allows you to coordinate multiple AWS services into serverless workflows. It's designed for building resilient, auditable, and scalable state machines without managing servers. This aligns perfectly with the requirement of least management overhead.
Option A directly leverages the strengths of Step Functions:
Orchestration: Step Functions can easily orchestrate the execution of the Lambda function and the Glue job in a defined order (Lambda first, then Glue). Integration: Step Functions integrates seamlessly with AWS Lambda and AWS Glue, making it straightforward to call these services from within the state machine definition. Serverless: Step Functions is serverless, meaning you don't need to provision or manage any infrastructure. This minimizes operational overhead. State Management: Step Functions manages the state of the workflow, including error handling and retries. This simplifies the overall pipeline design and improves resilience.
Options B and D involve using Apache Airflow, which, while a powerful orchestration tool, requires more management overhead. You have to manage the Airflow environment itself, whether deployed on EC2 or EKS. This includes maintaining the underlying infrastructure, scaling resources, and handling upgrades. Airflow is a more complex solution than needed for simply orchestrating two tasks.
Option C, using AWS Glue workflows, is specifically designed for orchestrating Glue jobs and crawlers.
While Glue workflows can trigger external actions through triggers, they are primarily focused on Glue's ecosystem and aren't the most efficient way to integrate and orchestrate arbitrary Lambda functions before a Glue job. Step Functions offers more general-purpose orchestration capabilities.
Therefore, Step Functions (Option A) provides the least management overhead because it is a fully managed, serverless orchestration service that integrates directly with the required AWS services (Lambda and Glue). It simplifies the coordination of the pipeline without requiring you to manage any underlying infrastructure.
Relevant Links:
AWS Step Functions: https://aws.amazon.com/step-functions/ AWS Lambda: https://aws.amazon.com/lambda/ AWS Glue: https://aws.amazon.com/glue/



A company needs to set up a data catalog and metadata management for data sources that run in the AWS Cloud. The company will use the data catalog to maintain the metadata of all the objects that are in a set of data stores. The data stores include structured sources such as Amazon RDS and Amazon Redshift. The data stores also include semistructured sources such as JSON files and .xml files that are stored in Amazon S3. The company needs a solution that will update the data catalog on a regular basis. The solution also must detect changes to the source metadata.
Which solution will meet these requirements with the LEAST operational overhead?

  1. Use Amazon Aurora as the data catalog. Create AWS Lambda functions that will connect to the data catalog. Configure the Lambda functions to gather the metadata information from multiple sources and to update the Aurora data catalog. Schedule the Lambda functions to run periodically.
  2. Use the AWS Glue Data Catalog as the central metadata repository. Use AWS Glue crawlers to connect to multiple data stores and to update the Data Catalog with metadata changes. Schedule the crawlers to run periodically to update the metadata catalog.
  3. Use Amazon DynamoDB as the data catalog. Create AWS Lambda functions that will connect to the data catalog. Configure the Lambda functions to gather the metadata information from multiple sources and to update the DynamoDB data catalog. Schedule the Lambda functions to run periodically.
  4. Use the AWS Glue Data Catalog as the central metadata repository. Extract the schema for Amazon RDS and Amazon Redshift sources, and build the Data Catalog. Use AWS Glue crawlers for data that is in Amazon S3 to infer the schema and to automatically update the Data Catalog.

Answer(s): B

Explanation:

The correct answer is
B: Use the AWS Glue Data Catalog as the central metadata repository. Use AWS Glue crawlers to connect to multiple data stores and to update the Data Catalog with metadata changes. Schedule the crawlers to run periodically to update the metadata catalog.
Here's a detailed justification:
AWS Glue is specifically designed as a fully managed ETL (Extract, Transform, Load) service and a data catalog. Its Data Catalog serves as a central repository for storing metadata about data assets. AWS Glue crawlers are automated programs that connect to data stores (like RDS, Redshift, and S3) to infer the schema, data types, and other metadata information. Crawlers automatically detect changes to the source metadata and update the Glue Data Catalog accordingly, minimizing operational overhead. Scheduling these crawlers allows for regular updates to the catalog, ensuring that the metadata is current.
Option A is less suitable because managing an Aurora database solely as a data catalog adds unnecessary operational overhead. It requires manually building and maintaining the metadata schema, connection logic, and update mechanisms using Lambda functions, which increases complexity.
Option C, using DynamoDB, suffers from similar issues as Option A. While DynamoDB is a powerful NoSQL database, it's not designed specifically for metadata management like Glue's Data Catalog. Building the metadata structure and update processes manually would be complex and time-consuming.
Option D is partially correct in using AWS Glue, but it incorrectly suggests manually extracting schema for RDS and Redshift. AWS Glue crawlers can automatically infer the schema for these data sources as well, simplifying the process. The added step to extract the schema manually creates more operational overhead.
The crawlers also handle detecting changes to the source metadata automatically.
AWS Glue directly addresses the requirements of a data catalog, metadata management, automated discovery of metadata, and change detection with minimal operational overhead. It is purpose-built for this type of task.
Authoritative Links:
AWS Glue Documentation: https://aws.amazon.com/glue/ AWS Glue Data Catalog: https://docs.aws.amazon.com/glue/latest/dg/components-datacatalog.html AWS Glue Crawlers: https://docs.aws.amazon.com/glue/latest/dg/add-crawler.html



A company stores data from an application in an Amazon DynamoDB table that operates in provisioned capacity mode. The workloads of the application have predictable throughput load on a regular schedule. Every Monday, there is an immediate increase in activity early in the morning. The application has very low usage during weekends. The company must ensure that the application performs consistently during peak usage times.
Which solution will meet these requirements in the MOST cost-effective way?

  1. Increase the provisioned capacity to the maximum capacity that is currently present during peak load times.
  2. Divide the table into two tables. Provision each table with half of the provisioned capacity of the original table. Spread queries evenly across both tables.
  3. Use AWS Application Auto Scaling to schedule higher provisioned capacity for peak usage times. Schedule lower capacity during off-peak times.
  4. Change the capacity mode from provisioned to on-demand. Configure the table to scale up and scale down based on the load on the table.

Answer(s): C

Explanation:

The most cost-effective solution for handling predictable, fluctuating workloads on a DynamoDB table using provisioned capacity mode is C. Use AWS Application Auto Scaling to schedule higher provisioned capacity for peak usage times. Schedule lower capacity during off-peak times.
Here's why:
Predictable Workload: The problem explicitly states the workload has predictable throughput load on a regular schedule (Monday morning peaks, weekend lulls). This makes scheduled capacity adjustments ideal.
Cost Efficiency: Application Auto Scaling allows you to define scaling schedules. You can automatically increase read and write capacity units (RCUs/WCUs) before the Monday morning peak and decrease them during weekends when usage is low. This ensures you only pay for the capacity you need, minimizing costs.
Consistent Performance: By pre-emptively scaling up capacity before the peak, the application maintains consistent performance without throttling. Scaling down during off-peak times prevents over-provisioning and reduces unnecessary costs.
Why not A? Simply increasing provisioned capacity to the maximum peak requirement and leaving it there would be the least cost-effective. You would be paying for unused capacity during off-peak times.
Why not B? Dividing the table would add unnecessary complexity in managing and querying data. It's also not guaranteed to provide better performance without proper sharding and load balancing strategies, which would add more complexity. Spreading queries evenly doesn't address the fundamental problem of insufficient total capacity during the peak.
Why not D? Switching to on-demand capacity mode would incur a different cost structure.
While convenient for unpredictable workloads, it's generally more expensive for predictable workloads where you can accurately provision capacity. Furthermore, the question emphasizes cost-effectiveness, and scheduled scaling in provisioned mode directly addresses that. Auto scaling within provisioned mode can be implemented but would not be as effective as scheduled scaling because the response to the load would take some time.
Supporting Concepts:
Provisioned Capacity Mode: DynamoDB's provisioned capacity mode allows you to specify the expected read and write throughput for your table. You are charged based on these provisioned units. AWS Application Auto Scaling: This service enables you to automatically scale your DynamoDB table capacity based on utilization metrics or, in this case, a schedule. Scheduled Scaling: A feature of Application Auto Scaling that allows you to set up scaling actions to occur at specific times.
Authoritative Links:
DynamoDB Provisioned Capacity Mode AWS Application Auto Scaling Scaling DynamoDB Tables Automatically with AWS Application Auto Scaling



A company is planning to migrate on-premises Apache Hadoop clusters to Amazon EMR. The company also needs to migrate a data catalog into a persistent storage solution. The company currently stores the data catalog in an on-premises Apache Hive metastore on the Hadoop clusters. The company requires a serverless solution to migrate the data catalog.
Which solution will meet these requirements MOST cost-effectively?

  1. Use AWS Database Migration Service (AWS DMS) to migrate the Hive metastore into Amazon S3. Configure AWS Glue Data Catalog to scan Amazon S3 to produce the data catalog.
  2. Configure a Hive metastore in Amazon EMR. Migrate the existing on-premises Hive metastore into Amazon EMR. Use AWS Glue Data Catalog to store the company's data catalog as an external data catalog.
  3. Configure an external Hive metastore in Amazon EMR. Migrate the existing on-premises Hive metastore into Amazon EMR. Use Amazon Aurora MySQL to store the company's data catalog.
  4. Configure a new Hive metastore in Amazon EMR. Migrate the existing on-premises Hive metastore into Amazon EMR. Use the new metastore as the company's data catalog.

Answer(s): B

Explanation:

The correct answer is B because it offers a cost-effective and serverless approach to migrating the Hive metastore. Here's why:
Serverless Solution: AWS Glue Data Catalog is a fully managed, serverless service. This eliminates the need to manage infrastructure for the data catalog, reducing operational overhead and cost. Metastore Migration: Configuring a Hive metastore in Amazon EMR provides a temporary environment for the on-premises metastore to be migrated. This is a necessary step to transition the data catalog to the cloud. External Data Catalog: Configuring the Hive metastore in EMR to use AWS Glue Data Catalog as an external metastore makes the Glue Data Catalog the persistent, central repository for the data catalog after migration. This also avoids the need to retain the EMR metastore. Cost-Effectiveness: This approach is cost-effective because AWS Glue's pricing is based on usage (storage and metadata operations), which scales with the company's needs. Avoids the cost of more complex solutions like DMS or persistent database infrastructure for Aurora.
Here's why the other options are less ideal:
A: Migrating directly to S3 using DMS and then scanning with Glue is unusual and likely more complex than setting up a temporary Hive metastore. C: Using Amazon Aurora MySQL is viable, but over-engineered and more costly than AWS Glue Data Catalog, which is specifically designed for this purpose. Also using the term external is misleading as Aurora would be the main metastore in this scenario. D: Using the EMR Hive metastore directly is not a persistent, serverless solution, as it relies on the EMR cluster's lifecycle. You would have to keep your EMR cluster running indefinitely.
Authoritative Links:
AWS Glue Data Catalog: https://aws.amazon.com/glue/ Amazon EMR: https://aws.amazon.com/emr/



A company uses an Amazon Redshift provisioned cluster as its database. The Redshift cluster has five reserved ra3.4xlarge nodes and uses key distribution. A data engineer notices that one of the nodes frequently has a CPU load over 90%. SQL Queries that run on the node are queued. The other four nodes usually have a CPU load under 15% during daily operations. The data engineer wants to maintain the current number of compute nodes. The data engineer also wants to balance the load more evenly across all five compute nodes.
Which solution will meet these requirements?

  1. Change the sort key to be the data column that is most often used in a WHERE clause of the SQL SELECT statement.
  2. Change the distribution key to the table column that has the largest dimension.
  3. Upgrade the reserved node from ra3.4xlarge to ra3.16xlarge.
  4. Change the primary key to be the data column that is most often used in a WHERE clause of the SQL SELECT statement.

Answer(s): B

Explanation:

The best solution is B. Change the distribution key to the table column that has the largest dimension.
Here's a detailed justification:
The problem describes a situation where one node in a Redshift cluster is heavily overloaded while the others are relatively idle. This strongly indicates data skewness – an uneven distribution of data across the compute nodes.
When data is skewed, queries involving the skewed data will primarily execute on the node holding the majority of that data, leading to high CPU utilization on that node and query queuing.
The distribution key determines how data is distributed across the nodes. If the distribution key is poorly chosen, it can lead to skew. Choosing a column with a large dimension (high cardinality, many unique values) as the distribution key generally promotes a more even distribution of data. Redshift will attempt to distribute rows as evenly as possible based on the distribution key's values.
When the column has a wide range of values, this tends to result in a better spread across the nodes. This strategy is effective in balancing the workload without altering the cluster size.
Option A (changing the sort key) addresses query performance by influencing the order in which data is stored within each node. It does not directly impact how data is distributed across nodes and therefore won't solve the load imbalance issue.
While a proper sort key improves query execution speed within a node, it doesn't address the root cause of one node being overloaded.
Option C (upgrading the node size) could temporarily alleviate the problem by providing more CPU resources to the overloaded node. However, it doesn't address the underlying data skewness issue. The problem will likely reappear as data volumes grow. Upgrading is also a more expensive solution than simply adjusting the distribution key. Furthermore, the question specifically asks for a solution that maintains the current number of compute nodes.
Option D (changing the primary key) is incorrect because Redshift does not enforce primary keys as constraints.
While defining a primary key can aid query optimization and documentation, it has no impact on data distribution or query execution from a data skew perspective. Redshift's query optimizer can leverage primary key information, but changing the definition does not change the distribution of data across nodes.
Therefore, changing the distribution key to a high-cardinality column is the most effective way to redistribute the data more evenly across the nodes, balancing the workload and reducing the CPU load on the overloaded node without increasing the cluster size. This improves overall query performance and cluster efficiency.
Further reading:
Amazon Redshift documentation on distribution keys AWS Big Data Blog: Choosing the right distribution style for Amazon Redshift



A security company stores IoT data that is in JSON format in an Amazon S3 bucket. The data structure can change when the company upgrades the IoT devices. The company wants to create a data catalog that includes the IoT data. The company's analytics department will use the data catalog to index the data.
Which solution will meet these requirements MOST cost-effectively?

  1. Create an AWS Glue Data Catalog. Configure an AWS Glue Schema Registry. Create a new AWS Glue workload to orchestrate the ingestion of the data that the analytics department will use into Amazon Redshift Serverless.
  2. Create an Amazon Redshift provisioned cluster. Create an Amazon Redshift Spectrum database for the analytics department to explore the data that is in Amazon S3. Create Redshift stored procedures to load the data into Amazon Redshift.
  3. Create an Amazon Athena workgroup. Explore the data that is in Amazon S3 by using Apache Spark through Athena. Provide the Athena workgroup schema and tables to the analytics department.
  4. Create an AWS Glue Data Catalog. Configure an AWS Glue Schema Registry. Create AWS Lambda user defined functions (UDFs) by using the Amazon Redshift Data API. Create an AWS Step Functions job to orchestrate the ingestion of the data that the analytics department will use into Amazon Redshift Serverless.

Answer(s): A

Explanation:

The best solution is A because it leverages AWS Glue for cost-effective data cataloging and schema management while providing a modern serverless data warehousing option. AWS Glue Data Catalog provides a central metadata repository for data sources, making data discoverable and accessible. The AWS Glue
Schema Registry tackles the evolving data structure challenge by automatically detecting and managing schema changes, which is crucial for handling data from upgraded IoT devices. Amazon Redshift Serverless allows running analytics queries without managing infrastructure, providing a cost-effective solution compared to provisioned clusters (option B).
Option B involves creating a provisioned Redshift cluster and Redshift Spectrum, which is more expensive than a serverless solution, especially when resource utilization is sporadic or unpredictable. It also necessitates manual maintenance of the Redshift cluster.
Option C relies on Amazon Athena for data exploration but doesn't offer a robust data ingestion and transformation pipeline into a dedicated data warehouse. Athena's primary use case is direct querying of data in S3, not building an optimized analytics platform.
Option D, while using Glue Data Catalog and Schema Registry, employs Lambda UDFs and Step Functions for data ingestion into Redshift Serverless. This introduces unnecessary complexity and operational overhead compared to directly using AWS Glue's ETL capabilities. Glue's ETL can handle data ingestion, transformation, and loading into Redshift Serverless seamlessly.
Therefore, option A offers the most cost-effective and efficient solution by combining the data cataloging capabilities of AWS Glue with the serverless data warehousing power of Redshift Serverless, effectively addressing the requirements while minimizing cost and operational complexity.
Supporting Links:
AWS Glue Data Catalog: https://aws.amazon.com/glue/features/ AWS Glue Schema Registry: https://aws.amazon.com/blogs/big-data/using-the-aws-glue-schema-registry-to-centrally-manage-schemas-and-improve-data-quality/ Amazon Redshift Serverless: https://aws.amazon.com/redshift/serverless/



  PDF
Viewing page 8 of 47
Viewing questions 57 - 64 out of 366 questions


Post your Comments and Discuss Amazon Amazon-DEA-C01 exam prep with other Community members:

AI Tutor AI Tutor 👋 I’m here to help!