You need to trigger an Azure Data Factory pipeline when a file arrives in an Azure Data Lake Storage Gen2 container. Which resource provider should you enable?
Answer(s): C
Event-driven architecture (EDA) is a common data integration pattern that involves production, detection, consumption, and reaction to events. Data integration scenarios often require Data Factory customers to trigger pipelines based on events happening in storage account, such as the arrival or deletion of a file in Azure Blob Storage account. Data Factory natively integrates with Azure Event Grid, which lets you trigger pipelines on such events.
https://docs.microsoft.com/en-us/azure/data-factory/how-to-create-event-triggerhttps://docs.microsoft.com/en-us/azure/data-factory/concepts-pipeline-execution-triggers
You plan to perform batch processing in Azure Databricks once daily. Which type of Databricks cluster should you use?
Answer(s): B
Azure Databricks has two types of clusters: interactive and automated. You use interactive clusters to analyze data collaboratively with interactive notebooks. You use automated clusters to run fast and robust automated jobs.Example: Scheduled batch workloads (data engineers running ETL jobs)This scenario involves running batch job JARs and notebooks on a regular cadence through the Databricks platform.The suggested best practice is to launch a new cluster for each run of critical jobs. This helps avoid any issues (failures, missing SLA, and so on) due to an existing workload (noisy neighbor) on a shared cluster.
https://docs.databricks.com/administration-guide/cloud-configurations/aws/cmbp.html#scenario-3-scheduled-batch-workloads-data-engineers-running- etl-jobs
HOTSPOT (Drag and Drop is not supported)You are processing streaming data from vehicles that pass through a toll booth.You need to use Azure Stream Analytics to return the license plate, vehicle make, and hour the last vehicle passed during each 10-minute window. How should you complete the query? To answer, select the appropriate options in the answer area.NOTE: Each correct selection is worth one point.Hot Area:
Answer(s): A
Box 1: MAXThe first step on the query finds the maximum time stamp in 10-minute windows, that is the time stamp of the last event for that window. The second step joins the results of the first query with the original stream to find the event that match the last time stamps in each window.Query:WITH LastInWindow AS( SELECT MAX(Time) AS LastEventTime FROM Input TIMESTAMP BY Time GROUP BY TumblingWindow(minute, 10))SELECT Input.License_plate, Input.Make, Input.TimeFROM Input TIMESTAMP BY Time INNER JOIN LastInWindow ON DATEDIFF(minute, Input, LastInWindow) BETWEEN 0 AND 10 AND Input.Time = LastInWindow.LastEventTimeBox 2: TumblingWindowTumbling windows are a series of fixed-sized, non-overlapping and contiguous time intervals.Box 3: DATEDIFFDATEDIFF is a date-specific function that compares and returns the time difference between two DateTime fields, for more information, refer to date functions.
https://docs.microsoft.com/en-us/stream-analytics-query/tumbling-window-azure-stream-analytics
You have an Azure Data Factory instance that contains two pipelines named Pipeline1 and Pipeline2. Pipeline1 has the activities shown in the following exhibit.Pipeline2 has the activities shown in the following exhibit. You execute Pipeline2, and Stored procedure1 in Pipeline1 fails. What is the status of the pipeline runs?
Activities are linked together via dependencies. A dependency has a condition of one of the following: Succeeded, Failed, Skipped, or Completed.Consider Pipeline1:If we have a pipeline with two activities where Activity2 has a failure dependency on Activity1, the pipeline will not fail just because Activity1 failed. If Activity1 fails and Activity2 succeeds, the pipeline will succeed. This scenario is treated as a try-catch block by Data Factory.The failure dependency means this pipeline reports success.Note:If we have a pipeline containing Activity1 and Activity2, and Activity2 has a success dependency on Activity1, it will only execute if Activity1 is successful. In this scenario, if Activity1 fails, the pipeline will fail.
https://datasavvy.me/category/azure-data-factory/
HOTSPOT (Drag and Drop is not supported)A company plans to use Platform-as-a-Service (PaaS) to create the new data pipeline process. The process must meet the following requirements:Ingest:-Access multiple data sources.-Provide the ability to orchestrate workflow.-Provide the capability to run SQL Server Integration Services packages.Store:-Optimize storage for big data workloads.-Provide encryption of data at rest.-Operate with no size limits.Prepare and Train:-Provide a fully-managed and interactive workspace for exploration and visualization.-Provide the ability to program in R, SQL, Python, Scala, and Java.-Provide seamless user authentication with Azure Active Directory.Model & Serve:-Implement native columnar storage.-Support for the SQL language-Provide support for structured streaming.You need to build the data integration pipeline.Which technologies should you use? To answer, select the appropriate options in the answer area.NOTE: Each correct selection is worth one point.
Ingest: Azure Data FactoryAzure Data Factory pipelines can execute SSIS packages.In Azure, the following services and tools will meet the core requirements for pipeline orchestration, control flow, and data movement: Azure Data Factory, Oozie on HDInsight, and SQL Server Integration Services (SSIS).Store: Data Lake StorageData Lake Storage Gen1 provides unlimited storage.Note: Data at rest includes information that resides in persistent storage on physical media, in any digital format. Microsoft Azure offers a variety of data storage solutions to meet different needs, including file, disk, blob, and table storage. Microsoft also provides encryption to protect Azure SQL Database, Azure Cosmos DB, and Azure Data Lake.Prepare and Train: Azure DatabricksAzure Databricks provides enterprise-grade Azure security, including Azure Active Directory integration.With Azure Databricks, you can set up your Apache Spark environment in minutes, autoscale and collaborate on shared projects in an interactive workspace. Azure Databricks supports Python, Scala, R, Java and SQL, as well as data science frameworks and libraries including TensorFlow, PyTorch and scikit-learn.Model and Serve: Azure Synapse AnalyticsAzure Synapse Analytics/ SQL Data Warehouse stores data into relational tables with columnar storage.Azure SQL Data Warehouse connector now offers efficient and scalable structured streaming write support for SQL Data Warehouse. Access SQL Data Warehouse from Azure Databricks using the SQL Data Warehouse connector.Note: Note: As of November 2019, Azure SQL Data Warehouse is now Azure Synapse Analytics.
https://docs.microsoft.com/bs-latn-ba/azure/architecture/data-guide/technology-choices/pipeline-orchestration-data-movementhttps://docs.microsoft.com/en-us/azure/azure-databricks/what-is-azure-databricks
Post your Comments and Discuss Microsoft DP-203 exam prep with other Community members:
Our website is free, but we have to fight against AI bots and content theft. We're sorry for the inconvenience caused by these security measures. You can access the rest of the DP-203 content, but please register or login to continue.