A data science team completes multiple training runs within an experiment by using MLflow.The team wants to store a selected model in Azure Machine Learning so that it can be versioned and deployed later.The model must be versioned centrally for reuse across environments.You need to version the trained model.Which two actions should you perform? Each correct answer presents part of the solution. Choose two.Note: Each correct selection is worth one point.
Answer(s): A,B
To set up versioning for a trained model in an Azure Machine Learning (Azure ML) workspace using MLflow, you must capture the model artifacts during the training run and then register the model into the centralized registry.[A] 1. Capture Model Artifacts During each training run, use the MLflow SDK to log the model. This ensures that all necessary files (the model binary, environment dependencies, and the MLmodel metadata) are stored as run outputs in the workspace. Manual Logging: Use a flavor-specific method like mlflow.sklearn.log_model(model, "model_path") within an active run.Automatic Logging: Call mlflow.autolog() before starting your training. This automatically captures metrics, parameters, and the model artifacts for supported frameworks.Artifact Location: Once logged, artifacts are typically found in the outputs/ folder of the specific run, accessible via the Azure Machine Learning Studio.[B] 2. Register the Model After identifying the best-performing run, you register it to the Model Registry. This creates a named, versioned entity that can be accessed across different environments for deployment.To set up versioning for an MLflow model in Azure Machine Learning (Azure ML) that is accessible across different environments, you should use a centralized Azure ML Registry. While a standard Azure ML Workspace acts as an MLflow server for individual experiments, an Azure ML Registry is the specifically designed feature for sharing models, environments, and components across multiple workspaces and environments within an Azure tenant.3. Centralized Reuse By registering the model in the workspace's registry, you establish a single source of truth. You can then load this specific version in any environment (e.g., staging or production) using its registry URI: models:/ <model_name>/<version_or_alias>.Incorrect: [Not D] Must use an Azure ML Registry (Central).
https://mlflow.org/docs/latest/ml/model-registry/
DRAG DROP (Drag and Drop is not supported)A team deploys a classification model to production and monitors performance and data changes.The team wants to ensure that significant drops in prediction accuracy automatically trigger the following:-Stakeholders must be notified of the drops.-Retraining must be initiated when thresholds are exceededYou need to configure monitoring to meet the requirements.Which four actions should you perform in sequence? To answer, move the appropriate actions from the list of actions to the answer area and arrange them in the correct order.Select and Place:
Answer(s): A
To meet the requirements of automatic notification and retraining, the steps should follow this sequence:Step 1: Define performance thresholds for monitored metrics. Define performance thresholds Establish what constitutes a "significant drop." Set specific KPIs (e.g., Accuracy < 90% or Data Drift > 0.5).Step 2: Configure alerts to notify stakeholders. Configure alerts and triggers Connect thresholds to notification systems (Azure Monitor/Action Groups). Define the "Event" that will kick off the next stepsStep 3: Monitor baseline metrics from production traffic. Monitor baseline and production metrics Collect real-time data from the deployed endpoint. Compare live telemetry against the established baseline.Step 4: Enable schedule or event-based retraining. Execute the retraining pipeline automatically once the monitor detects a threshold breach.
https://learn.microsoft.com/en-us/azure/machine-learning/how-to-monitor-model-performance
An Azure Machine Learning workspace contains multiple registered versions of a model that is used in production.An older model version must no longer be deployable, but it must remain available for compliance review and potential rollback.You need to change the state of the model version to meet the requirements.What should you do?
Answer(s): C
To ensure that older Azure Machine Learning model versions are no longer deployable but remain available for reference or historical tracking, you should archive those specific versions. How to Archive Model Versions Archiving a model version effectively hides it from standard list queries and management interfaces, preventing accidental deployment while maintaining its metadata and files in the workspace.Behavior of Archived Models: Hidden by Default: They will no longer appear in the Azure Machine Learning Studio model list unless you explicitly filter for archived assets.Referenceable: You can still reference and use an archived model version in existing workflows if you have its specific version number.Restoreable: If you need to redeploy the model in the future, you can use the restore command to make it active again.
https://learn.microsoft.com/en-us/azure/machine-learning/how-to-manage-models
A team is deploying machine learning models to a production inference endpoint in Azure Machine Learning.The team requires a safe way to validate a new model version without disrupting existing users.You need to recommend a deployment strategy for controlled testing of a new model version.What should you configure?
The best strategy for controlled testing of a new model version in Azure Machine Learning is Blue-Green Deployment, often referred to as a safe rollout.This approach allows you to deploy a new model version alongside the current one within the same Managed Online Endpoint without disrupting existing users. Key Features of Blue-Green Deployment in Azure ML Simultaneous Versions: Both the current "Blue" and new "Green" models run concurrently on the same endpoint.*-> Traffic Shifting: You can use the endpoint's load balancer to allocate a specific percentage (e.g., 10%) of live production traffic to the new version.Mirrored Traffic: For even lower risk, you can test the new model with mirrored traffic, where production requests are copied to the new model for validation without using its responses for the end user.Instant Rollback: If the new model performs poorly, you can instantly shift 100% of traffic back to the original version.Deployment Headers: You can bypass general traffic splitting to test the "Green" deployment specifically by adding an azureml-model-deployment header to your HTTP requests.
https://learn.microsoft.com/en-us/azure/machine-learning/how-to-safely-rollout-online-endpoints
HOTSPOT (Drag and Drop is not supported)You are monitoring a fine-tuned large language model deployed in Microsoft Foundry.You evaluate the model before and after fine-tuning by using the same evaluation dataset.You review the following evaluation results:You need to determine whether the fine-tuned model shows improved performance without introducing regression. For each of the following statements, select Yes if the statement is true. Otherwise, select No.Note: Each correct selection is worth one point.Hot Area:
Box 1: Yes Yes - The fine-tuned model shows improved grounding compared to the baseline model.The groundness score increased from 0.82 to 0.91 after fine-tuning. In Microsoft Foundry monitoring, a higher groundedness score indicates that the fine-tuned model's answers are better supported by the provided input source (context or database) compared to the baseline model.Box 2: No No - The fine-tuned model introduced a regression in hallucination behavior.The hallucination rate decreased from 0.06 to 0.05 after fine-tuning. If the hallucination rate decreases when comparing a fine-tuned model against a baseline model (base model), it indicates that the fine-tuned model is performing better and has introduced a reduction in hallucination behavior, not a regression.Box 3: No No - The fine-tuned model's response latency indicates a functional failure.The response latency (ms) increased from 420 to 430 after fine-tuning. In Microsoft AI Foundry, an increase in response latency (ms) for a fine-tuned model compared to its baseline does not necessarily indicate a "functional failure" in the sense of a system crash or broken code. However, it is a critical performance regression that may signal a failure to meet the specific business or technical goals of the fine-tuning project.Box 4: Yes Yes - The evaluation results indicate that fine-tuning improved overall response quality.The relevance score increased from 0.85 to 0.88 after fine-tuning. In the context of monitoring a fine-tuned large language model (LLM) in Microsoft (Azure) AI Foundry, an increase in the relevance score when comparing the fine-tuned model against the baseline model generally indicates that the fine-tuning process has improved the overall response quality.
https://learn.microsoft.com/en-us/azure/machine-learning/prompt-flow/concept-model-monitoring-generative-ai-evaluation-metrics
DRAG DROP (Drag and Drop is not supported)A team runs training jobs by using multiple Azure Machine Learning pipelines.The team must ensure that all runs use the same Python packages and system libraries. The solution must allow dependency updates to be versioned without modifying training code.You need to configure the workspace so that runtime dependencies are consistent and reusable.Which four actions should you perform in sequence? To answer, move the appropriate actions from the list of actions to the answer area and arrange them in the correct order.Select and Place:
To ensure that training jobs across multiple Azure Machine Learning pipelines use consistent and reusable runtime dependencies, you should use Azure Machine Learning environments. These act as versioned containers for your software dependencies, allowing you to update packages without touching your core training logic.Key Actions to Take To achieve this setup, follow these steps in your Azure Machine Learning workspace:Step 1: Define the required dependencies in an environment specification Action 1: To ensure consistent and reusable runtimes across multiple Azure Machine Learning (Azure ML) pipelines, you should define and store dependencies using a Conda specification file (YAML) or a Docker context, which is then used to create a Versioned Azure ML Environment.Can use a Conda specification file (conda.yml). This handles both Python packages and system-level libraries (via conda-forge). Python: Use pip for specific libraries. System: Use conda for binaries or drivers.Step 2: Register the environment in the Azure Machine Learning workspace Action 2: Register a Custom Environment: Define your Python packages and system libraries in a conda.yml file or a Dockerfile. Use the CLI or SDK to register this as a named Environment asset in your workspace.Step 3: Create a new version of the environment when dependencies change Action3: Version the Environment: When dependencies need to change, create a new version of the same environment (e.g., my-env:2). This allows older pipelines to continue using version 1 while new runs transition to version 2.Step 4: Reference the Environment in training jobs and pipelines Action 4: Reference the Environment by Name: In your pipeline components or job configurations, point to the environment by its registered name and version. This decouples the infrastructure from the code.
https://immrbhattarai.medium.com/azure-machine-learning-studio-750756c90234 https://blog.devops.dev/continuous-integration-and-continuous-deployment-ci-cd-for-machine-learning-projects-using-github-8bf450b9a228
Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution.After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear in the review screen.You have an Azure Machine Learning workspace. You connect to a terminal session from the Notebooks page in Azure Machine Learning studio.You plan to add a new Jupyter kernel that will be accessible from the same terminal session.You need to perform the task that must be completed before you can add the new kernel.Solution: Delete the Python 3.6 - AzureML kernel.Does the solution meet the goal?
Answer(s): B
Correct: * Create an environment.Incorrect: * Delete the Python 3.6 - AzureML kernel. * Delete the Python 3.8 - AzureML kernel.Note: Before you can add a new Jupyter kernel on an Azure Machine Learning compute instance terminal, you must create a Conda environment.Required Workflow To officially provision and expose the new kernel to your Azure Machine Learning studio Notebooks, you need to execute the following full process from your terminal session: 1. Create the environment: Provision a new isolated environment (e.g., using conda create -n newenv python=3.10). 2. Activate the environment: Run conda activate newenv.3. Install dependencies: Add the required ipykernel package using conda install ipykernel or pip install ipykernel. 4. Register the kernel: Bind the new environment configuration to the global Jupyter directory by running: python -m ipykernel install --user --name newenv --display-name "My New Kernel"
https://docs.azure.cn/en-us/machine-learning/how-to-access-terminal
Note: This question is part of a series of questions that present the same scenario. Each question in the series contains a unique solution that might meet the stated goals. Some question sets might have more than one correct solution, while others might not have a correct solution.After you answer a question in this section, you will NOT be able to return to it. As a result, these questions will not appear in the review screen.You have an Azure Machine Learning workspace. You connect to a terminal session from the Notebooks page in Azure Machine Learning studio.You plan to add a new Jupyter kernel that will be accessible from the same terminal session.You need to perform the task that must be completed before you can add the new kernel.Solution: Delete the Python 3.8 - AzureML kernel.Does the solution meet the goal?
Correct: * Create an environment.Incorrect: * Delete the Python 3.6 - AzureML kernel. * Delete the Python 3.8 - AzureML kernel.Note: Before you can add a new Jupyter kernel on an Azure Machine Learning compute instance terminal, you must create a Conda environment.Required Workflow To officially provision and expose the new kernel to your Azure Machine Learning studio Notebooks, you need to execute the following full process from your terminal session: 1. Create the environment: Provision a new isolated environment (e.g., using conda create -n newenv python=3.10). 2. Activate the environment: Run conda activate newenv. 3. Install dependencies: Add the required ipykernel package using conda install ipykernel or pip install ipykernel. 4. Register the kernel: Bind the new environment configuration to the global Jupyter directory by running: python -m ipykernel install --user --name newenv --display-name "My New Kernel"
Post your Comments and Discuss Microsoft AI-300 exam prep with other Community members:
Our website is free, but we have to fight against AI bots and content theft. We're sorry for the inconvenience caused by these security measures. You can access the rest of the AI-300 content, but please register or login to continue.