HOTSPOT (Drag and Drop is not supported)You have a Microsoft Founcy project that contains a Retrieval Augmented Generation (RAG) solution.You need to run a pre-production evaluation by using labeled CSV dataset that contains the query, context,response and ground truth. The evaluation must measure the following:▪ Whether responses address the user query▪ Whether responses are supported by the provided context▪ Whether responses contain sensitive or proprietary informationWhich AI quality evaluation metrics should you use? To answer, select the appropriate options in the answer area.Note: Each correct selection is worth one point.Hot Area:
Answer(s): A
Box 1: Groundedness and RelevanceThe two best AI quality evaluation metrics for your pre-production evaluation in Microsoft Foundry areRelevance and Groundedness.Microsoft Foundry provides built-in, AI-assisted quality evaluators specifically designed to measure these two critical dimensions of a Retrieval-Augmented Generation (RAG) system:Relevance: This metric directly addresses your first requirement: whether responses address the user query. It evaluates how well the gens.nerated response aligns with and answers the initial user query.Groundedness: This metric directly addresses your second requirement: whether responses are supported by the provided context. It evaluates the degree to which the generated response is factually supported by and derived strictly from the retrieved context, which helps capture model hallucinatioBox 2: Protected materialThe best AI quality evaluation metric for this scenario is Protected material.Direct alignment: The Protected material metric specifically checks whether an AI model outputs copyrighted text, proprietary data, or sensitive information that matches its training data or internal context.
https://learn.microsoft.com/en-us/azure/foundry/concepts/observability https://learn.microsoft.com/en-us/azure/foundry/how-to/develop/langchain-middleware
You plan to configure an evaluation in Microsoft Foundry for a Retrieval Augmented Generation (RAG) chat app.You need to provide scores for groundedness, relevance, and harmful content categories.Which two evaluation categories can you use? Each correct answer presents a complete solution.Note: Each correct selection is worth one point.
Answer(s): A,E
The two evaluation categories that would be of use are AI quality (AI assisted) metrics and risk and safety metricsRisk and safety metrics: Scoring harmful content categories (such as violence, sexual content, hate speech,and self-harm) is handled directly by the specialized safety evaluators contained within this category.AI quality (AI assisted) metrics: In Microsoft Foundry, evaluating RAG-specific criteria like groundedness and relevance requires large language models (LLMs) acting as a judge to assess semantic context. These are explicitly categorized under AI-assisted quality metrics rather than standard natural language processing (NLP)rules.
https://learn.microsoft.com/en-us/azure/foundry/concepts/evaluation-evaluators/rag-evaluators https://learn.microsoft.com/en-us/azure/foundry-classic/how-to/develop/evaluate-sdk
You have a Microsoft Foundry project that contains an agent. The agent uses two tools to perform the following actions:▪ Use Azure AI Search to retrieve answers from a private product documentation index.▪ Use the web search tool to retrieve public information on the internet.You need to ensure that for a specific run, the agent deterministically retrieves information only from the internet.To what should you set tool_choice?
To force a specific run to exclusively use public information from the internet instead of your private documents,override the tool orchestration payload at the run level. Set the tool_choice directive specifically to the web grounding tool (e.g., bing_grounding or web_search) and constrain the tools array for that run.
https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/tools/ai-search
You have a Microsoft Foundry project that contains a customer support agent built on a deployed chat model.The agent responses are validated by using an automated testing system that compares generated answers to stored expected outputs. Identical prompts must return consistent response to prevent automated test failures.You need to reduce response variability, without modifying the prompt or reducing factual accuracy.What should you do for the model?
Answer(s): C
To reduce response variability and ensure identical prompts return consistent answers, you should decrease the temperature parameter.Temperature controls the randomness of the model's output. Setting the temperature closer to 0 makes the model deterministic. It forces the model to choose the highest-probability words every time, ensuring that identical prompts consistently yield identical or near-identical responses to pass your automated testing.
https://learn.microsoft.com/en-us/azure/foundry/agents/concepts/runtime-components
You are developing prompts for a Microsoft Foundry project that classifies incoming support tickets by category.You need to improve accuracy by showing the model how correct classifications look, without retaining the model or storing knowledge permanently.Which prompt engineering approach should you use?
Answer(s): D
Few-shot prompting is the best approach for this project. This technique improves classification accuracy by including a few high-quality examples directly inside the prompt, giving the model a clear pattern to follow without modifying its weights or storing data permanently.Zero retraining: Works entirely through in-context learning during the API call.No permanent storage: The knowledge disappears as soon as the inference request is completed.Immediate accuracy boost: Demonstrates formatting, nuances, and edge cases directly to the model.
https://www.linkedin.com/pulse/top-interview-questions-answers-prompt-engineering-nitin-sharma-ka4fc
Post your Comments and Discuss Microsoft AI-103 exam prep with other Community members:
Our website is free, but we have to fight against AI bots and content theft. We're sorry for the inconvenience caused by these security measures. You can access the rest of the AI-103 content, but please register or login to continue.