CASE STUDYPlease use the following to answer the next question: A local police department in the United States procured an AI system to monitor and analyze social media feeds, online marketplaces and other sources of public information to detect evidence of illegal activities (e.g., sale of drugs or stolen goods). The AI system works by surveying the public sites in order to identify individuals that are likely to have committed a crime. It cross-references the individuals against data maintained by law enforcement and then assigns a percentage score of the likelihood of criminal activity based on certain factors like previous criminal history, location, time, race and gender. The police department retained a third-party consultant to assist in the procurement process, specifically to evaluate two finalists. Each of the vendors provided information about their system’s accuracy rates, the diversity of their training data and how their system works. The consultant determined that the first vendor’s system has a higher accuracy rate and based on this information, recommended this vendor to the police department. The police department chose the first vendor and implemented its AI system. As part of the implementation, the department and consultant created a usage policy for the system, which includes training police officers on how the system works and how to incorporate it into their investigation process. The police department has now been using the AI system for a year. An internal review has found that every time the system scored a likelihood of criminal activity at or above 90%, the police investigation subsequently confirmed that the individual had, in fact, committed a crime. Based on these results, the police department wants to forego investigations for cases where the AI system gives a score of at least 90% and proceed directly with an arrest. Which AI risk would NOT have been identified during the procurement process based on the categories of information requested by the third-party consultant?
Answer(s): A
A is correct because the procurement process focused on system accuracy, explainability, and discrimination, but did not sufficiently address security vulnerabilities associated with the AI system.The third-party consultant evaluated the vendors on their accuracy rates, training data diversity, and operational mechanics, all of which fall under performance and fairness considerations. However, factors such as data breaches or cyber threats—which fundamentally impact the security of the AI system—were not explicitly analyzed. Security risks, particularly those associated with potential misuse or unauthorized access to sensitive data, can lead to harmful consequences and undermine public trust in law enforcement.B: Accuracy is incorrect because this was a primary focus during the procurement process. The consultant specifically reviewed the accuracy rates of the AI systems provided by both vendors and utilized this information for recommendation.C: Explainability is also incorrect since the procurement process required vendors to detail how their systems functioned. By understanding the rationale behind the AI's decision-making process, law enforcement could better evaluate the technology's use and implications.D: Discrimination is ruled out because the consultant assessed the diversity of the training data, which is a foundational element in identifying potential biases. This focus aimed to ensure that the AI system did not disproportionately target individuals based on race or other protected characteristics.In summary, while accuracy, explainability, and discrimination were adequately considered in the procurement process, security risks were overlooked, potentially jeopardizing the integrity and efficacy of the AI application in policing.References:https://www.nist.gov/publications/ai-risk-management-framework https://www.ojp.gov/library/publications/ethical-considerations-use-ai-policing https://www.aclublm.org/research-repository/ai-and-security-risks
References:https://www.nist.gov/publications/ai-risk-management-framework https://www.ojp.gov/library/publications/ethical-considerations-use-ai-policing https://www.aclublm.org/research-repository/ai-and-security-risks
CASE STUDYPlease use the following to answer the next question: A local police department in the United States procured an AI system to monitor and analyze social media feeds, online marketplaces and other sources of public information to detect evidence of illegal activities (e.g., sale of drugs or stolen goods). The AI system works by surveying the public sites in order to identify individuals that are likely to have committed a crime. It cross-references the individuals against data maintained by law enforcement and then assigns a percentage score of the likelihood of criminal activity based on certain factors like previous criminal history, location, time, race and gender. The police department retained a third-party consultant to assist in the procurement process, specifically to evaluate two finalists. Each of the vendors provided information about their system’s accuracy rates, the diversity of their training data and how their system works. The consultant determined that the first vendor’s system has a higher accuracy rate and based on this information, recommended this vendor to the police department. The police department chose the first vendor and implemented its AI system. As part of the implementation, the department and consultant created a usage policy for the system, which includes training police officers on how the system works and how to incorporate it into their investigation process. The police department has now been using the AI system for a year. An internal review has found that every time the system scored a likelihood of criminal activity at or above 90%, the police investigation subsequently confirmed that the individual had, in fact, committed a crime. Based on these results, the police department wants to forego investigations for cases where the AI system gives a score of at least 90% and proceed directly with an arrest. The best human oversight mechanism for the police department to implement is that a police officer should?
Answer(s): D
D is correct because the police officer should consider the AI recommendation as part of a broader criminal investigation rather than rely solely on it to make decisions.In criminal justice, the integration of AI recommendations into the investigative process must ensure that human oversight mitigates potential biases and acknowledges the limitations inherent in machine learning algorithms. Although the AI system has shown high accuracy in detecting criminal activity, these scores should not replace human judgment, as they do not account for contextual variables such as the nuances of individual cases or potential algorithmic bias. According to legal precedents, police officers must adopt a holistic approach that includes evaluating situational context, corroborating evidence, and ensuring fairness in the application of justice.Evaluation of Incorrect Options:A: Explain to the accused how the AI system works. This option focuses too much on transparency but does not actively enhance oversight or ensure that the AI’s recommendations are critically evaluated in the broader context of a criminal investigation, which is paramount in safeguarding justice.B: Confirm the AI recommendation prior to sentencing. While confirming the AI recommendation before sentencing might seem diligent, it occurs too late in the process. Decisions prior to arrest and charge must be informed by a comprehensive investigation that includes but does not rely on AI outputs.C: Ensure an accused is given notice that the AI system was used. Notification alone does not contribute to effective oversight. Understanding that AI was used does not equate to ensuring that its findings were adequately scrutinized or sufficiently integrated into investigative procedures.In summary, implementing a process where police officers consider AI recommendations as part of the criminal investigation helps balance technological advancements with the ethical imperatives of justice. This ensures that critical human insight and oversight are maintained.1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6536706/ 2. https://www.rand.org/pubs/research_reports/RR3081.html 3. https://www.acm.org/publications/policies/ai-code-of-ethics
CASE STUDYPlease use the following to answer the next question: A local police department in the United States procured an AI system to monitor and analyze social media feeds, online marketplaces and other sources of public information to detect evidence of illegal activities (e.g., sale of drugs or stolen goods). The AI system works by surveying the public sites in order to identify individuals that are likely to have committed a crime. It cross-references the individuals against data maintained by law enforcement and then assigns a percentage score of the likelihood of criminal activity based on certain factors like previous criminal history, location, time, race and gender. The police department retained a third-party consultant to assist in the procurement process, specifically to evaluate two finalists. Each of the vendors provided information about their system’s accuracy rates, the diversity of their training data and how their system works. The consultant determined that the first vendor’s system has a higher accuracy rate and based on this information, recommended this vendor to the police department. The police department chose the first vendor and implemented its AI system. As part of the implementation, the department and consultant created a usage policy for the system, which includes training police officers on how the system works and how to incorporate it into their investigation process. The police department has now been using the AI system for a year. An internal review has found that every time the system scored a likelihood of criminal activity at or above 90%, the police investigation subsequently confirmed that the individual had, in fact, committed a crime. Based on these results, the police department wants to forego investigations for cases where the AI system gives a score of at least 90% and proceed directly with an arrest. During the procurement process, what is the most likely reason that the third-party consultant asked each vendor for information about the diversity of their datasets?
Answer(s): B
B is correct because assessing the diversity of datasets is fundamental to evaluating the fairness of the AI system.The diversity of training data directly impacts the AI's ability to make impartial assessments, which is paramount in law enforcement applications. A lack of diversity can result in biased outcomes, leading to disproportionate targeting of specific demographic groups and perpetuating systemic inequality. For instance, if the training data primarily includes data from a single racial or socio-economic group, the AI may inaccurately augment policing against those underrepresented groups, thereby introducing bias and unfairness to the policing process.Evaluation of Alternatives:A: To comply with applicable law. While legal compliance is critical, the primary intent in evaluating dataset diversity pertains to fairness rather than mere legal adherence. Laws have yet to mandate comprehensive diversity assessments, making this option less compelling.C: To evaluate the reliability of the AI system. Reliability generally focuses on the AI's performance and accuracy under varying conditions rather than the demographic representation in the datasets. While diverse data can enhance reliability, it is not the primary reason for assessing diversity.D: To determine the explainability of the AI system. Explainability concerns the model's ability to clarify its decision-making process. Though diverse datasets can contribute to a model's overall performance, they do not intrinsically affect how decisions are explained. Evaluating diversity is primarily about ensuring equitable outcomes rather than elucidating the system's operations.In conclusion, assessing dataset diversity is crucial for evaluating the fairness of AI systems in policing environments, as it has profound implications for civil rights and community trust.References:https://www.nist.gov/publications/measuring-fairness-machine-learning https://www.aaai.org/ojs/index.php/aimagazine/article/view/2864 https://www.un.org/en/universal-declaration-of-human-rights
References:https://www.nist.gov/publications/measuring-fairness-machine-learning https://www.aaai.org/ojs/index.php/aimagazine/article/view/2864 https://www.un.org/en/universal-declaration-of-human-rights
CASE STUDYPlease use the following to answer the next question: A local police department in the United States procured an AI system to monitor and analyze social media feeds, online marketplaces and other sources of public information to detect evidence of illegal activities (e.g., sale of drugs or stolen goods). The AI system works by surveying the public sites in order to identify individuals that are likely to have committed a crime. It cross-references the individuals against data maintained by law enforcement and then assigns a percentage score of the likelihood of criminal activity based on certain factors like previous criminal history, location, time, race and gender. The police department retained a third-party consultant to assist in the procurement process, specifically to evaluate two finalists. Each of the vendors provided information about their system’s accuracy rates, the diversity of their training data and how their system works. The consultant determined that the first vendor’s system has a higher accuracy rate and based on this information, recommended this vendor to the police department. The police department chose the first vendor and implemented its AI system. As part of the implementation, the department and consultant created a usage policy for the system, which includes training police officers on how the system works and how to incorporate it into their investigation process. The police department has now been using the AI system for a year. An internal review has found that every time the system scored a likelihood of criminal activity at or above 90%, the police investigation subsequently confirmed that the individual had, in fact, committed a crime. Based on these results, the police department wants to forego investigations for cases where the AI system gives a score of at least 90% and proceed directly with an arrest. What should a police officer explain about the use of the AI system at the time of arrest?
D is correct because it is imperative to inform the individual of how they were specifically identified by the AI system, ensuring transparency and due process in law enforcement.The justification for selecting option D centers on the ethical obligation of law enforcement to provide an understanding of the decision-making processes leading to an arrest. Transparency in how the AI identifies individuals promotes accountability, mitigates biases, and sustains public trust in police actions (e.g., transparency in algorithmic decision-making). Furthermore, clarifying how the individual was identified aligns with legal standards of fair treatment and can aid in any subsequent legal considerations or challenges (Cohen & Kamar, https://www.cs.cmu.edu/~././~kamar/pdf/AI-In-Law-Enforcement.pdf ).Evaluation of Incorrect Options:Option A (Information about the scoring protocol of the AI system): While understanding scoring is important, it does not directly address how the individual was specifically identified. This could create confusion without contextualizing the individual’s unique situation, lacking relevancy to their case.Option B (Information about the public sites surveilled by the AI system): This option does not address the individual’s identification process. Knowledge of the sources does not directly contribute to the individual’s understanding of why they were flagged or arrested.Option C (Information about the use of the AI system in prosecuting the case): This option focuses on the potential outcomes post-arrest rather than the immediate identification rationale. This fails to provide the needed clarity at the time of arrest, which is crucial for the individual's awareness of their situation.In conclusion, option D delivers vital information that impacts the individual's understanding of the arrest process, reinforcing ethical standards and upholding legal principles in AI-assisted law enforcement (Graham, https://www.technologyreview.com/2020/05/11/1001247/ai-police-surveillance-justice-bias/ ); hence, it is the correct choice.References:1. https://www.cs.cmu.edu/~././~kamar/pdf/AI-In-Law-Enforcement.pdf 2. https://www.technologyreview.com/2020/05/11/1001247/ai-police-surveillance-justice-bias/ 3. https://www.nap.edu/catalog/25133/ai-in-healthcare-a-report-published-by-the-national-academy-of-sciences-engineering-and-medicine
References:1. https://www.cs.cmu.edu/~././~kamar/pdf/AI-In-Law-Enforcement.pdf 2. https://www.technologyreview.com/2020/05/11/1001247/ai-police-surveillance-justice-bias/ 3. https://www.nap.edu/catalog/25133/ai-in-healthcare-a-report-published-by-the-national-academy-of-sciences-engineering-and-medicine
You are an engineer that developed an AI-based ad recommendation tool. Which of the following should be monitored to evaluate the tool’s effectiveness?
A is correct as monitoring output data provides the most direct measure of the tool’s effectiveness by comparing predicted ad clicks to actual outcomes.The primary objective of an AI-based ad recommendation tool is to enhance user engagement by increasing ad click rates. Monitoring output data enables engineers to assess the performance of the model quantitatively and iteratively refine it based on empirical evidence. Tracking this delta facilitates targeted improvements and adjustments, ensuring alignment with business objectives.Critique of Other Options:B: Monitoring algorithmic patterns is less effective as it primarily focuses on model performance metrics such as accuracy and precision, which do not directly reflect real-world effectiveness. High accuracy may not guarantee improved ad performance if the model fails to engage users.C: While assessing input data is crucial for ensuring ads reach the desired audience, it does not measure the direct outcome of the recommendation tool. Inputs do not account for how the audience interacts with the ads, making this metric less relevant for evaluating effectiveness post-deployment.D: Evaluating GPU performance is important for assessing the computational efficiency of the tool but provides limited insight into the tool's actual performance in terms of user engagement and revenue generation. A robust system may still yield suboptimal results if the output is not effectively translating to user interactions.References:1. https://towardsdatascience.com/evaluating-the-performance-of-machine-learning-models-a37068cfe183 2. https://www.oreilly.com/library/view/architecting-the-cloud/9781491984556/ch04.html 3. https://www.analyticsvidhya.com/blog/2020/12/evaluating-machine-learning-models-a-beginners-guide/
References:1. https://towardsdatascience.com/evaluating-the-performance-of-machine-learning-models-a37068cfe183 2. https://www.oreilly.com/library/view/architecting-the-cloud/9781491984556/ch04.html 3. https://www.analyticsvidhya.com/blog/2020/12/evaluating-machine-learning-models-a-beginners-guide/
Post your Comments and Discuss IAPP AIGP exam prep with other Community members:
Our website is free, but we have to fight against AI bots and content theft. We're sorry for the inconvenience caused by these security measures. You can access the rest of the AIGP content, but please register or login to continue.