CASE STUDYPlease use the following to answer the next question: A mid-size US healthcare network has decided to develop an AI solution to detect a type of cancer that is most likely to arise in adults. Specifically, the healthcare network intends to create a recognition algorithm that will perform an initial review of all imaging and then route records to a radiologist for secondary review pursuant to agreed-upon criteria (e.g., a confidence score below a threshold). To date, the healthcare network has: Defined its AI ethical principles. Conducted discovery to identify the intended uses and success criteria for the system. Established an AI risk committee. Assembled a cross-functional team with clear roles and responsibilities. Created policies and procedures to document standards, workflows, timelines and risk thresholds during the project. The healthcare network intends to retain a cloud provider to host the solution. It also intends to retain a large consulting firm to supplement its small data science team and help develop the algorithm using the healthcare network’s existing data and de-identified data that is licensed from a large US clinical research partner. In the design phase, which of the following steps is most important in gathering the data from the clinical research partner?
Answer(s): D
The most important step in gathering data from the clinical research partner during the design phase is reviewing the terms of use (D). Here's why:Before any data combination, security measures, or quality assessments can occur, understanding the legal and ethical boundaries defined within the data usage agreement is paramount. The terms of use dictate how the healthcare network is legally permitted to use the licensed data. This includes specifying permissible uses, restrictions on data manipulation, data transfer limitations, liability clauses, and data retention policies.Failing to understand these terms could lead to several issues:Legal non-compliance: Using the data in a way that violates the terms of use can result in legal action, fines, and reputational damage. Invalid Model: If the data quality or format deviates from what the healthcare network anticipates without it being made clear in the terms, the resulting AI model may be ineffective. Breach of ethical principles: The terms of use may also outline restrictions necessary to safeguard patient privacy or prevent biased model outcomes. Ignoring these would directly contradict the healthcare network's already defined AI ethical principles. Security gaps: The terms of use could mandate specific security measures or compliance frameworks that must be adhered to when handling the data. Neglecting these requirements could expose the organization to data breaches and regulatory penalties.Therefore, ensuring the healthcare network's intended use of the data aligns with the terms of use is the foundation upon which all subsequent data handling and model development activities should be built. Anonymizing the data, securing it, and performing quality assessments are all essential, but they cannot proceed ethically or legally without first verifying compliance with the terms of use agreement.Supporting Links:Data Governance Principles: https://www.dama.org/content/body-knowledge HIPAA guidelines on Data Usage Agreements: https://www.hhs.gov/hipaa/for-professionals/special-topics/de-identification/index.html
CASE STUDYPlease use the following to answer the next question: A mid-size US healthcare network has decided to develop an AI solution to detect a type of cancer that is most likely to arise in adults. Specifically, the healthcare network intends to create a recognition algorithm that will perform an initial review of all imaging and then route records to a radiologist for secondary review pursuant to agreed-upon criteria (e.g., a confidence score below a threshold). To date, the healthcare network has: Defined its AI ethical principles. Conducted discovery to identify the intended uses and success criteria for the system. Established an AI risk committee. Assembled a cross-functional team with clear roles and responsibilities. Created policies and procedures to document standards, workflows, timelines and risk thresholds during the project. The healthcare network intends to retain a cloud provider to host the solution. It also intends to retain a large consulting firm to supplement its small data science team and help develop the algorithm using the healthcare network’s existing data and de-identified data that is licensed from a large US clinical research partner. What is the most important step for the healthcare network to take when mapping its existing data to the clinical research partner data?
Answer(s): B
Justification:The most crucial step when mapping existing data to the clinical research partner's data is identifying the "fits and gaps" (Option B). Before any other action, the healthcare network must understand the degree of compatibility and discrepancies between its data and the partner's data. This mapping exercise reveals crucial information about data consistency, completeness, and relevance to the cancer detection model.Understanding "fits" allows the network to leverage compatible data points for training the AI model. Identifying "gaps" highlights missing information or inconsistencies that could bias the model or reduce its accuracy. These gaps can arise from variations in data collection methods, patient demographics, or imaging protocols between the healthcare network and the clinical research partner. Addressing these gaps proactively is critical.Failing to identify fits and gaps could lead to several problems. Training the model with incompatible data could result in a biased and inaccurate AI system that misdiagnoses or fails to detect cancer effectively. Furthermore, it can lead to violations of ethical principles and regulatory requirements related to data quality and fairness in AI. Before storing or labeling, the data's usefulness for the intended purpose must be determined. While data security (Option A), country of origin (Option C), and labeling/cleansing (Option D) are important, they are secondary to understanding the fundamental data compatibility. Country of origin may be relevant in other circumstances that have not been presented in the case study. If the data cannot be mapped effectively, the data will not be useful. Therefore, option B is the most important first step.Relevant Links:Data Quality Dimensions: https://www.ibm.com/docs/en/i/7.3?topic=concepts-data-quality-dimensions AI Bias: https://www.brookings.edu/research/how-to-address-ai-bias-in-algorithms/
CASE STUDYPlease use the following to answer the next question: A mid-size US healthcare network has decided to develop an AI solution to detect a type of cancer that is most likely to arise in adults. Specifically, the healthcare network intends to create a recognition algorithm that will perform an initial review of all imaging and then route records to a radiologist for secondary review pursuant to agreed-upon criteria (e.g., a confidence score below a threshold). To date, the healthcare network has: Defined its AI ethical principles. Conducted discovery to identify the intended uses and success criteria for the system. Established an AI risk committee. Assembled a cross-functional team with clear roles and responsibilities. Created policies and procedures to document standards, workflows, timelines and risk thresholds during the project. The healthcare network intends to retain a cloud provider to host the solution. It also intends to retain a large consulting firm to supplement its small data science team and help develop the algorithm using the healthcare network’s existing data and de-identified data that is licensed from a large US clinical research partner. The most significant risk from combining the healthcare network’s existing data with the clinical research partner data is?
Answer(s): A
The most significant risk stemming from combining the healthcare network's data with the clinical research partner's de-identified data is A. Privacy risk.While the clinical research partner data is ostensibly de-identified, the process of combining it with the healthcare network's existing data introduces a substantial risk of re-identification. This risk arises from the potential to link the supposedly anonymous clinical research data points to specific patients within the healthcare network's dataset. Even seemingly harmless attributes, when combined across datasets, can become unique identifiers. This could lead to unauthorized disclosure of protected health information (PHI), violating HIPAA and other privacy regulations.The risk is magnified in a cloud environment if security controls are not adequate. Privacy risk arises when the process of data integration allows data to be identified. Consider the need to match patients in the healthcare networks data set to a similar patient in the clinical research partners data set. De-identification usually requires masking to protect data privacy of the patient. If the model is trained on identifiable information then it is not de-identified. Model outputs are also an issue of privacy since inferences made from the model also can expose information that should be protected.While security, operational, and reputational risks are also present, the direct and immediate threat posed by combining these datasets is the potential to breach patient privacy. Security breaches could certainly exacerbate a privacy incident, and operational failures could lead to data mishandling. However, the initial mixing of datasets creates the possibility of re-identification, making privacy the primary concern. Reputational damage would follow after a privacy breach occurs.Therefore, the potential re-identification of patients is the most significant risk associated with combining the datasets, necessitating rigorous privacy safeguards, de-identification techniques, and data governance policies.Relevant Links:HHS Guidance on HIPAA & De-identification: https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/index.html NIST Guidance on Data De-identification: https://nvlpubs.nist.gov/nistpubs/ir/2015/nist.ir.8053.pdf
When should an ethical impact assessment for AI be performed?
The correct answer is D because ethical considerations should be integrated into the AI lifecycle from the very beginning. Performing an ethical impact assessment (EIA) only at the production stage (A) is too late to address fundamental biases or unintended consequences baked into the system's design or data. Waiting for a user threshold (B) or actual ethical issues (C) means reactive rather than proactive governance, potentially causing harm and reputational damage.An EIA at the initial stages allows for identifying potential ethical risks early, incorporating fairness and transparency principles into the AI system's architecture, and selecting appropriate data sets. This proactive approach aligns with the principles of "responsible AI" and helps ensure the AI aligns with organizational values and societal norms. Continual assessment throughout the lifecycle allows for adapting to evolving contexts, monitoring for unintended biases emerging from new data or usage patterns, and ensuring accountability as the AI system matures. Cloud computing, often used for AI deployment, adds complexity to ethical governance due to shared responsibility models and data residency issues. Regular EIAs become crucial for maintaining oversight in such environments. Continuous monitoring and assessment contribute to a feedback loop, allowing developers to refine the system's ethical performance over time. It's about building ethical AI, not just patching problems after they arise.Authoritative links for further research:European Commission's Ethics Guidelines for Trustworthy AI: https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai OECD Principles on AI: https://www.oecd.org/digital/artificial-intelligence/recommendation-of-the-council-on-artificial-intelligence-C-2019-111.htm NIST AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework
In the machine learning context, feature engineering is the process of:
Feature engineering, in the realm of machine learning, focuses on transforming raw data into a format that is more suitable for a machine learning model. It involves carefully extracting, selecting, and transforming relevant features (attributes, variables) from the initial dataset. This process directly impacts the model's performance, interpretability, and generalization ability.Option A, "Converting raw data into clean data," is more aligned with data cleaning, a preliminary step that addresses issues like missing values, outliers, and inconsistencies. Data cleaning prepares the data for further processing, including feature engineering.Option B, "Creating learning schema for a model to apply," relates to model selection and architecture design. The learning schema defines the model's structure and how it learns from the data, which is separate from feature engineering's focus on data transformation.Option C, "Developing guidelines to train and test a model," involves establishing a framework for model evaluation, including defining metrics, splitting the data into training and testing sets, and setting hyperparameters. This is crucial for model development but distinct from feature engineering.Option D, "Extracting attributes and variables from raw data," accurately captures the core of feature engineering. By carefully choosing and transforming the data's properties, feature engineering aims to create features that effectively represent the underlying patterns and relationships in the data, leading to improved model accuracy. This might involve tasks like creating interaction terms, polynomial features, or using domain knowledge to derive new relevant features. For example, converting a date field into day of the week or month could expose seasonality patterns that a model could then use for prediction. This is why the correct answer is D.For further information, explore these resources:Feature Engineering Techniques: https://www.analyticsvidhya.com/blog/2023/07/complete-guide-for-feature-engineering-techniques/ Feature Selection in Machine Learning: https://www.datacamp.com/tutorial/feature-selection-techniques-in-python
Post your Comments and Discuss IAPP AIGP exam prep with other Community members:
Our website is free, but we have to fight against AI bots and content theft. We're sorry for the inconvenience caused by these security measures. You can access the rest of the AIGP content, but please register or login to continue.