diff --git "a/contextual_compression_ragas_results.csv" "b/contextual_compression_ragas_results.csv" new file mode 100644--- /dev/null +++ "b/contextual_compression_ragas_results.csv" @@ -0,0 +1,464 @@ +question,contexts,answer,ground_truth,faithfulness,answer_relevancy,context_recall,context_precision,answer_correctness +What is the significance of providing notice and explanation as a legal requirement in the context of automated systems?,"['Providing notice has long been a standard practice, and in many cases is a legal requirement, when, for example, making a video recording of someone (outside of a law enforcement or national security context). In some cases, such as credit, lenders are required to provide notice and explanation to consumers. While notice and explanation requirements are already in place in some sectors or situations, the American public deserve to know consistently and across sectors if an automated system is being used in a way that impacts their rights, opportunities, or access. This knowledge should provide confidence in how the public is being treated, and trust in the validity and reasonable use of automated systems. The lack of a timely explanation made it harder to understand and contest the decision. The lack of notice or an explanation makes it harder for those performing child maltreatment assessments to validate the risk assessment and denies parents knowledge that could help them contest a decision.' + 'You should know that an automated system is being used, and understand how and why it contributes to outcomes that impact you. Designers, developers, and deployers of automated systems should provide generally accessible plain language documentation including clear descriptions of the overall system functioning and the role automation plays, notice that such systems are in use, the individual or organization responsible for the system, and explanations of outcomes that are clear, timely, and accessible. Such notice should be kept up-to-date and people impacted by the system should be notified of significant use case or key functionality changes. You should know how and why an outcome impacting you was determined by an automated system, including when the automated system is not the sole input determining the outcome. Automated systems should provide explanations that are technically valid, meaningful and useful to you and to any operators or others who need to understand the system, and calibrated to the level of risk based on the context. Reporting that includes summary information about these automated systems in plain language and assessments of the clarity and quality of the notice and explanations should be made public whenever possible.' + 'An automated system should provide demonstrably clear, timely, understandable, and accessible notice of use, and explanations as to how and why a decision was made or an action was taken by the system. Notices should clearly identify the entity responsible for designing each component of the system and the entity using it. Users should receive notice of the use of automated systems in advance of using or while being impacted by the technology. An explanation should be available with the decision itself, or soon thereafter. Notice should be kept up-to-date and people impacted by the system should be notified of use case or key functionality changes. Notices and explanations should be assessed, such as by research on users’ experiences, including user testing, to ensure that the people using or impacted by the automated system are able to easily.' + 'NOTICE & \nEXPLANATION \nWHY THIS PRINCIPLE IS IMPORTANT\nThis section provides a brief summary of the problems which the principle seeks to address and protect \nagainst, including illustrative examples. \nAutomated systems now determine opportunities, from employment to credit, and directly shape the American \npublic’s experiences, from the courtroom to online classrooms, in ways that profoundly impact people’s lives. But this expansive impact is not always visible. An applicant might not know whether a person rejected their resume or a hiring algorithm moved them to the bottom of the list. A defendant in the courtroom might not know if a judge denying their bail is informed by an automated system that labeled them “high risk.” From correcting errors to contesting decisions, people are often denied the knowledge they need to address the impact of automated systems on their lives. Notice and explanations also serve an important safety and efficacy purpose, allowing experts to verify the reasonableness of a recommendation before enacting it. \nIn order to guard against potential harms, the American public needs to know if an automated system is being used. Clear, brief, and understandable notice is a prerequisite for achieving the other protections in this framework. Like-\nwise, the public is often unable to ascertain how or why an automated system has made a decision or contributed to a particular outcome. The decision-making processes of automated systems tend to be opaque, complex, and, therefore, unaccountable, whether by design or by omission. These factors can make explanations both more challenging and more important, and should not be used as a pretext to avoid explaining important decisions to the people impacted by those choices. In the context of automated systems, clear and valid explanations should be recognized as a baseline requirement.']","Providing notice and explanation as a legal requirement in the context of automated systems is significant for several reasons: + +1. **Transparency**: It ensures that individuals are aware when automated systems are being used to make decisions that affect their rights, opportunities, or access. This transparency helps build trust in the technology and the entities deploying it. + +2. **Empowerment**: When people receive clear explanations about how decisions are made by automated systems, they are better equipped to understand and contest those decisions if necessary. This is particularly important in sensitive areas like employment, credit, and legal proceedings, where outcomes can have profound impacts on individuals' lives. + +3. **Accountability**: Notice and explanation requirements hold organizations accountable for their automated systems. By clearly identifying the entities responsible for the design and use of these systems, it becomes easier to address any issues or biases that may arise. + +4. **Error Correction**: Providing notice allows individuals to identify and correct errors in automated decision-making processes. Without this knowledge, individuals may be left without recourse to challenge incorrect or unfair outcomes. + +5. **Public Confidence**: Consistent and clear communication about the use of automated systems can enhance public confidence in these technologies. When people understand how and why decisions are made, they are more likely to trust the systems and the organizations that use them. + +6. **Safety and Efficacy**: Clear explanations enable experts to verify the reasonableness of recommendations made by automated systems before they are enacted. This is crucial for ensuring that the systems operate safely and effectively. + +In summary, notice and explanation requirements are essential for protecting individuals' rights, fostering accountability, and ensuring that automated systems are used responsibly and ethically (Source: [document name], p. [page number]).","Providing notice and explanation as a legal requirement in the context of automated systems is significant because it allows individuals to understand how automated systems are impacting their lives. It helps in correcting errors, contesting decisions, and verifying the reasonableness of recommendations before enacting them. Clear and valid explanations are essential to ensure transparency, accountability, and trust in the use of automated systems across various sectors.",1.0,0.9713211543186683,1.0,0.999999999975,0.5852599714847575 +"How can structured human feedback exercises, such as GAI red-teaming, be beneficial for GAI risk measurement and management?","['AI Red -teaming \nAI red -teaming is an evolving practice that references exercises often conducted in a controlled \nenvironment and in collaboration with AI developers building AI models to identify potential adverse \nbehavior or outcomes of a GAI model or system, how they could occur, an d stress test safeguards” . AI \nred-teaming can be performed before or after AI models or systems are made available to the broader' + 'GV-4.1-002 Establish policies, procedures, and processes detailing risk measurement in context of use with standardized measurement protocols and structured public feedback exercises such as AI red -teaming or independent external evaluations.' + 'A.1.5. Structured Public Feedback \nStructured public feedback can be used to evaluate whether GAI systems are performing as intended and to calibrate and verify traditional measurement methods. Examples of structured feedback include, but are not limited to:' + '• Human / AI: Performed by GAI in combination with specialist or non-specialist human teams. GAI-led red-teaming can be more cost effective than human red-teamers alone. Human or GAI-led AI red-teaming may be better suited for eliciting different types of harms. \nProvenance data tracking and synthetic content detection can help facilitate greater information access about both authentic and synthetic content to users, enabling better knowledge of trustworthiness in AI systems. When combined with other organizational accountability mechanisms, digital content transparency approaches can enable processes to trace negative outcomes back to their source, improve information integrity, and uphold public trust. Provenance data tracking and synthetic content detection mechanisms provide information about the origin and history of content to assist in GAI risk management efforts.']","Structured human feedback exercises, such as GAI red-teaming, can significantly benefit GAI risk measurement and management in several ways: + +1. **Identifying Vulnerabilities**: Red-teaming allows teams to simulate potential adversarial attacks or adverse behaviors in a controlled environment. This helps identify vulnerabilities in the GAI system that may not be apparent during standard testing. + +2. **Stress Testing Safeguards**: By putting the GAI system through various scenarios, red-teaming can evaluate the effectiveness of existing safeguards. This process helps ensure that the measures in place are robust enough to handle unexpected situations or malicious attempts to exploit the system. + +3. **Diverse Perspectives**: Involving both human and GAI participants in red-teaming exercises can provide a broader range of insights. Human teams can bring unique perspectives and experiences that may highlight different types of risks or harms that GAI alone might not identify. + +4. **Cost-Effectiveness**: GAI-led red-teaming can be more cost-effective than relying solely on human red-teamers. This approach can optimize resources while still providing valuable insights into the system's performance and potential risks. + +5. **Calibration of Measurement Methods**: Structured feedback from red-teaming can help calibrate and verify traditional measurement methods used to assess GAI systems. This ensures that the evaluation processes are aligned with real-world performance and risks. + +6. **Enhancing Public Trust**: By demonstrating a commitment to rigorous testing and risk management through red-teaming, organizations can enhance public trust in their GAI systems. Transparency in how risks are identified and managed can reassure stakeholders about the safety and reliability of the technology. + +7. **Provenance and Transparency**: Incorporating mechanisms like provenance data tracking and synthetic content detection during red-teaming can improve information integrity. This allows organizations to trace negative outcomes back to their source, facilitating better risk management and accountability. + +In summary, structured human feedback exercises like GAI red-teaming are crucial for effectively measuring and managing risks associated with GAI systems, ultimately leading to safer and more trustworthy AI applications (Source: GV-4.1-002, p. [specific page number]).","Structured human feedback exercises, such as GAI red-teaming, can be beneficial for GAI risk measurement and management by defining use cases, contexts of use, capabilities, and negative impacts where such exercises would be most beneficial. These exercises help in monitoring and improving outputs, evaluating the quality and integrity of data used in training, and tracking and documenting risks or opportunities related to GAI risks that cannot be measured quantitatively. Additionally, seeking active and direct feedback from affected communities through red-teaming can enhance information integrity and help in identifying harmful bias and homogenization in AI systems.",1.0,0.98830909998763,1.0,0.999999999975,0.32050120616847055 +How do measurement gaps between laboratory and real-world settings impact the assessment of GAI systems in the context of pre-deployment testing?,"['Currently available pre -deployment TEVV processes used for GAI applications may be inadequate, non- systematically applied, or fail to reflect or mismatched to deployment contexts. For example, the anecdotal testing of GAI system capabilities through video games or standardized tests designed for humans (e.g., intelligence tests, professional licensing exams) does not guarantee GAI system validity or reliability in those domains. Similarly, jailbreaking or prompt engineering tests may not systematically assess validity or reliability risks. \nMeasurement gaps can arise from mismatches between laboratory and real -world settings. Current testing approaches often remain focused on laboratory conditions or restricted to benchmark test datasets and in silico techniques that may not extrapolate well to —or directly assess GAI impacts in real -world conditions. For example, current measurement gaps for GAI make it difficult to precisely estimate its potential ecosystem -level or longitudinal risks and related political, social, and economic impacts. Gaps between benchmarks and real-world use of GAI systems may likely be exacerbated due to prompt sensitivity and broad heterogeneity of contexts of use.']","Measurement gaps between laboratory and real-world settings can significantly affect the assessment of GAI systems during pre-deployment testing in several ways: + +1. **Limited Contextual Relevance**: Laboratory tests often focus on controlled environments that may not accurately reflect the complexities and variabilities of real-world situations. This can lead to a mismatch between how a GAI system performs in a lab versus in actual deployment contexts, making it difficult to predict its effectiveness and reliability in practice. + +2. **Overfitting to Benchmarks**: Many current testing approaches rely heavily on benchmark datasets and standardized tests. While these benchmarks can provide useful insights, they may not capture the full range of scenarios the GAI system will encounter once deployed. This can result in systems that perform well in tests but fail to generalize to real-world applications. + +3. **Prompt Sensitivity**: GAI systems can be highly sensitive to the specific prompts or inputs they receive. In a lab setting, the prompts may be carefully curated, but in real-world use, the inputs can vary widely. This variability can lead to unpredictable outcomes that were not anticipated during testing. + +4. **Ecosystem-Level Risks**: Current measurement approaches may not adequately assess the broader ecosystem-level or longitudinal risks associated with GAI systems. This includes potential political, social, and economic impacts that could arise from their deployment, which are often difficult to measure in a lab setting. + +5. **Inadequate Validation**: If pre-deployment testing does not account for the diverse contexts in which GAI systems will be used, it may fail to identify critical validity and reliability risks. This can lead to the deployment of systems that are not robust enough to handle real-world challenges. + +6. **Feedback Loops**: Real-world deployment can create feedback loops that are not present in laboratory settings. For example, user interactions and societal responses can influence how a GAI system behaves over time, which may not be captured during initial testing phases. + +In summary, these measurement gaps can lead to significant challenges in ensuring that GAI systems are safe, effective, and reliable when deployed in real-world environments (Source: [document name], p. [specific page number]).","Measurement gaps between laboratory and real-world settings can impact the assessment of GAI systems in the context of pre-deployment testing by limiting the extrapolation of results from laboratory conditions to real-world scenarios. Current testing approaches often focus on benchmark test datasets and in silico techniques that may not accurately assess the impacts of GAI systems in real-world conditions. This can make it difficult to estimate the ecosystem-level or longitudinal risks associated with GAI deployment, as well as the political, social, and economic impacts. Additionally, the prompt sensitivity and broad heterogeneity of real-world contexts of use can exacerbate the gaps between benchmarks and actual GAI system performance.",0.9583333333333334,0.9965951763678161,1.0,0.9999999999,0.5972506683410433 +How should data collection and use-case scope limits be determined and implemented in automated systems to prevent 'mission creep'?,"['Data collection should be limited in scope, with specific, narrow identified goals, to avoid ""mission creep."" Anticipated data collection should be determined to be strictly necessary to the identified goals and should be minimized as much as possible. Data collected based on these identified goals and for a specific context should not be used in a different context without assessing for new privacy risks and implementing appropriate mitigation measures, which may include express consent. Clear timelines for data retention should be established, with data deleted as soon as possible in accordance with legal or policy-based limitations. Determined data retention timelines should be documented and justified.' + 'Limited and proportionate surveillance. Surveillance should be avoided unless it is strictly necessary to achieve a legitimate purpose and it is proportionate to the need. Designers, developers, and deployers of surveillance systems should use the least invasive means of monitoring available and restrict monitoring to the minimum number of subjects possible. To the greatest extent possible consistent with law enforcement and national security needs, individuals subject to monitoring should be provided with clear and specific notice before it occurs and be informed about how the data gathered through surveillance will be used. \nScope limits on surveillance to protect rights and democratic values. Civil liberties and civil' + '- Relevant and high-quality data. Data used as part of any automated system’s creation, evaluation, or deployment should be relevant, of high quality, and tailored to the task at hand. Relevancy should be established based on research-backed demonstration of the causal influence of the data to the specific use case or justified more generally based on a reasonable expectation of usefulness in the domain and/or for the system design or ongoing development. Relevance of data should not be established solely by appealing to its historical connection to the outcome. High quality and tailored data should be representative of the task at hand and errors from data entry or other sources should be measured and limited. Any data used as the target of a prediction process should receive particular attention to the quality and validity of the predicted outcome.']","To prevent ""mission creep"" in automated systems, data collection and use-case scope limits should be determined and implemented through a structured approach that includes the following steps: + +1. **Define Clear Objectives**: Start by establishing specific, narrow goals for the automated system. This involves identifying the precise purpose of data collection and ensuring that it aligns with the intended use of the system. Clear objectives help prevent the system from being used for unintended purposes. + +2. **Conduct a Privacy Impact Assessment**: Before data collection begins, conduct a thorough assessment to evaluate potential privacy risks associated with the data being collected. This assessment should consider how the data will be used, who will have access to it, and the potential impact on individuals' privacy rights. + +3. **Limit Data Collection**: Collect only the data that is strictly necessary to achieve the defined objectives. Avoid gathering excessive or unrelated data that could lead to mission creep. This principle of data minimization is crucial for protecting individual privacy and ensuring compliance with legal standards. + +4. **Establish Use-Case Boundaries**: Clearly define the contexts in which the collected data can be used. This includes specifying the types of analyses that can be performed and the decisions that can be made based on the data. Any use outside these boundaries should require a new assessment of privacy risks and potential impacts. + +5. **Implement Data Retention Policies**: Set clear timelines for how long data will be retained. Data should be deleted as soon as it is no longer necessary for the identified goals, in accordance with legal or policy-based limitations. Documenting and justifying these retention timelines is essential for accountability. + +6. **Regularly Review and Update Policies**: Periodically review data collection practices and use-case boundaries to ensure they remain relevant and effective. This includes assessing whether the original objectives are still valid and whether any changes in technology or regulations necessitate adjustments. + +7. **Engage Stakeholders**: Involve relevant stakeholders, including legal, compliance, and privacy experts, in the development and implementation of data collection policies. Their insights can help ensure that the policies are comprehensive and aligned with best practices. + +8. **Provide Transparency and Notice**: Inform individuals about what data is being collected, how it will be used, and their rights regarding their data. Transparency helps build trust and allows individuals to make informed decisions about their participation. + +By following these steps, organizations can effectively determine and implement data collection and use-case scope limits in automated systems, thereby minimizing the risk of mission creep and protecting individual privacy rights (Source: [document name], p. [specific page number]).","Data collection and use-case scope limits in automated systems should be determined by setting specific, narrow goals to avoid 'mission creep.' Anticipated data collection should be strictly necessary for the identified goals and minimized as much as possible. Data collected for a specific context should not be used in a different context without assessing new privacy risks and implementing appropriate mitigation measures, which may include obtaining express consent. Clear timelines for data retention should be established, with data deleted as soon as possible in accordance with legal or policy-based limitations. The determined data retention timelines should be documented and justified.",0.43902439024390244,0.9223762283128042,1.0,0.9999999999666667,0.5516061624221906 +What action did the Federal Trade Commission take against Kochava regarding the sale of sensitive location tracking data?,[],"The Federal Trade Commission (FTC) took action against Kochava, a data broker, for allegedly selling sensitive location tracking data that could be used to identify individuals' movements and habits. The FTC's complaint highlighted that Kochava's practices posed significant privacy risks, as the data could reveal sensitive information about individuals, including visits to places like medical facilities, places of worship, and domestic violence shelters. + +The FTC's action aimed to protect consumer privacy by addressing the sale of this type of data without adequate safeguards. The agency emphasized that such practices could lead to harmful consequences for individuals, particularly those in vulnerable situations. The FTC's move reflects a broader trend of increasing scrutiny on data brokers and their handling of personal information, especially in light of growing concerns about privacy and data security in the digital age (Source: FTC press release, p. [specific page number]). + +If you have any more questions about AI regulations or related topics, feel free to ask!","FTC sued Kochava for selling data that tracks people at reproductive health clinics, places of worship, and other sensitive locations.",0.8333333333333334,0.9250718639749307,0.0,0.0,0.5296801698864523 +How should explanatory mechanisms be built into system design to ensure full behavior transparency in high-risk settings?,"['explanatory mechanisms should be built into the system design so that the system’s full behavior can be explained in advance (i.e., only fully transparent models should be used), rather than as an after-the-decision interpretation. In other settings, the extent of explanation provided should be tailored to the risk level.' + 'GV-4.1-001 Establish policies and procedures that address continual improvement processes for GAI risk measurement. Address general risks associated with a lack of explainability and transparency in GAI systems by using ample documentation and techniques such as: application of gradient -based attributions, occlusion/term reduction, counterfactual prompts and prompt engineering, and analysis of embeddings; Assess and update risk measurement approaches at regular cadences.' + 'Meaningful access to examine the system. Designers, developers, and deployers of automated systems should consider limited waivers of confidentiality (including those related to trade secrets) where necessary in order to provide meaningful oversight of systems used in sensitive domains, incorporating measures to protect intellectual property and trade secrets from unwarranted disclosure as appropriate. This includes (potentially private and protected) meaningful access to source code, documentation, and related data during any associated legal discovery, subject to effective confidentiality or court orders. Such meaningful access should include (but is not limited to) adhering to the principle on Notice and Explanation using the highest level of risk so the system is designed with built-in explanations; such systems should use fully-transparent models where the model itself can be understood by people needing to directly examine it.']","To guarantee complete behavior transparency in high-risk environments, explanatory mechanisms should be integrated into system design through the following strategies: + +1. **Use Fully Transparent Models**: Design AI systems using models that are inherently interpretable. This means selecting algorithms that allow users to understand how decisions are made without needing complex post-hoc explanations. For example, decision trees or linear models can be more transparent than deep learning models. + +2. **Built-in Explanation Features**: Incorporate features that provide real-time explanations of the system's behavior. This could include visualizations of decision pathways, confidence scores for predictions, or summaries of the factors influencing a decision. These features should be accessible to users and stakeholders. + +3. **Documentation and Training**: Provide comprehensive documentation that outlines how the system works, including its limitations and the rationale behind its design choices. Additionally, training for users on how to interpret the explanations and understand the system's behavior is crucial. + +4. **Regular Updates and Assessments**: Establish processes for regularly assessing and updating the explanatory mechanisms as the system evolves. This includes revisiting the explanations provided to ensure they remain relevant and accurate as the model is refined or as new data is introduced. + +5. **User-Centric Design**: Engage with end-users during the design process to understand their needs and expectations regarding explanations. Tailoring the level of detail and type of explanations to the users' expertise and context can enhance understanding and trust. + +6. **Feedback Loops**: Implement mechanisms for users to provide feedback on the explanations they receive. This feedback can be used to improve the clarity and usefulness of the explanations over time. + +7. **Risk-Based Tailoring**: Adjust the extent and type of explanations based on the risk level associated with the system's decisions. For high-risk applications, more detailed and comprehensive explanations should be provided, while lower-risk applications may require simpler explanations. + +8. **Meaningful Access for Oversight**: Ensure that designers, developers, and deployers of the system provide meaningful access to the underlying data, algorithms, and decision-making processes. This may involve limited waivers of confidentiality to allow for oversight while protecting sensitive information. + +By integrating these strategies into the system design, organizations can enhance transparency and accountability in high-risk environments, ultimately fostering trust in AI technologies (Source: GV-4.1-001, p. [specific page number]). If you have more questions or need further clarification, feel free to ask!","In settings where the consequences are high as determined by a risk assessment, or extensive oversight is expected (e.g., in criminal justice or some public sector settings), explanatory mechanisms should be built into the system design so that the system’s full behavior can be explained in advance (i.e., only fully transparent models should be used), rather than as an after-the-decision interpretation.",0.6666666666666666,0.9361264901628866,1.0,0.9999999999666667,0.35389204039442157 +What are some examples of GAI risks that organizations need to consider in the development and deployment of AI systems?,"['GAI risks can vary along many dimensions: \n• Stage of the AI lifecycle: Risks can arise during design, development, deployment, operation, and/or decommissioning. \n�� Scope: Risks may exist at individual model or system levels, at the application or implementation levels (i.e., for a specific use case), or at the ecosystem level – that is, beyond a single system or organizational context. Examples of the latter include the expansion of “algorithmic monocultures,” resulting from repeated use of the same model, or impacts on access to opportunity, labor markets, and the creative economies. \n• Source of risk: Risks may emerge from factors related to the design, training, or operation of the GAI model itself, stemming in some cases from GAI model or system inputs, and in other cases, from GAI system outputs. Many GAI risks, however, originate from human behavior.' + 'Organizations can restrict AI applications that cause harm, exceed stated risk tolerances, or that conflict with their tolerances or values. Governance tools and protocols that are applied to other types of AI systems can be applied to GAI systems. These p lans and actions include: \n• Accessibility and reasonable accommodations \n• AI actor credentials and qualifications \n• Alignment to organizational values \n• Auditing and assessment \n• Change -management controls \n• Commercial use \n• Data provenance' + 'For instance , prompt injection involves modifying what input is provided to a GAI system so that it \nbehave s in unintended ways. In direct prompt injections, attackers might craft malicious prompts and \ninput them directly to a GAI system , with a variety of downstream negative consequences to \ninterconnected systems. Indirect prompt injection attacks occur when adversaries remotely (i.e., without \na direct interface) exploit LLM -integrated applications by injecting prompts into data likely to be \nretrieved. Security researchers have already demonstrated how indirect prompt injections can exploit \nvulnerabilities by steal ing proprietary data or running malicious code remotely on a machine. \nAnother cybersecurity risk to GAI is data poisoning , in which an adversary compromises a training \ndataset used by a model to manipulate its output s or operation. Malicious tampering with data or parts \nof the model could exacerbate risks associated with GAI system outputs. \nIntellectual property risks from GAI systems may arise where the use of copyrighted works is not a fair \nuse under the fair use doctrine. If a GAI system’s training data included copyrighted material, GAI \noutputs displaying instances of training data memorization (see Data Privacy above) could infringe on \ncopyright.' + 'A.1. Governance \nA.1.1. Overview \nLike any other technology system, governance principles and techniques can be used to manage risks related to generative AI models, capabilities, and applications. Organizations may choose to apply their existing risk tiering to GAI systems, or they may opt to revise or update AI system risk levels to address these unique GAI risks. This section describes how organizational governance regimes may be re-evaluated and adjusted for GAI contexts. It also addresses third-party considerations for governing across the AI value chain. \nA.1.2. Organizational Governance \nGAI opportunities, risks and long-term performance characteristics are typically less well-understood than non-generative AI tools and may be perceived and acted upon by humans in ways that vary greatly. Accordingly, GAI may call for different levels of oversight from AI Actors or different human-AI configurations in order to manage their risks effectively. Organizations’ use of GAI systems may also warrant additional human review, tracking and documentation, and greater management oversight.']","Organizations need to consider several risks associated with Generative AI (GAI) during the development and deployment of AI systems. Here are some key examples: + +1. **Data Privacy Risks**: GAI systems often require large datasets for training, which may include sensitive personal information. If not handled properly, this can lead to breaches of privacy and violations of data protection regulations (e.g., GDPR). + +2. **Bias and Discrimination**: GAI models can inadvertently learn and perpetuate biases present in the training data. This can result in outputs that are discriminatory or unfair, impacting marginalized groups and leading to reputational damage for organizations. + +3. **Security Vulnerabilities**: GAI systems can be susceptible to various cybersecurity threats, such as prompt injection attacks, where malicious inputs are crafted to manipulate the system's behavior. This can lead to unintended consequences and exploitation of the system (Source: [document name], p. [specific page number]). + +4. **Intellectual Property Issues**: The use of copyrighted materials in training datasets can raise legal concerns. If a GAI system generates outputs that closely resemble copyrighted works, it may infringe on intellectual property rights (Source: [document name], p. [specific page number]). + +5. **Misinformation and Manipulation**: GAI can be used to generate misleading or false information, which can be exploited for malicious purposes, such as creating deepfakes or spreading disinformation. This poses risks to public trust and safety. + +6. **Operational Risks**: The deployment of GAI systems can lead to operational challenges, such as unexpected system failures or performance issues in real-world scenarios that were not anticipated during testing (Source: [document name], p. [specific page number]). + +7. **Ethical Implications**: The use of GAI raises ethical questions about accountability, transparency, and the potential for misuse. Organizations must navigate these ethical dilemmas to ensure responsible AI use. + +8. **Ecosystem-Level Risks**: GAI systems can contribute to broader societal issues, such as algorithmic monocultures, where reliance on similar models leads to reduced diversity in decision-making processes and potential negative impacts on labor markets and creative economies (Source: [document name], p. [specific page number]). + +By being aware of these risks, organizations can implement appropriate governance frameworks and risk management strategies to mitigate potential negative impacts associated with GAI systems. If you have further questions or need more details, feel free to ask!","Organizations need to consider various GAI risks in the development and deployment of AI systems, including unacceptable use identified by stakeholder communities, harmful bias and homogenization, dangerous, violent, or hateful content, immature safety or risk cultures related to AI and GAI design, development, and deployment, public information integrity risks impacting democratic processes, unknown long-term performance characteristics of GAI, and risks related to generating illegal content or violating laws.",0.8918918918918919,0.9493978201836971,0.25,0.8055555555287036,0.47170020092246534 +How should the validity of explanations provided by automated systems be ensured?,"['Tailored to the level of risk. An assessment should be done to determine the level of risk of the automated system. In settings where the consequences are high as determined by a risk assessment, or extensive oversight is expected (e.g., in criminal justice or some public sector settings), explanatory mechanisms should be built into the system design so that the system’s full behavior can be explained in advance (i.e., only fully transparent models should be used), rather than as an after-the-decision interpretation. In other settings, the extent of explanation provided should be tailored to the risk level. Valid. The explanation provided by a system should accurately reflect the factors and the influences that led to a particular decision, and should be meaningful for the particular customization based on purpose, target, and level of risk. While approximation and simplification may be necessary for the system to succeed based on the explanatory purpose and target of the explanation, or to account for the risk of fraud or other concerns related to revealing decision-making information, such simplifications should be done in a scientifically supportable way. Where appropriate based on the explanatory system, error ranges for the explanation should be calculated and included in the explanation, with the choice of presentation of such information balanced with usability and overall interface complexity concerns.' + ""Maintained. The human consideration and fallback process and any associated automated processes should be maintained and supported as long as the relevant automated system continues to be in use. Institute training, assessment, and oversight to combat automation bias and ensure any human-based components of a system are effective. Training and assessment. Anyone administering, interacting with, or interpreting the outputs of an automated system should receive training in that system, including how to properly interpret outputs of a system in light of its intended purpose and in how to mitigate the effects of automation bias. The training should reoccur regularly to ensure it is up to date with the system and to ensure the system is used appropriately. Assessment should be ongoing to ensure that the use of the system with human involvement provides for appropriate results, i.e., that the involvement of people does not invalidate the system's assessment as safe and effective or lead to algorithmic discrimination. Oversight. Human-based systems have the potential for bias, including automation bias, as well as other concerns that may limit their effectiveness. The results of assessments of the efficacy and potential bias of such human-based systems should be overseen by governance structures that have the potential to update the operation of the human-based system in order to mitigate these effects."" + 'While notice and explanation requirements are already in place in some sectors or situations, the American public deserve to know consistently and across sectors if an automated system is being used in a way that impacts their rights, opportunities, or access. This knowledge should provide confidence in how the public is being treated, and trust in the validity and reasonable use of automated systems. The lack of a timely explanation made it harder to understand and contest the decision. The lack of notice or an explanation makes it harder for those performing child maltreatment assessments to validate the risk assessment and denies parents knowledge that could help them contest a decision.' + 'Ongoing monitoring. Automated systems should have ongoing monitoring procedures, including recalibra - tion procedures, in place to ensure that their performance does not fall below an acceptable level over time, based on changing real-world conditions or deployment contexts, post-deployment modification, or unexpect - ed conditions. This ongoing monitoring should include continuous evaluation of performance metrics and harm assessments, updates of any systems, and retraining of any machine learning models as necessary, as well as ensuring that fallback mechanisms are in place to allow reversion to a previously working system. Monitor - ing should take into account the performance of both technical system components (the algorithm as well as any hardware components, data inputs, etc.) and human operators. It should include mechanisms for testing the actual accuracy of any predictions or recommendations generated by a system, not just a human operator’s determination of their accuracy. Ongoing monitoring procedures should include manual, human-led monitor - ing as a check in the event there are shortcomings in automated monitoring systems. These monitoring proce - dures should be in place for the lifespan of the deployed automated system.']","To ensure the validity of explanations provided by automated systems, organizations should implement several key practices: + +1. **Rigorous Testing and Validation**: Conduct thorough testing of the automated system to ensure that the explanations it provides are accurate and reflect the underlying decision-making processes. This includes validating the model against known outcomes and ensuring that the explanations align with the system's behavior. + +2. **Clear Documentation**: Maintain comprehensive documentation that outlines how the system generates explanations. This should include details about the algorithms used, the data inputs, and the rationale behind the decision-making process. Clear documentation helps stakeholders understand the basis for the explanations. + +3. **User-Centric Design**: Design explanations with the end-user in mind. This means tailoring the complexity and detail of the explanations to the users' expertise and needs. Engaging users during the design process can help ensure that the explanations are meaningful and useful. + +4. **Regular Updates and Reassessments**: Continuously monitor and update the system and its explanations as new data becomes available or as the system evolves. Regular reassessments help ensure that the explanations remain valid and relevant over time. + +5. **Incorporate Feedback Mechanisms**: Implement mechanisms for users to provide feedback on the explanations they receive. This feedback can be used to refine and improve the quality of the explanations, ensuring they meet user needs and expectations. + +6. **Error Analysis**: Conduct error analysis to identify instances where the system's explanations may have been misleading or incorrect. Understanding the causes of these errors can inform improvements in the explanation generation process. + +7. **Transparency in Limitations**: Clearly communicate the limitations of the explanations provided by the system. Users should be made aware of any uncertainties or potential inaccuracies in the explanations, which can help manage expectations and foster trust. + +8. **Independent Audits**: Consider engaging third-party auditors to evaluate the validity of the explanations provided by the automated system. Independent assessments can provide an objective perspective on the system's performance and the quality of its explanations. + +By implementing these practices, organizations can enhance the validity of the explanations provided by automated systems, thereby improving transparency and trust in AI technologies (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!","The explanation provided by a system should accurately reflect the factors and influences that led to a particular decision, and should be meaningful for the particular customization based on purpose, target, and level of risk. While approximation and simplification may be necessary for the system to succeed based on the explanatory purpose and target of the explanation, or to account for the risk of fraud or other concerns related to revealing decision-making information, such simplifications should be done in a scientifically supportable way. Where appropriate based on the explanatory system, error ranges for the explanation should be calculated and included in the explanation, with the choice of presentation of such information balanced with usability and overall interface complexity concerns.",1.0,0.9626130651354985,1.0,0.999999999975,0.4702229048074685 +How do generative models like LLMs generate outputs that can lead to confabulations in GAI systems?,"['“Confabulation” refers to a phenomenon in which GAI systems generate and confidently present erroneous or false content in response to prompts. Confabulations also include generated outputs that diverge from the prompts or other input or that contradict previously generated statements in the same context. Th ese phenomena are colloquially also referred to as “hallucination s” or “fabrication s.” Confabulations can occur across GAI outputs and contexts. Confabulations are a natural result of the way generative models are designed: they generate outputs that approximate the statistical distribution of their training data; for example, LLMs predict the next token or word in a sentence or phrase. While such statistical prediction can produce factually accurate and consistent outputs, it can also produce outputs that are factually inaccurate or internally inconsistent. This dynamic is particularly relevant when it comes to open-ended prompts for long-form responses and in domains which require highly contextual and/or domain expertise. Risks from confabulations may arise when users believe false content – often due to the confident nature of the response – leading users to act upon or promote the false information. This poses a challenge for many real-world applications, such as in healthcare, where a confabulated summary of patient information reports could cause doctors to make incorrect diagnoses and/or recommend the wrong treatments. Risks of confabulated content may be especially important to monitor when integrating GAI into applications involving consequential decision making. GAI outputs may also include confabulated logic or citations that purport to justify or explain the system’s answer, which may further mislead humans into inappropriately trusting the system’s output.']","Generative models like Large Language Models (LLMs) produce outputs based on statistical patterns learned from vast amounts of training data. However, this process can lead to confabulations, which are instances where the model generates outputs that are factually incorrect or inconsistent. Here’s how this happens: + +1. **Statistical Prediction**: LLMs generate text by predicting the next word or token in a sequence based on the context provided. This prediction is based on the statistical distribution of words and phrases in the training data. While this can lead to coherent and contextually relevant outputs, it can also result in the generation of incorrect or nonsensical information if the model encounters unfamiliar contexts or prompts. + +2. **Lack of Understanding**: LLMs do not possess true understanding or knowledge; they operate based on patterns rather than comprehension. As a result, they may produce outputs that sound plausible but are factually inaccurate or misleading. This is particularly problematic in complex or specialized domains where precise information is critical. + +3. **Open-Ended Prompts**: When given open-ended prompts, LLMs may generate long-form responses that can diverge from the original question or context. This can lead to inconsistencies within the same response or between different responses, as the model attempts to fill in gaps based on its training data rather than adhering strictly to factual accuracy. + +4. **Confident Outputs**: LLMs often present their outputs with a high degree of confidence, which can mislead users into believing that the information is accurate. This confidence can be particularly dangerous in applications where users may act on the information provided, such as in healthcare or legal contexts. + +5. **Absence of Real-Time Verification**: LLMs do not have the capability to verify facts in real-time or access external databases to confirm the accuracy of their outputs. This limitation means that they can generate information that is outdated or incorrect without any mechanism for correction. + +6. **Contextual Sensitivity**: The performance of LLMs can vary significantly based on the specific context or phrasing of the prompt. If the context is ambiguous or poorly defined, the model may generate outputs that are irrelevant or incorrect, leading to confabulations. + +In summary, the combination of statistical prediction, lack of true understanding, and the nature of open-ended prompts contributes to the occurrence of confabulations in GAI systems. This highlights the importance of careful oversight and validation when deploying LLMs in real-world applications (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!","Generative models like LLMs generate outputs that can lead to confabulations in GAI systems by approximating the statistical distribution of their training data. While this statistical prediction can result in factually accurate and consistent outputs, it can also produce outputs that are factually inaccurate or internally inconsistent. This becomes particularly relevant in open-ended prompts for long-form responses and domains requiring contextual or domain expertise.",1.0,0.9397842420418132,1.0,0.9999999999,0.34463250823091496 +How can appropriate diligence on training data use help assess intellectual property risks in AI systems?,"['27 MP-4.1-0 10 Conduct appropriate diligence on training data use to assess intellectual property, and privacy, risks, including to examine whether use of proprietary or sensitive training data is consistent with applicable laws.' + 'Carlini, N., et al. (2021) Extracting Training Data from Large Language Models. Usenix . https://www.usenix.org/conference/usenixsecurity21/presentation/carlini -extracting \nCarlini, N. et al. (2023) Quantifying Memorization Across Neural Language Models. ICLR 2023.' + 'Intellectual property risks from GAI systems may arise where the use of copyrighted works is not a fair use under the fair use doctrine. If a GAI system’s training data included copyrighted material, GAI outputs displaying instances of training data memorization could infringe on copyright.']","Appropriate diligence on training data use is crucial for assessing intellectual property (IP) risks in AI systems. Here are several ways this diligence can help: + +1. **Identifying Data Sources**: Conducting thorough diligence involves tracing the origins of the training data used in AI systems. By identifying whether the data comes from proprietary, copyrighted, or open-source sources, organizations can evaluate the potential IP risks associated with its use. + +2. **Evaluating Fair Use**: Understanding the legal framework surrounding fair use is essential. Diligence helps organizations assess whether the use of copyrighted materials in training datasets qualifies as fair use under copyright law. This involves analyzing the purpose, nature, amount, and effect of the use on the market for the original work. + +3. **Assessing Licensing Agreements**: Organizations should review any licensing agreements related to the training data. Diligence ensures that the terms of these agreements are understood and followed, which can help mitigate the risk of infringing on IP rights. + +4. **Monitoring Data Composition**: By examining the composition of training datasets, organizations can identify any proprietary or sensitive data that may pose IP risks. This includes ensuring that the data does not contain trade secrets or other confidential information that could lead to legal challenges. + +5. **Implementing Data Minimization**: Diligence encourages the principle of data minimization, where organizations collect only the data necessary for their AI systems. This reduces the likelihood of inadvertently using copyrighted or proprietary materials, thereby lowering IP risks. + +6. **Documenting Data Use**: Keeping detailed records of how training data is sourced, used, and processed can provide a clear audit trail. This documentation is valuable in demonstrating compliance with IP laws and can be crucial in defending against potential claims. + +7. **Engaging Legal Expertise**: Involving legal experts during the diligence process can help organizations navigate complex IP issues. Legal counsel can provide guidance on compliance with IP laws and help assess the risks associated with specific datasets. + +8. **Regular Reviews and Updates**: Conducting regular reviews of training data practices ensures that organizations stay informed about changes in IP law and emerging risks. This proactive approach can help mitigate potential legal challenges related to IP infringement. + +By implementing these diligence practices, organizations can better assess and manage the intellectual property risks associated with the training data used in their AI systems, ultimately leading to more responsible and compliant AI development (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!","Appropriate diligence on training data use can help assess intellectual property risks in AI systems by examining whether the use of proprietary or sensitive training data aligns with relevant laws. This includes evaluating the likelihood and magnitude of potential impacts, both beneficial and harmful, based on past uses of AI systems in similar contexts, public incident reports, feedback from external parties, and other relevant data. By identifying and documenting these impacts, organizations can better understand the risks associated with their training data and take appropriate measures to mitigate them.",1.0,0.981217057423747,0.3333333333333333,0.9999999999666667,0.46364472289220054 +How do integrated human-AI systems benefit companies in providing customer service?,[],"Integrated human-AI systems can significantly enhance customer service for companies in several ways: + +1. **24/7 Availability**: AI systems, such as chatbots, can provide round-the-clock support, ensuring that customers can receive assistance at any time, even outside of regular business hours. This increases customer satisfaction by providing immediate responses to inquiries. + +2. **Efficiency and Speed**: AI can handle routine queries and tasks quickly, allowing human agents to focus on more complex issues. This leads to faster resolution times and improved overall efficiency in customer service operations. + +3. **Personalization**: AI systems can analyze customer data and interactions to provide personalized recommendations and responses. This tailored approach enhances the customer experience by making interactions more relevant and engaging. + +4. **Scalability**: Integrated human-AI systems can easily scale to handle varying volumes of customer inquiries. During peak times, AI can manage the increased load, ensuring that customers do not experience long wait times. + +5. **Data Insights**: AI can analyze customer interactions and feedback to identify trends and areas for improvement. This data-driven approach allows companies to refine their customer service strategies and enhance service quality over time. + +6. **Consistency**: AI systems provide consistent responses to customer inquiries, reducing the variability that can occur with human agents. This consistency helps build trust and reliability in the customer service experience. + +7. **Cost Savings**: By automating routine tasks and inquiries, companies can reduce operational costs associated with customer service. This allows for better allocation of resources and potentially lower prices for customers. + +8. **Enhanced Human Support**: AI can assist human agents by providing them with relevant information and suggested responses during customer interactions. This support enables agents to resolve issues more effectively and improves their overall performance. + +9. **Multichannel Support**: Integrated systems can provide support across various channels, such as social media, email, and live chat. This omnichannel approach ensures that customers can reach out through their preferred method, enhancing their experience. + +10. **Continuous Learning**: AI systems can learn from interactions over time, improving their responses and capabilities. This continuous learning process helps ensure that the system evolves to meet changing customer needs and preferences. + +By leveraging integrated human-AI systems, companies can create a more efficient, responsive, and personalized customer service experience, ultimately leading to higher customer satisfaction and loyalty. If you have more questions or need further clarification, feel free to ask!","Integrated human-AI systems benefit companies in providing customer service by allowing for faster customer care while maintaining human agents to handle complicated requests. These systems use partially automated platforms to answer common customer questions and compile issues for human agents to review, ensuring a balance between efficiency and personalized service.",0.0,0.9832089771566145,0.0,0.0,0.37183756529281276 +What was the purpose of the year of public engagement that informed the development of the Blueprint for an AI Bill of Rights?,"['""people throughout the United States, public servants across Federal agencies, and members of the international community spoke up about both the promises and potential harms of these technologies, and played a central role in shaping the Blueprint for an AI Bill of Rights.""' + 'Its release follows a year of public engagement to inform this initiative.' + 'APPENDIX\n• OSTP conducted meetings with a variety of stakeholders in the private sector and civil society. Some of these meetings were specifically focused on providing ideas related to the development of the Blueprint for an AI Bill of Rights while others provided useful general context on the positive use cases, potential harms, and/or oversight possibilities for these technologies.']","The year of public engagement that informed the development of the Blueprint for an AI Bill of Rights aimed to gather insights, perspectives, and feedback from a diverse range of stakeholders, including individuals, public servants across federal agencies, and members of the international community. The purpose of this engagement was to: + +1. **Identify Promises and Harms**: Understand both the potential benefits and risks associated with AI technologies. This involved discussing positive use cases as well as potential harms that could arise from the deployment of AI systems. + +2. **Shape Policy Frameworks**: Collect ideas and suggestions that could help shape effective policies and guidelines for the responsible use of AI. The engagement sought to ensure that the resulting framework would reflect the values and concerns of various communities. + +3. **Enhance Transparency and Accountability**: Foster a dialogue about the importance of transparency and accountability in AI systems, ensuring that the voices of those affected by these technologies were heard and considered in the policy-making process. + +4. **Promote Inclusivity**: Ensure that the development of the AI Bill of Rights was inclusive and representative of different demographics, including marginalized and underrepresented groups, to address their specific concerns and needs. + +5. **Inform Oversight Possibilities**: Explore potential oversight mechanisms that could be implemented to monitor and regulate AI technologies effectively, ensuring they align with ethical standards and public interests. + +Overall, the public engagement process was crucial for creating a comprehensive and informed Blueprint for an AI Bill of Rights that addresses the complexities of AI technologies and their impact on society (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!",The purpose of the year of public engagement that informed the development of the Blueprint for an AI Bill of Rights was to gather input and feedback from the public to shape the framework and ensure it reflects the values and concerns of the American people.,0.25,0.98254660364619,0.0,0.9999999999666667,0.3482109359165827 +How can automated systems prevent 'mission creep' while ensuring privacy and user control?,"['>>>\nProtect the public from unchecked surveillance \nHeightened oversight of surveillance. Surveillance or monitoring systems should be subject to heightened oversight that includes at a minimum assessment of potential harms during design (before deployment) and in an ongoing manner, to ensure that the American public’s rights, opportunities, and access are protected. This assessment should be done before deployment and should give special attention to ensure there is not algorithmic discrimination, especially based on community membership, when deployed in a specific real-world context. Such assessment should then be reaffirmed in an ongoing manner as long as the system is in use. \nLimited and proportionate surveillance. Surveillance should be avoided unless it is strictly necessary to achieve a legitimate purpose and it is proportionate to the need. Designers, developers, and deployers of surveillance systems should use the least invasive means of monitoring available and restrict monitoring to the minimum number of subjects possible. To the greatest extent possible consistent with law enforcement and national security needs, individuals subject to monitoring should be provided with clear and specific notice before it occurs and be informed about how the data gathered through surveillance will be used. \nScope limits on surveillance to protect rights and democratic values. Civil liberties and civil\n>>>' + 'In discussion of technical and governance interventions that that are needed to protect against the harms of these technologies, panelists individually described the importance of: receiving community input into the design and use of technologies, public reporting on crucial elements of these systems, better notice and consent procedures that ensure privacy based on context and use case, ability to opt-out of using these systems and receive a fallback to a human process, providing explanations of decisions and how these systems work, the need for governance including training in using these systems, ensuring the technological use cases are genuinely related to the goal task and are locally validated to work, and the need for institution and protection of third party audits to ensure systems continue to be accountable and valid.' + 'Data collection and use-case scope limits. Data collection should be limited in scope, with specific, narrow identified goals, to avoid ""mission creep."" Anticipated data collection should be determined to be strictly necessary to the identified goals and should be minimized as much as possible. Data collected based on these identified goals and for a specific context should not be used in a different context without assessing for new privacy risks and implementing appropriate mitigation measures, which may include express consent. Clear timelines for data retention should be established, with data deleted as soon as possible in accordance with legal or policy-based limitations. Determined data retention timelines should be documented and justified. \n\nPrivacy-preserving security. Entities creating, using, or governing automated systems should follow privacy and security best practices designed to ensure data and metadata do not leak beyond the specific consented use case. Best practices could include using privacy-enhancing cryptography or other types of privacy-enhancing technologies or fine-grained permissions and access control mechanisms, along with conventional system security protocols.']","To prevent ""mission creep"" while ensuring privacy and user control in automated systems, organizations can implement several key strategies: + +1. **Define Clear Objectives**: Establish specific, narrow goals for the automated system from the outset. This clarity helps ensure that the system is used only for its intended purpose and prevents the expansion of its use beyond those defined objectives. + +2. **Data Minimization**: Collect only the data that is strictly necessary to achieve the defined goals. By limiting data collection to what is essential, organizations can reduce the risk of using data for unintended purposes, thereby preventing mission creep. + +3. **Scope Limits on Data Use**: Clearly define the contexts in which the collected data can be used. This includes specifying the types of analyses that can be performed and the decisions that can be made based on the data. Any use outside these boundaries should require a new assessment of privacy risks and potential impacts. + +4. **User Consent and Control**: Implement mechanisms that allow users to provide informed consent regarding their data. Users should have control over how their data is used, including the ability to opt-out of data collection or request deletion of their data. + +5. **Regular Audits and Reviews**: Conduct regular audits of the automated system to ensure compliance with the defined objectives and data use policies. These audits can help identify any deviations from the intended use and allow for corrective actions to be taken. + +6. **Transparency and Communication**: Maintain open communication with users about what data is being collected, how it will be used, and their rights regarding their data. Transparency helps build trust and allows users to make informed decisions about their participation. + +7. **Feedback Mechanisms**: Establish channels for users to provide feedback on the system's operation and data use. This feedback can be valuable for identifying potential issues related to mission creep and ensuring that user concerns are addressed. + +8. **Privacy Impact Assessments**: Conduct privacy impact assessments before deploying the automated system. These assessments should evaluate potential privacy risks associated with data collection and use, helping to ensure that the system aligns with privacy principles. + +By implementing these strategies, organizations can effectively prevent mission creep while ensuring that privacy and user control are prioritized in the design and operation of automated systems (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!","Automated systems can prevent 'mission creep' and ensure privacy and user control by limiting data collection to specific, narrow goals that are strictly necessary for the identified purposes. Data collection should be minimized, clearly communicated to users, and used only for legal and expected purposes. Any use of data in a different context should be assessed for new privacy risks and appropriate mitigation measures should be implemented, potentially including obtaining express consent. Clear timelines for data retention should be established, with data deleted as soon as possible in accordance with legal or policy-based limitations. Entities should proactively identify and manage privacy risks, avoiding processing data when risks outweigh benefits and implementing measures to mitigate acceptable risks. Privacy-preserving security measures, such as privacy-enhancing cryptography and access control mechanisms, should be employed to prevent data leakage beyond consented use cases.",0.8888888888888888,0.9615231465422364,0.8333333333333334,0.9999999999666667,0.721101137599911 +"How can GAI tech improve red-teaming with human teams, ensuring content origin and incident disclosure?","['• Human / AI: Performed by GAI in combination with specialist or non-specialist human teams. \nGAI-led red-teaming can be more cost effective than human red-teamers alone. Human or GAI-led AI red-teaming may be better suited for eliciting different types of harms. \nA.1.6. Content Provenance \nOverview \nGAI technologies can be leveraged for many applications such as content generation and synthetic data. Some aspects of GAI outputs, such as the production of deepfake content, can challenge our ability to distinguish human-generated content from AI-generated synthetic content. To help manage and mitigate these risks, digital transparency mechanisms like provenance data tracking can trace the origin and history of content. Provenance data tracking and synthetic content detection can help facilitate greater information access about both authentic and synthetic content to users, enabling better knowledge of trustworthiness in AI systems. When combined with other organizational accountability mechanisms, digital content transparency approaches can enable processes to trace negative outcomes back to their source, improve information integrity, and uphold public trust. Provenance data tracking and synthetic content detection mechanisms provide information about the origin and history of content to assist in GAI risk management efforts. \nProvenance metadata can include information about GAI model developers or creators of GAI content, date/time of creation, location, modifications, and sources. Metadata can be tracked for text, images, videos, audio, and underlying datasets. The implementation of provenance data tracking techniques can' + '53 Documenting, reporting, and sharing information about GAI incidents can help mitigate and prevent harmful outcomes by assisting relevant AI Actors in tracing impacts to their source. Greater awareness and standardization of GAI incident reporting could promote this transparency and improve GAI risk management across the AI ecosystem. Documentation and Involvement of AI Actors AI Actors should be aware of their roles in reporting AI incidents. To better understand previous incidents and implement measures to prevent similar ones in the future, organizations could consider developing guidelines for publicly available incident reporting which include information about AI actor responsibilities. These guidelines would help AI system operators identify GAI incidents across the AI lifecycle and with AI Actors regardless of role. Documentation and review of third-party inputs and plugins for GAI systems is especially important for AI Actors in the context of incident disclosure; LLM inputs and content delivered through these plugins is often distributed, with inconsistent or insufficient access control. Documentation practices including logging, recording, and analyzing GAI incidents can facilitate smoother sharing of information with relevant AI Actors. Regular information sharing, change management records, version history and metadata can also empower AI Actors responding to and managing AI incidents.' + 'AI technology can produce varied outputs in multiple modalities and present many classes of user interfaces. Organizations can restrict AI applications that cause harm, exceed stated risk tolerances, or that conflict with their tolerances or values. Governance tools and protocols that are applied to other types of AI systems can be applied to GAI systems. These plans and actions include: \n• Data provenance']","GAI technology can significantly enhance red-teaming efforts when combined with human teams by providing advanced capabilities for identifying vulnerabilities and simulating potential attacks. Here’s how GAI can contribute while ensuring the origin of content and proper disclosure of incidents: + +1. **Automated Vulnerability Detection**: GAI can analyze large datasets and codebases quickly to identify potential vulnerabilities that human teams might overlook. This capability allows for a more thorough assessment of security weaknesses in systems. + +2. **Simulating Adversarial Attacks**: GAI can generate realistic attack scenarios and simulate adversarial behavior, helping human teams understand how systems might be exploited. This can include generating phishing emails, crafting social engineering scenarios, or creating malicious payloads. + +3. **Enhanced Scenario Generation**: GAI can assist in creating diverse and complex attack scenarios that reflect real-world threats. This helps human teams prepare for a wider range of potential security incidents and develop more robust defense strategies. + +4. **Content Provenance Tracking**: To ensure the origin of content generated during red-teaming exercises, organizations can implement provenance data tracking mechanisms. This involves documenting the source of GAI-generated content, including information about the model used, the data inputs, and any modifications made. This transparency is crucial for accountability and understanding the context of the generated content. + +5. **Incident Disclosure Protocols**: Establishing clear protocols for documenting and disclosing incidents identified during red-teaming exercises is essential. Organizations should create guidelines that outline the responsibilities of AI actors in reporting incidents, including the nature of the incident, its impact, and the steps taken to address it. This ensures that all relevant stakeholders are informed and can take appropriate action. + +6. **Collaboration and Feedback Loops**: GAI can facilitate collaboration between human teams by providing real-time insights and suggestions during red-teaming exercises. This collaborative approach allows human teams to leverage GAI's capabilities while maintaining oversight and critical thinking in evaluating the generated scenarios. + +7. **Continuous Learning and Improvement**: GAI systems can learn from past red-teaming exercises, improving their ability to generate relevant scenarios and identify vulnerabilities over time. This continuous learning process enhances the effectiveness of red-teaming efforts and helps organizations stay ahead of emerging threats. + +8. **Documentation and Reporting**: GAI can assist in documenting the findings and outcomes of red-teaming exercises, ensuring that all incidents are recorded accurately. This documentation can be used for future reference, training, and compliance purposes. + +By integrating GAI technology into red-teaming efforts, organizations can enhance their security posture while ensuring that the origin of content is tracked and incidents are disclosed appropriately. This combination of advanced technology and human expertise leads to more effective risk management and improved overall security (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!","GAI technologies can improve red-teaming with human teams by combining GAI with specialist or non-specialist human teams. GAI-led red-teaming can be more cost-effective than human red-teamers alone and may be better suited for eliciting different types of harms. Content provenance mechanisms like provenance data tracking can trace the origin and history of content, helping to manage and mitigate risks associated with GAI output. These approaches can enable processes to trace negative outcomes back to their source, improve information integrity, and uphold public trust.",0.3877551020408163,0.9374423875174976,1.0,0.9999999999666667,0.9306885019426268 +Why is it important for lenders to inform consumers about decisions made under FCRA in automated systems?,"['Lenders are required by federal law to notify consumers about certain decisions made about them. Both the Fair Credit Reporting Act and the Equal Credit Opportunity Act require in certain circumstances that consumers who are denied credit receive ""adverse action"" notices. Anyone who relies on the information in a credit report to deny a consumer credit must, under the Fair Credit Reporting Act, provide an ""adverse action"" notice to the consumer, which includes ""notice of the reasons a creditor took adverse action on the application.""' + 'You should know that an automated system is being used, and understand how and why it contributes to outcomes that impact you. Designers, developers, and deployers of automated systems should provide generally accessible plain language documentation including clear descriptions of the overall system functioning and the role automation plays, notice that such systems are in use, the individual or organization responsible for the system, and explanations of outcomes that are clear, timely, and accessible. Such notice should be kept up-to-date and people impacted by the system should be notified of significant use case or key functionality changes. You should know how and why an outcome impacting you was determined by an automated system, including when the automated system is not the sole input determining the outcome. Automated systems should provide explanations that are technically valid, meaningful and useful to you and to any operators or others who need to understand the system, and calibrated to the level of risk based on the context. Reporting that includes summary information about these automated systems in plain language and assessments of the clarity and quality of the notice and explanations should be made public whenever possible.' + '""In addition, under the risk-based pricing rule,91 lenders must either inform borrowers of their credit score, or else tell consumers when \'they are getting worse terms because of information in their credit report.\' The CFPB has also asserted that \'[t]he law gives every applicant the right to a specific explanation if their application for credit was denied, and that right is not diminished simply because a company uses a complex algorithm that it doesn\'t understand.\'92 Such explanations illustrate a shared value that certain decisions need to be explained.""']","It is important for lenders to inform consumers about decisions made under the Fair Credit Reporting Act (FCRA) in automated systems for several key reasons: + +1. **Consumer Rights**: The FCRA mandates that consumers have the right to be informed when adverse actions are taken based on information in their credit reports. This includes providing consumers with ""adverse action"" notices that explain the reasons for the decision. Informing consumers helps uphold their rights and ensures they are aware of how their credit information is being used (Source: [document name], p. [specific page number]). + +2. **Transparency**: Providing notice about decisions made by automated systems fosters transparency in the lending process. Consumers need to understand how their creditworthiness is assessed, especially when automated systems are involved. Transparency builds trust between lenders and consumers, as it allows individuals to see the factors influencing their credit decisions. + +3. **Opportunity for Correction**: When consumers are informed about adverse decisions, they have the opportunity to review their credit reports and identify any inaccuracies or errors. This is crucial for allowing consumers to correct any misinformation that may have negatively impacted their credit decisions. + +4. **Accountability**: Informing consumers about automated decision-making processes holds lenders accountable for their actions. It ensures that lenders cannot rely solely on opaque algorithms without providing explanations for their decisions. This accountability is essential for ethical lending practices. + +5. **Understanding of Automated Systems**: As automated systems become more prevalent in lending, it is vital for consumers to understand how these systems operate. Providing clear explanations about the role of automation in decision-making helps demystify the process and empowers consumers to make informed decisions regarding their credit applications. + +6. **Regulatory Compliance**: Lenders are legally required to comply with the FCRA and provide the necessary notifications to consumers. Failing to do so can result in legal repercussions and damage to the lender's reputation. Compliance with these regulations is essential for maintaining trust and credibility in the financial industry. + +7. **Consumer Education**: Informing consumers about the decision-making process helps educate them about credit reporting and scoring. This knowledge can empower consumers to take proactive steps to improve their credit profiles and make better financial decisions in the future. + +In summary, informing consumers about decisions made under the FCRA in automated systems is crucial for protecting consumer rights, ensuring transparency, promoting accountability, and fostering trust in the lending process (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!","It is important for lenders to inform consumers about decisions made under FCRA in automated systems because the Fair Credit Reporting Act requires that consumers who are denied credit receive ""adverse action"" notices. These notices must include the reasons for the adverse action taken on the application or an existing credit account. Additionally, under the risk-based pricing rule, lenders must inform borrowers of their credit score or explain when they are receiving worse terms due to information in their credit report. This transparency is crucial to ensure that consumers understand the basis for credit decisions, especially when complex algorithms are involved.",0.5652173913043478,0.9785683808084092,0.75,0.9999999999666667,0.4542976287867849 +Why is public transparency important in automated systems affecting people's lives and decisions?,"['You should know that an automated system is being used and understand how and why it contributes to outcomes that impact you. Designers, developers, and deployers of automated systems should provide generally accessible plain language documentation including clear descriptions of the overall system functioning and the role automation plays, notice that such systems are in use, the individual or organization responsible for the system, and explanations of outcomes that are clear, timely, and accessible. Such notice should be kept up-to-date and people impacted by the system should be notified of significant use case or key functionality changes. You should know how and why an outcome impacting you was determined by an automated system, including when the automated system is not the sole input determining the outcome. Automated systems should provide explanations that are technically valid, meaningful and useful to you and to any operators or others who need to understand the system, and calibrated to the level of risk based on the context. Reporting that includes summary information about these automated systems in plain language and assessments of the clarity and quality of the notice and explanations should be made public whenever possible.' + 'In the criminal justice system, employment, education, healthcare, and other sensitive domains, automated systems are used for many purposes, from pre-trial risk assessments and parole decisions to technologies that help doctors diagnose disease. Absent appropriate safeguards, these technologies can lead to unfair, inaccurate, or dangerous outcomes. These sensitive domains require extra protections. It is critically important that there is extensive human oversight in such settings.' + 'While technologies are being deployed to solve problems across a wide array of issues, our reliance on technology can also lead to its use in situations where it has not yet been proven to work—either at all or within an acceptable range of error. In other cases, technologies do not work as intended or as promised, causing substantial and unjustified harm. Automated systems sometimes rely on data from other systems, including historical data, allowing irrelevant information from past decisions to infect decision-making in unrelated situations. Many of the harms resulting from these technologies are preventable, and actions are already being taken to protect the public. Some companies have put in place safeguards that have prevented harm from occurring by ensuring that key development decisions are vetted by an ethics review; others have identified and mitigated harms found through pre-deployment testing and ongoing monitoring processes. Expanded, proactive protections could build on these existing practices, increase confidence in the use of automated systems, and protect the American public.' + 'While notice and explanation requirements are already in place in some sectors or situations, the American public deserve to know consistently and across sectors if an automated system is being used in a way that impacts their rights, opportunities, or access. This knowledge should provide confidence in how the public is being treated, and trust in the validity and reasonable use of automated systems. \n\nThe lack of a timely explanation made it harder to understand and contest the decision.\n\nThe lack of notice or an explanation makes it harder for those performing child maltreatment assessments to validate the risk assessment and denies parents knowledge that could help them contest a decision.']","Public transparency is crucial in automated systems that impact individuals' lives and decisions for several reasons: + +1. **Accountability**: Transparency ensures that organizations are held accountable for the decisions made by automated systems. When individuals understand how these systems operate and the criteria used for decision-making, it becomes easier to identify and address any potential biases or errors. + +2. **Trust Building**: When organizations are transparent about their automated systems, it fosters trust among users and the public. People are more likely to accept and engage with technologies when they feel informed and confident about how their data is being used and how decisions are made. + +3. **Informed Consent**: Transparency allows individuals to make informed choices about their participation in systems that affect them. When people understand what data is being collected, how it will be used, and the implications of automated decisions, they can provide meaningful consent. + +4. **Error Identification and Correction**: Public transparency enables individuals to identify errors or inaccuracies in automated decision-making processes. When people are aware of how decisions are made, they can contest or challenge outcomes that they believe are incorrect, leading to better overall system performance. + +5. **Protection of Rights**: Automated systems can significantly impact individuals' rights and opportunities, such as in employment, credit, and healthcare. Transparency helps ensure that these systems operate fairly and do not infringe upon individuals' rights, allowing for oversight and intervention when necessary. + +6. **Public Engagement**: Transparency encourages public engagement and dialogue about the use of automated systems. This engagement can lead to valuable feedback and insights that help improve the systems and address community concerns. + +7. **Ethical Standards**: By promoting transparency, organizations can demonstrate their commitment to ethical practices in the development and deployment of automated systems. This can help establish industry standards and best practices that prioritize the well-being of individuals. + +8. **Regulatory Compliance**: Many regulations require transparency in automated decision-making processes, especially in sensitive areas. Adhering to these requirements not only ensures compliance but also enhances the organization's reputation and credibility. + +In summary, public transparency in automated systems is essential for accountability, trust, informed consent, error correction, protection of rights, public engagement, ethical standards, and regulatory compliance. It ultimately leads to more responsible and equitable use of technology that respects individuals' interests and rights (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!","Public transparency is crucial in automated systems affecting people's lives and decisions because it allows both the authorities and the public to understand why and how decisions are being made. Without transparency, individuals may be subject to decisions made by automated systems without any explanation or accountability, leading to potential errors, biases, and injustices. Transparency also enables experts to verify the reasonableness of recommendations before they are implemented, ensuring safety and efficacy. In summary, public transparency in automated systems is essential for accountability, fairness, and the protection of individuals' rights and opportunities.",0.85,0.9680746218583742,0.75,0.999999999975,0.7976556311423452 +How can governance principles manage risks of GAI effectively?,"['A.1. Governance \nA.1.1. Overview \nLike any other technology system, governance principles and techniques can be used to manage risks related to generative AI models, capabilities, and applications. Organizations may choose to apply their existing risk tiering to GAI systems, or they may opt to revise or update AI system risk levels to address these unique GAI risks. This section describes how organizational governance regimes may be re-evaluated and adjusted for GAI contexts. It also addresses third-party considerations for governing across the AI value chain. \nA.1.2. Organizational Governance \nGAI opportunities, risks and long-term performance characteristics are typically less well-understood than non-generative AI tools and may be perceived and acted upon by humans in ways that vary greatly. Accordingly, GAI may call for different levels of oversight from AI Actors or different human-AI configurations in order to manage their risks effectively. Organizations’ use of GAI systems may also warrant additional human review, tracking and documentation, and greater management oversight.' + 'risks, and creates unique risks. GAI risks can vary along many dimensions: \n• Stage of the AI lifecycle: Risks can arise during design, development, deployment, operation, and/or decommissioning. \n• Scope: Risks may exist at individual model or system levels, at the application or implementation levels (i.e., for a specific use case), or at the ecosystem level – that is, beyond a single system or organizational context. Examples of the latter include the expansion of “algorithmic monocultures,” resulting from repeated use of the same model, or impacts on access to opportunity, labor markets, and the creative economies. \n• Source of risk: Risks may emerge from factors related to the design, training, or operation of the GAI model itself, stemming in some cases from GAI model or system inputs, and in other cases, from GAI system outputs. Many GAI risks, however, originate from human behavior.' + 'Organizations can restrict AI applications that cause harm, exceed stated risk tolerances, or that conflict with their tolerances or values. Governance tools and protocols that are applied to other types of AI systems can be applied to GAI systems. These p lans and actions include: \n• Accessibility and reasonable accommodations \n• AI actor credentials and qualifications \n• Alignment to organizational values \n• Auditing and assessment \n• Change -management controls \n• Commercial use \n• Data provenance']","Governance principles can effectively manage the risks associated with Generative AI (GAI) by implementing a structured framework that addresses the unique challenges posed by these technologies. Here are several key strategies: + +1. **Risk Assessment Framework**: Establish a comprehensive risk assessment framework that identifies and evaluates the specific risks associated with GAI systems. This includes assessing risks at various stages of the AI lifecycle—design, development, deployment, operation, and decommissioning. + +2. **Clear Policies and Guidelines**: Develop clear policies and guidelines that outline acceptable use cases for GAI technologies. These policies should define the boundaries of GAI applications, ensuring that they align with organizational values and ethical standards. + +3. **Human Oversight**: Implement mechanisms for human oversight in GAI decision-making processes. This may involve requiring human review for high-stakes decisions or providing tools for human operators to intervene when necessary. Human oversight helps mitigate risks related to automated decision-making. + +4. **Transparency and Explainability**: Promote transparency in GAI systems by ensuring that their decision-making processes are explainable. This includes providing clear documentation on how GAI models operate and the data used for training, which can help stakeholders understand the rationale behind outputs. + +5. **Bias Mitigation Strategies**: Actively work to identify and mitigate biases in GAI systems. This involves conducting regular audits of training data and model outputs to ensure fairness and equity in the generated content. Organizations should also implement strategies to address any identified biases. + +6. **Data Governance**: Establish robust data governance practices to manage the data used in GAI systems. This includes ensuring data quality, provenance, and compliance with privacy regulations. Organizations should also implement data minimization principles to limit the collection of unnecessary information. + +7. **Stakeholder Engagement**: Engage with a diverse range of stakeholders, including users, affected communities, and experts, to gather input on GAI systems. This engagement can help identify potential risks and inform the development of governance frameworks that address community concerns. + +8. **Regular Monitoring and Evaluation**: Implement continuous monitoring and evaluation processes to assess the performance and impact of GAI systems over time. This includes tracking outcomes, identifying emerging risks, and making necessary adjustments to governance practices. + +9. **Incident Response Plans**: Develop and maintain incident response plans to address any adverse events or failures associated with GAI systems. These plans should outline procedures for reporting, investigating, and mitigating incidents, ensuring that organizations can respond effectively to risks. + +10. **Training and Awareness**: Provide training and resources for employees and stakeholders on the ethical use of GAI technologies. Raising awareness about the potential risks and ethical considerations associated with GAI can foster a culture of responsible AI use within organizations. + +By implementing these governance principles, organizations can effectively manage the risks associated with Generative AI, ensuring that these technologies are used responsibly and ethically while maximizing their benefits (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!","Governance principles can be used to manage risks related to generative AI models, capabilities, and applications. Organizations may choose to apply their existing risk tiering to GAI systems or revise/update AI system risk levels to address unique GAI risks. Organizational governance regimes may need to be re-evaluated and adjusted for GAI contexts, including third-party considerations across the AI value chain. GAI may require different levels of oversight from AI actors or different human-AI configurations to manage risks effectively. Organizations using GAI systems may need additional human review, tracking, documentation, and management oversight. Governance tools and protocols applied to other AI systems can also be applied to GAI systems, including accessibility, AI actor credentials, alignment to organizational values, auditing, change-management controls, commercial use, and data provenance.",0.6190476190476191,0.9239523900214239,1.0,0.9999999999666667,0.8627920775141443 +"Why is accuracy important in reviewing and documenting data throughout the AI life cycle, considering factors like bias, IP, integrity, and GAI risks?","['Information Integrity \nMS-3.3-003 Evaluate potential biases and stereotypes that could emerge from the AI - generated content using appropriate methodologies including computational testing methods as well as evaluating structured feedback input. Harmful Bias and Homogenization' + 'Information integrity describes the “ spectrum of information and associated patterns of its creation, exchange, and consumption in society .” High-integrity information can be trusted; “distinguishes fact from fiction, opinion, and inference; acknowledges uncertainties; and is transparent about its level of vetting. This information can be linked to the original source(s) with appropriate evidence. High- integrity information is also accurate and reliable, can be verified and authenticated, has a clear chain of custody, and creates reasonable expectations about when its validity may expire. ”' + 'MS-2.2-001 Assess and manage statistical biases related to GAI content provenance through techniques such as re-sampling, re-weighting, or adversarial training. Information Integrity ; Information Security ; Harmful Bias and Homogenization \nMS-2.2-002 Document how content provenance data is tracked and how that data interacts with privacy and security. Consider: Anonymizing data to protect the privacy of human subjects; Leveraging privacy output filters; Removing any personally identifiable information (PII) to prevent potential harm or misuse. Data Privacy ; Human AI Configuration; Information Integrity ; Information Security ; Dangerous, Violent, or Hateful Content \nMS-2.2-003 Provide human subjects with options to withdraw participation or revoke their consent for present or future use of their data in GAI applications. Data Privacy ; Human-AI Configuration; Information Integrity \nMS-2.2-004 Use techniques such as anonymization, differential privacy or other privacy-enhancing technologies to minimize the risks associated with linking AI-generated content back to individual human subjects. Data Privacy ; Human-AI Configuration']","Accuracy in reviewing and documenting data throughout the AI life cycle is crucial for several reasons, particularly concerning bias, intellectual property, integrity, and risks associated with Generative AI (GAI): + +1. **Bias Mitigation**: Accurate data documentation helps identify and address biases in training datasets. If the data is not accurately reviewed, biases may go unnoticed, leading to unfair or discriminatory outcomes in AI systems. Ensuring accuracy allows organizations to implement corrective measures and promote fairness in AI applications. + +2. **Intellectual Property Protection**: Accurate documentation of data sources and usage is essential for protecting intellectual property rights. Organizations must ensure that they have the right to use the data they collect and that they comply with licensing agreements. Inaccurate documentation can lead to legal disputes and potential infringement claims. + +3. **Data Integrity**: Maintaining accuracy in data documentation is vital for ensuring the integrity of the AI system. Inaccurate or incomplete data can compromise the reliability of the model's outputs, leading to erroneous conclusions and decisions. Accurate records help verify the quality and provenance of the data used in training and testing AI systems. + +4. **Risk Management**: Accurate documentation allows organizations to assess and manage risks associated with GAI effectively. By keeping detailed records of data sources, model training processes, and decision-making criteria, organizations can identify potential risks and implement strategies to mitigate them. This proactive approach is essential for maintaining compliance with regulations and ethical standards. + +5. **Transparency and Accountability**: Accurate documentation fosters transparency in AI systems, enabling stakeholders to understand how decisions are made and the data that informs those decisions. This transparency is critical for building trust with users and the public, as it allows for scrutiny and accountability in the use of AI technologies. + +6. **Regulatory Compliance**: Many regulations require organizations to maintain accurate records of data usage and decision-making processes. Ensuring accuracy in documentation helps organizations comply with legal requirements, reducing the risk of penalties and reputational damage. + +7. **Continuous Improvement**: Accurate data documentation supports continuous improvement efforts in AI systems. By reviewing and analyzing documented data, organizations can identify areas for enhancement, refine their models, and improve overall performance. This iterative process is essential for adapting to changing conditions and user needs. + +8. **Ethical Considerations**: Accuracy in data documentation is fundamental to ethical AI practices. Organizations have a responsibility to ensure that their AI systems operate fairly and transparently. Accurate records help uphold ethical standards and demonstrate a commitment to responsible AI use. + +In summary, accuracy in reviewing and documenting data throughout the AI life cycle is essential for mitigating bias, protecting intellectual property, ensuring data integrity, managing risks, promoting transparency, complying with regulations, supporting continuous improvement, and upholding ethical standards. By prioritizing accuracy, organizations can enhance the effectiveness and trustworthiness of their AI systems (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!","Accuracy is crucial in reviewing and documenting data throughout the AI life cycle to ensure the data's reliability, representativeness, relevance, and suitability at different stages. This is particularly important due to factors like harmful bias, homogenization, intellectual property concerns, information integrity, and GAI risks. Ensuring accuracy helps in verifying the information generated by GAI systems, identifying potential biases or harmful content, and maintaining the trustworthiness of AI systems.",1.0,0.9600586340557004,1.0,0.9999999999666667,0.9872112119338794 +How can feedback be used to gather user input on AI content while aligning with values and detecting quality shifts?,"['• Participatory Engagement Methods : Methods used to solicit feedback from civil society groups, affected communities, and users, including focus groups, small user studies, and surveys. \n• Field Testing : Methods used to determine how people interact with, consume, use, and make sense of AI -generated information, and subsequent actions and effects, including UX, usability, and other structured, randomized experiments. \nInformation gathered from structured public feedback can inform design, implementation, deployment approval, maintenance, or decommissioning decisions. Results and insights gleaned from these exercises can serve multiple purposes, including improving data quality and preprocessing, bolstering governance decision making, and enhancing system documentation and debugging practices. When implementing feedback activities, organizations should follow human subjects research requirements and best practices such as informed consent and subject compensation.' + 'Evaluate potential biases and stereotypes that could emerge from the AI - generated content using appropriate methodologies including computational testing methods as well as evaluating structured feedback input.' + 'Organizations can restrict AI applications that cause harm, exceed stated risk tolerances, or that conflict with their tolerances or values. Governance tools and protocols that are applied to other types of AI systems can be applied to GAI systems. These p lans and actions include: \n• Alignment to organizational values • Auditing and assessment' + 'Enhancing Content Provenance through Structured Public Feedback \nWhile indirect feedback methods such as automated error collection systems are useful, they often lack the context and depth that direct input from end users can provide. Organizations can leverage feedback approaches described in the Pre-Deployment Testing section to capture input from external sources such as through AI red-teaming. \nIntegrating pre- and post-deployment external feedback into the monitoring process for GAI models and corresponding applications can help enhance awareness of performance changes and mitigate potential risks and harms from outputs. There are many ways to capture and make use of user feedback – before and after GAI systems and digital content transparency approaches are deployed – to gain insights about authentication efficacy and vulnerabilities, impacts of adversarial threats on techniques, and unintended consequences resulting from the utilization of content provenance approaches on users.']","Feedback can be effectively utilized to collect user input on AI-generated content while ensuring alignment with organizational values and identifying shifts in quality through several key strategies: + +1. **Structured Feedback Mechanisms**: Implement structured feedback tools such as surveys, rating systems, or focus groups that allow users to provide specific input on AI-generated content. These tools can be designed to capture qualitative and quantitative data, helping organizations assess user satisfaction and identify areas for improvement. + +2. **Alignment with Organizational Values**: Develop feedback questions that explicitly relate to the organization's core values and ethical standards. This ensures that user input is evaluated not only for quality but also for alignment with the organization's mission and principles. For example, questions could focus on fairness, transparency, and the impact of the content on diverse communities. + +3. **Regular Monitoring and Analysis**: Establish a routine for monitoring and analyzing user feedback over time. This can help organizations identify trends and shifts in user perceptions regarding the quality of AI-generated content. Regular analysis allows for timely adjustments to content generation processes and helps maintain alignment with organizational values. + +4. **User Engagement**: Actively engage users in the feedback process by encouraging them to share their experiences and suggestions. This can be done through community forums, user panels, or interactive sessions where users can discuss their thoughts on AI-generated content. Engaging users fosters a sense of ownership and investment in the content produced. + +5. **Incorporating Diverse Perspectives**: Ensure that feedback is collected from a diverse range of users, including those from different backgrounds and experiences. This diversity can provide valuable insights into how AI-generated content is perceived across various demographics, helping organizations identify potential biases or misalignments with their values. + +6. **Feedback Loop Integration**: Create a feedback loop where user input is not only collected but also acted upon. Communicate to users how their feedback has influenced changes in AI-generated content. This transparency reinforces trust and encourages ongoing participation in the feedback process. + +7. **Quality Metrics Development**: Develop specific quality metrics that reflect both user satisfaction and alignment with organizational values. These metrics can be used to evaluate AI-generated content systematically and identify any shifts in quality over time. Metrics might include accuracy, relevance, user engagement, and adherence to ethical guidelines. + +8. **Training and Calibration**: Use feedback to inform the training and calibration of AI models. By incorporating user input into the model refinement process, organizations can enhance the quality of AI-generated content and ensure it aligns with user expectations and organizational values. + +9. **Ethical Review Processes**: Establish ethical review processes for evaluating AI-generated content based on user feedback. This can involve cross-functional teams that assess content against organizational values and ethical standards, ensuring that any shifts in quality are addressed promptly. + +By implementing these strategies, organizations can effectively utilize feedback to collect user input on AI-generated content, ensuring that it aligns with their values and helps identify shifts in quality. This approach fosters a culture of continuous improvement and accountability in AI content generation (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!",Use structured feedback mechanisms to solicit and capture user input about AI-generated content to detect subtle shifts in quality or alignment with community and societal values.,0.75,0.9490951878900963,1.0,0.9166666666361111,0.9814550884789243 +What measures are being taken to address issues for transgender travelers at airport checkpoints?,"['43. Arli Christian. Four Ways the TSA Is Making Flying Easier for Transgender People . American Civil Liberties Union. Apr. 5, 2022. https://www.aclu.org/news/lgbtq-rights/four-ways-the-tsa-is-making-flying-easier-for-transgender-people \n44. U.S. Transportation Security Administration. Transgender/ Non Binary / Gender Nonconforming Passengers. TSA. Accessed Apr. 21, 2022. https://www.tsa.gov/transgender-passengers' + '• Body scanners, used by TSA at airport checkpoints, require the operator to select a “male” or “female” scanning setting based on the passenger’s sex, but the setting is chosen based on the operator’s perception of the passenger’s gender identity. These scanners are more likely to flag transgender travelers as requiring extra screening done by a person. Transgender travelers have described degrading experiences associated with these extra screenings. TSA has recently announced plans to implement a gender-neutral algorithm.']","I'm sorry, I don't know the answer to that question.",TSA has announced plans to implement a gender-neutral algorithm at airport checkpoints to address issues for transgender travelers. This algorithm aims to enhance security effectiveness capabilities while reducing the likelihood of flagging transgender travelers for extra screening based on gender identity perceptions.,0.0,0.0,0.5,0.99999999995,0.17744743782659386 +How do ballot curing laws help voters fix ballot issues despite flaws in signature matching systems?,"['•An automated signature matching system is used as part of the voting process in many parts of the country to determine whether the signature on a mail-in ballot matches the signature on file. These signature matching systems are less likely to work correctly for some voters, including voters with mental or physical disabilities, voters with shorter or hyphenated names, and voters who have changed their name. A human curing process, which helps voters to confirm their signatures and correct other voting mistakes, is important to ensure all votes are counted, and it is already standard practice in much of the country for both an election official and the voter to have the opportunity to review and correct any such issues.' + '110 Ballot \ncuring processes vary among states, and include direct phone calls, emails, or mail contact by election \nofficials. Voters are asked to provide alternative information or a new signature to verify the validity of their \nballot.' + '110. Rachel Orey and Owen Bacskai. The Low Down on Ballot Curing. Nov. 04, 2020. https:// bipartisanpolicy.org/blog/the-low-down-on-ballot-curing/; Zahavah Levine and Thea Raymond-Seidel. Mail Voting Litigation in 2020, Part IV: Verifying Mail Ballots . Oct. 29, 2020. https://www.lawfareblog.com/mail-voting-litigation-2020-part-iv-verifying-mail-ballots \n111. National Conference of State Legislatures. Table 15: States With Signature Cure Processes. Jan. 18, 2022. https://www.ncsl.org/research/elections-and-campaigns/vopp-table-15-states-that-permit-voters-to-correct-signature-discrepancies.aspx']","Ballot curing laws play a crucial role in assisting voters in correcting issues with their ballots, particularly in the context of the shortcomings of signature matching systems. Here’s how these laws function: + +1. **Addressing Signature Matching Limitations**: Signature matching systems, which are used to verify the identity of voters by comparing signatures on mail-in ballots to those on file, can be problematic. They may not work correctly for certain groups, including individuals with disabilities, those with shorter or hyphenated names, and voters who have changed their names. Ballot curing laws provide a mechanism for these voters to address discrepancies. + +2. **Opportunity for Correction**: Ballot curing laws typically allow voters to be notified if there is an issue with their ballot, such as a signature mismatch. Voters are then given the opportunity to correct these issues before the ballot is ultimately rejected. This process is essential for ensuring that all eligible votes are counted. + +3. **Varied Curing Processes**: The specific processes for ballot curing can vary by state. Common methods include direct communication from election officials to voters via phone calls, emails, or mail. Voters may be asked to provide additional information or a new signature to verify their identity and confirm the validity of their ballot (Source: Rachel Orey and Owen Bacskai, ""The Low Down on Ballot Curing,"" Nov. 04, 2020). + +4. **Enhancing Voter Participation**: By allowing voters to correct issues with their ballots, ballot curing laws help enhance voter participation and confidence in the electoral process. Voters are more likely to engage in the voting process when they know there are mechanisms in place to address potential problems. + +5. **Legal Protections**: These laws also serve as legal protections for voters, ensuring that they have recourse if their ballots are challenged or rejected due to signature verification issues. This is particularly important in maintaining the integrity of the electoral process and protecting the rights of voters. + +In summary, ballot curing laws are essential for helping voters correct ballot issues, especially in light of the limitations of signature matching systems. They provide a necessary safety net to ensure that all eligible votes are counted and that voters have the opportunity to address any discrepancies (Source: National Conference of State Legislatures, ""Table 15: States With Signature Cure Processes,"" Jan. 18, 2022). If you have more questions or need further clarification, feel free to ask!","Ballot curing laws in at least 24 states provide a fallback system that allows voters to correct their ballot and have it counted in case a voter signature matching algorithm incorrectly flags their ballot as invalid or if there is another issue with their ballot that cannot be rectified by an election official review. This process ensures that voters have the opportunity to address any issues with their ballot and have their vote counted, as some federal courts have determined that such cure procedures are constitutionally required.",1.0,0.9248881628079485,0.5,0.9999999999666667,0.596967170921338 +How can feedback and red-teaming assess GAI equity and ensure content transparency?,"['• Human / AI: Performed by GAI in combination with specialist or non-specialist human teams. GAI-led red-teaming can be more cost effective than human red-teamers alone. Human or GAI-led AI red-teaming may be better suited for eliciting different types of harms. \nGAI technologies can be leveraged for many applications such as content generation and synthetic data. Some aspects of GAI outputs, such as the production of deepfake content, can challenge our ability to distinguish human-generated content from AI-generated synthetic content. To help manage and mitigate these risks, digital transparency mechanisms like provenance data tracking can trace the origin and history of content. Provenance data tracking and synthetic content detection can help facilitate greater information access about both authentic and synthetic content to users, enabling better knowledge of trustworthiness in AI systems. When combined with other organizational accountability mechanisms, digital content transparency approaches can enable processes to trace negative outcomes back to their source, improve information integrity, and uphold public trust. Provenance data tracking and synthetic content detection mechanisms provide information about the origin and history of content to assist in GAI risk management efforts. Provenance metadata can include information about GAI model developers or creators of GAI content, date/time of creation, location, modifications, and sources. Metadata can be tracked for text, images, videos, audio, and underlying datasets. The implementation of provenance data tracking techniques can' + '50 Participatory Engagement Methods \nOn an ad hoc or more structured basis, organizations can design and use a variety of channels to engage \nexternal stakeholders in product development or review. Focus groups with select experts can provide \nfeedback on a range of issues. Small user studies can provide feedback from representative groups or \npopulations. Anonymous surveys can be used to poll or gauge reactions to specific features. Participatory engagement methods are often less structured than field testing or red teaming, and are more \ncommonly used in early stages of AI or product development. \nField Testing \nField testing involves structured settings to evaluate risks and impacts and to simulate the conditions \nunder which the GAI system will be deployed. Field style tests can be adapted from a focus on user \npreferences and experiences towards AI risks and impacts – both negative and positive. When carried \nout with large groups of users, these tests can provide estimations of the likelihood of risks and impacts \nin real world interactions. \nOrganizations may also collect feedback on outcomes, harms, and user experience directly from users in the production environment after a model has been released, in accordance with human subject \nstandards such as informed consent and compensation. Organizations should follow applicable human \nsubjects research requirements, and best practices such as informed consent and subject compensation, \nwhen implementing feedback activities. \nAI Red-teaming \nAI red-teaming is an evolving practice that references exercises often conducted in a controlled \nenvironment and in collaboration with AI developers building AI models to identify potential adverse \nbehavior or outcomes of a GAI model or system, how they could occur, and stress test safeguards. AI \nred-teaming can be performed before or after AI models or systems are made available to the broader.' + '29 MS-1.1-006 Implement continuous monitoring of GAI system impacts to identify whether GAI outputs are equitable across various sub- populations. Seek active and direct feedback from affected communities via structured feedback mechanisms or red - teaming to monitor and improve outputs. \n\nMS-1.1-008 Define use cases, contexts of use, capabilities, and negative impacts where structured human feedback exercises, e.g., GAI red- teaming, would be most beneficial for GAI risk measurement and management based on the context of use. \n\nMEASURE 1.3: Internal experts who did not serve as front -line developers for the system and/or independent assessors are involved in regular assessments and updates. Domain experts, users, AI Actors external to the team that developed or deployed the AI system, and affected communities are consulted in support of assessments as necessary per organizational risk tolerance. \n\nAction ID Suggested Action GAI Risks \nMS-1.3-001 Define relevant groups of interest (e.g., demographic groups, subject matter experts, experience with GAI technology) within the context of use as part of plans for gathering structured public feedback.']","Feedback and red-teaming can be effectively utilized to assess equity in Generative AI (GAI) systems and ensure transparency in the content produced through the following strategies: + +1. **Diverse Feedback Mechanisms**: Implement structured feedback channels that solicit input from a diverse range of users, including those from marginalized or underrepresented communities. This feedback can help identify potential biases in the GAI outputs and assess whether the content produced is equitable and inclusive. + +2. **Equity-Focused Red-Teaming**: Conduct red-teaming exercises specifically aimed at evaluating the equity of GAI systems. This involves assembling teams that include individuals with expertise in social justice, equity, and diversity to simulate potential adverse outcomes and identify areas where the GAI system may produce biased or harmful content. + +3. **Scenario Testing**: Use red-teaming to create scenarios that reflect real-world contexts in which GAI systems will be deployed. By testing how the system responds to various inputs, teams can assess whether the outputs are fair and equitable across different demographic groups. + +4. **Transparency in Processes**: Ensure that the processes used for generating content are transparent. This includes documenting the data sources, algorithms, and decision-making criteria used in the GAI system. Transparency allows stakeholders to understand how content is produced and to identify any potential biases in the underlying data. + +5. **Regular Audits**: Conduct regular audits of GAI outputs to evaluate their equity and fairness. This can involve analyzing the content for representation, language use, and potential stereotypes. Feedback from these audits can inform necessary adjustments to the GAI system. + +6. **User-Centric Design**: Engage users in the design and development of GAI systems to ensure that their needs and perspectives are considered. User feedback can help identify potential equity issues early in the development process, leading to more inclusive content generation. + +7. **Iterative Improvement**: Use feedback and red-teaming results to iteratively improve the GAI system. By continuously refining the model based on user input and red-teaming findings, organizations can enhance the equity of the content produced and address any identified biases. + +8. **Public Reporting**: Share findings from feedback and red-teaming exercises with the public to promote accountability and transparency. Public reporting can help build trust in the GAI system and demonstrate a commitment to equity and responsible AI use. + +9. **Training and Awareness**: Provide training for developers and stakeholders on the importance of equity in AI systems. Raising awareness about potential biases and the impact of GAI outputs can foster a culture of responsibility and inclusivity within organizations. + +By integrating feedback and red-teaming into the assessment of equity in GAI systems, organizations can ensure that the content produced is fair, transparent, and aligned with ethical standards. This approach not only enhances the quality of GAI outputs but also promotes trust and accountability in AI technologies (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!","Implement continuous monitoring of GAI system impacts to identify whether GAI outputs are equitable across various sub-populations. Seek active and direct feedback from affected communities via structured feedback mechanisms or red-teaming to monitor and improve outputs. Evaluate the quality and integrity of data used in training and the provenance of AI-generated content by employing techniques like chaos engineering and seeking stakeholder feedback. Define use cases, contexts of use, capabilities, and negative impacts where structured human feedback exercises, e.g., GAI red-teaming, would be most beneficial for GAI risk measurement and management based on the context of use. Track and document risks or opportunities related to all GAI risks that cannot be measured quantitatively, including explanations as to why some risks cannot be measured (e.g., due to technological limitations, resource constraints, or trustworthy considerations). Include unmeasured risks in marginal risks.",1.0,0.9171600092824068,0.5,0.9999999999666667,0.6617750657844641 +How can algorithmic discrimination be prevented through proactive measures and equity assessments?,"['Designers, developers, and deployers of automated systems should take proactive and continuous measures to protect individuals and communities from algorithmic discrimination and to use and design systems in an equitable way. This protection should include proactive equity assessments as part of the system design, use of representative data and protection against proxies for demographic features, ensuring accessibility for people with disabilities in design and development, pre-deployment and ongoing disparity testing and mitigation, and clear organizational oversight. Independent evaluation and plain language reporting in the form of an algorithmic impact assessment, including disparity testing results and mitigation information, should be performed and made public whenever possible to confirm these protections.' + '50. Various organizations have offered proposals for how such assessments might be designed. See, e.g., Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, Madeleine Clare Elish, and Jacob Metcalf. Assembling Accountability: Algorithmic Impact Assessment for the Public Interest. Data & Society Research Institute Report. June 29, 2021. https://datasociety.net/library/assembling-accountability-algorithmic-impact-assessment-for-the-public-interest/; Nicol Turner Lee, Paul Resnick, and Genie Barton. Algorithmic bias detection and mitigation: Best practices and policies to reduce consumer harms. Brookings Report. May 22, 2019. https://www.brookings.edu/research/algorithmic-bias-detection-and-mitigation-best-practices-and-policies-to-reduce-consumer-harms/; Andrew D. Selbst. An Institutional View Of Algorithmic Impact Assessments. Harvard Journal of Law & Technology. June 15, 2021. https://ssrn.com/abstract=3867634; Dillon Reisman, Jason Schultz, Kate Crawford, and Meredith Whittaker. Algorithmic Impact Assessments: A Practical Framework for Public Agency Accountability. AI Now Institute Report. April 2018. https://ainowinstitute.org/aiareport2018.pdf' + 'The Equal Employment Opportunity Commission and the Department of Justice have clearly laid out how employers’ use of AI and other automated systems can result in discrimination against job applicants and employees with disabilities.53 The documents explain how employers’ use of software that relies on algorithmic decision-making may violate existing requirements under Title I of the Americans with Disabilities Act (“ADA”). This technical assistance also provides practical tips to employers on how to comply with the ADA, and to job applicants and employees who think that their rights may have been violated.' + '•The National Disabled Law Students Association expressed concerns that individuals with disabilities were more likely to be flagged as potentially suspicious by remote proctoring AI systems because of their disabili- ty-specific access needs such as needing longer breaks or using screen readers or dictation software.45 \n•An algorithm designed to identify patients with high needs for healthcare systematically assigned lower scores (indicating that they were not as high need) to Black patients than to those of white patients, even when those patients had similar numbers of chronic conditions and other markers of health.46 In addition, healthcare clinical algorithms that are used by physicians to guide clinical decisions may include sociodemographic variables that adjust or “correct” the algorithm’s output on the basis of a patient’s race or ethnicity, which can lead to race-based health inequities.47']","To prevent algorithmic discrimination, organizations can implement proactive measures and conduct equity assessments throughout the lifecycle of their automated systems. Here are several strategies to achieve this: + +1. **Proactive Equity Assessments**: Before deploying an algorithm, conduct thorough equity assessments to evaluate its potential impact on different demographic groups. This involves analyzing the data used for training, the algorithm's decision-making processes, and the expected outcomes to identify any biases or disparities. + +2. **Diverse Data Collection**: Ensure that the data used to train algorithms is representative of the diverse populations it will affect. This includes collecting data from various demographic groups to avoid reinforcing existing biases and ensuring that the algorithm performs equitably across different segments of the population. + +3. **Bias Detection Tools**: Utilize bias detection tools and techniques to identify and mitigate biases in algorithms. This can involve statistical tests to measure fairness and equity in the algorithm's outputs, as well as tools that can highlight potential discriminatory patterns in the data. + +4. **Inclusive Design Practices**: Involve diverse stakeholders in the design and development of algorithms. This includes engaging individuals from various backgrounds, including those who may be affected by the algorithm's decisions. Their insights can help identify potential issues and ensure that the system is designed with equity in mind. + +5. **Regular Monitoring and Auditing**: Implement ongoing monitoring and auditing of algorithms after deployment to assess their performance and impact. Regular audits can help identify any emerging biases or discriminatory outcomes, allowing for timely interventions and adjustments. + +6. **Transparency and Explainability**: Ensure that algorithms are transparent and their decision-making processes are explainable. Providing clear documentation about how algorithms work and the data they use can help stakeholders understand potential biases and hold organizations accountable. + +7. **Feedback Mechanisms**: Establish feedback channels for users and affected communities to report issues related to algorithmic discrimination. This feedback can be invaluable for identifying problems that may not be apparent during initial assessments and for making necessary adjustments. + +8. **Training and Awareness**: Provide training for developers, data scientists, and stakeholders on the importance of equity and the risks of algorithmic discrimination. Raising awareness about these issues can foster a culture of responsibility and encourage proactive measures to prevent discrimination. + +9. **Ethical Guidelines and Standards**: Develop and adhere to ethical guidelines and standards for algorithm development and deployment. These guidelines should prioritize fairness, accountability, and transparency, ensuring that organizations commit to preventing algorithmic discrimination. + +10. **Collaboration with Experts**: Collaborate with external experts, including ethicists, sociologists, and civil rights advocates, to gain insights into potential biases and discrimination in algorithms. Their expertise can help organizations identify blind spots and improve their equity assessments. + +By implementing these proactive measures and conducting thorough equity assessments, organizations can significantly reduce the risk of algorithmic discrimination and promote fair and equitable outcomes in their automated systems (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!","Algorithmic discrimination can be prevented through proactive measures and equity assessments by ensuring that automated systems are designed and used in an equitable manner. This includes conducting proactive equity assessments during system design, using representative data, avoiding proxies for demographic features, ensuring accessibility for individuals with disabilities, conducting pre-deployment and ongoing disparity testing, and maintaining clear organizational oversight. Independent evaluation and plain language reporting, such as algorithmic impact assessments that include testing results and mitigation information, should be performed and made public whenever possible to confirm these protections.",1.0,0.9301348671947912,1.0,0.999999999975,0.8243558980075278 +How can system design ensure behavior transparency in high-risk settings while meeting expectations for automated systems?,"['In settings where the consequences are high as determined by a risk assessment, or extensive oversight is expected (e.g., in criminal justice or some public sector settings), explanatory mechanisms should be built into the system design so that the system’s full behavior can be explained in advance (i.e., only fully transparent models should be used), rather than as an after-the-decision interpretation. In other settings, the extent of explanation provided should be tailored to the risk level. The explanation provided by a system should accurately reflect the factors and the influences that led to a particular decision, and should be meaningful for the particular customization based on purpose, target, and level of risk. While approximation and simplification may be necessary for the system to succeed based on the explanatory purpose and target of the explanation, or to account for the risk of fraud or other concerns related to revealing decision-making information, such simplifications should be done in a scientifically supportable way. Where appropriate based on the explanatory system, error ranges for the explanation should be calculated and included in the explanation, with the choice of presentation of such information balanced with usability and overall interface complexity concerns.' + 'Risk identification and mitigation. Before deployment, and in a proactive and ongoing manner, poten -\ntial risks of the automated system should be identified and mitigated. Identified risks should focus on the potential for meaningful impact on people’s rights, opportunities, or access and include those to impacted communities that may not be direct users of the automated system, risks resulting from purposeful misuse of the system, and other concerns identified via the consultation process. Assessment and, where possible, mea\n-\nsurement of the impact of risks should be included and balanced such that high impact risks receive attention and mitigation proportionate with those impacts. Automated systems with the intended purpose of violating the safety of others should not be developed or used; systems with such safety violations as identified unin\n-\ntended consequences should not be used until the risk can be mitigated. Ongoing risk mitigation may necessi -\ntate rollback or significant modification to a launched automated system.' + 'Meaningful access to examine the system. Designers, developers, and deployers of automated systems should consider limited waivers of confidentiality (including those related to trade secrets) where necessary in order to provide meaningful oversight of systems used in sensitive domains, incorporating measures to protect intellectual property and trade secrets from unwarranted disclosure as appropriate. This includes (potentially private and protected) meaningful access to source code, documentation, and related data during any associated legal discovery, subject to effective confidentiality or court orders. Such meaningful access should include (but is not limited to) adhering to the principle on Notice and Explanation using the highest level of risk so the system is designed with built-in explanations; such systems should use fully-transparent models where the model itself can be understood by people needing to directly examine it.' + 'Derived data sources tracked and reviewed carefully. Data that is derived from other data through the use of algorithms, such as data derived or inferred from prior model outputs, should be identified and tracked, e.g., via a specialized type in a data schema. Derived data should be viewed as potentially high-risk inputs that may lead to feedback loops, compounded harm, or inaccurate results. Such sources should be carefully validated against the risk of collateral consequences. \n\nData reuse limits in sensitive domains. Data reuse, and especially data reuse in a new context, can result in the spreading and scaling of harms. Data from some domains, including criminal justice data and data indicating adverse outcomes in domains such as finance, employment, and housing, is especially sensitive, and in some cases its reuse is limited by law. Accordingly, such data should be subject to extra oversight to ensure safety and efficacy. Data reuse of sensitive domain data in other contexts (e.g., criminal data reuse for civil legal matters or private sector use) should only occur where use of such data is legally authorized and, after examination, has benefits for those impacted by the system that outweigh identified risks and, as appropriate, reasonable measures have been implemented to mitigate the identified risks. Such data should be clearly labeled to identify contexts for limited reuse based on sensitivity. Where possible, aggregated datasets may be useful for replacing individual-level sensitive data. \n\nDemonstrate the safety and effectiveness of the system \nIndependent evaluation. Automated systems should be designed to allow for independent evaluation (e.g.,']","To ensure behavior transparency in high-risk settings while meeting expectations for automated systems, system design should incorporate several key principles and practices: + +1. **Use of Explainable Models**: Design systems using algorithms that are inherently interpretable. This means selecting models that allow users to understand how decisions are made without requiring complex post-hoc explanations. For example, decision trees or linear regression models can provide clearer insights compared to more opaque models like deep neural networks. + +2. **Built-in Explanation Features**: Integrate features that provide real-time explanations of the system's behavior. This could include visualizations of decision pathways, confidence scores for predictions, or summaries of the factors influencing a decision. These features should be user-friendly and accessible to stakeholders. + +3. **Documentation and Training**: Provide comprehensive documentation that outlines how the system operates, including its limitations and the rationale behind its design choices. Additionally, training for users on how to interpret the explanations and understand the system's behavior is crucial for effective use. + +4. **Regular Updates and Assessments**: Establish processes for regularly assessing and updating the explanatory mechanisms as the system evolves. This includes revisiting the explanations provided to ensure they remain relevant and accurate as the model is refined or as new data is introduced. + +5. **User-Centric Design**: Engage with end-users during the design process to understand their needs and expectations regarding explanations. Tailoring the level of detail and type of explanations to the users' expertise and context can enhance understanding and trust. + +6. **Feedback Loops**: Implement mechanisms for users to provide feedback on the explanations they receive. This feedback can be used to improve the clarity and usefulness of the explanations over time, ensuring they meet user needs. + +7. **Risk-Based Tailoring**: Adjust the extent and type of explanations based on the risk level associated with the system's decisions. For high-risk applications, more detailed and comprehensive explanations should be provided, while lower-risk applications may require simpler explanations. + +8. **Meaningful Access for Oversight**: Ensure that designers, developers, and deployers of the system provide meaningful access to the underlying data, algorithms, and decision-making processes. This may involve limited waivers of confidentiality to allow for oversight while protecting sensitive information. + +9. **Stakeholder Engagement**: Involve relevant stakeholders, including legal, compliance, and privacy experts, in the development and implementation of transparency measures. Their insights can help ensure that the measures are comprehensive and aligned with best practices. + +10. **Error Analysis and Reporting**: Establish protocols for analyzing errors and reporting them transparently. Users should be informed about the types of errors that can occur and how they are addressed, which can help manage expectations and improve trust in the system. + +By integrating these strategies into the system design, organizations can enhance transparency and accountability in high-risk environments, ultimately fostering trust in AI technologies and ensuring that they meet user expectations (Source: [document name], p. [specific page number]). If you have more questions or need further clarification, feel free to ask!","In settings where the consequences are high as determined by a risk assessment, or extensive oversight is expected (e.g., in criminal justice or some public sector settings), explanatory mechanisms should be built into the system design so that the system’s full behavior can be explained in advance (i.e., only fully transparent models should be used), rather than as an after-the-decision interpretation. The extent of explanation provided should be tailored to the risk level.",0.8780487804878049,0.9463660510234914,1.0,0.999999999975,0.29225269309241797