sentence-transformers safetensors bert sentence-similarity feature-extraction generated_from_trainer dataset_size:274 loss:MultipleNegativesRankingLoss arxiv:1908.10084 arxiv:1705.00652 base_model:sentence-transformers/all-MiniLM-L6-v2 base_model:finetune:sentence-transformers/all-MiniLM-L6-v2 autotrain_compatible text-embeddings-inference endpoints_compatible region:us
---
base_model: sentence-transformers/all-MiniLM-L6-v2
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:274
- loss:MultipleNegativesRankingLoss
widget:
- source_sentence: What are the key considerations for conducting diligence on training
    data use in relation to intellectual property and privacy risks?
  sentences:
  - "APPENDIX\n•\nJulia Simon-Mishel, Supervising Attorney, Philadelphia Legal Assistance\n\
    •\nDr. Zachary Mahafza, Research & Data Analyst, Southern Poverty Law Center\n\
    •\nJ. Khadijah Abdurahman, Tech Impact Network Research Fellow, AI Now Institute,\
    \ UCLA C2I1, and\nUWA Law School\nPanelists separately described the increasing\
    \ scope of technology use in providing for social welfare, including \nin fraud\
    \ detection, digital ID systems, and other methods focused on improving efficiency\
    \ and reducing cost. \nHowever, various panelists individually cautioned that\
    \ these systems may reduce burden for government \nagencies by increasing the\
    \ burden and agency of people using and interacting with these technologies. \n\
    Additionally, these systems can produce feedback loops and compounded harm, collecting\
    \ data from \ncommunities and using it to reinforce inequality. Various panelists\
    \ suggested that these harms could be \nmitigated by ensuring community input\
    \ at the beginning of the design process, providing ways to opt out of \nthese\
    \ systems and use associated human-driven mechanisms instead, ensuring timeliness\
    \ of benefit payments, \nand providing clear notice about the use of these systems\
    \ and clear explanations of how and what the \ntechnologies are doing. Some panelists\
    \ suggested that technology should be used to help people receive \nbenefits,\
    \ e.g., by pushing benefits to those in need and ensuring automated decision-making\
    \ systems are only \nused to provide a positive outcome; technology shouldn't\
    \ be used to take supports away from people who need \nthem. \nPanel 6: The Healthcare\
    \ System. This event explored current and emerging uses of technology in the \n\
    healthcare system and consumer products related to health. \nWelcome:\n•\nAlondra\
    \ Nelson, Deputy Director for Science and Society, White House Office of Science\
    \ and Technology\nPolicy\n•\nPatrick Gaspard, President and CEO, Center for American\
    \ Progress\nModerator: Micky Tripathi, National Coordinator for Health Information\
    \ Technology, U.S Department of \nHealth and Human Services. \nPanelists: \n•\n\
    Mark Schneider, Health Innovation Advisor, ChristianaCare\n•\nZiad Obermeyer,\
    \ Blue Cross of California Distinguished Associate Professor of Policy and Management,\n\
    University of California, Berkeley School of Public Health\n•\nDorothy Roberts,\
    \ George A. Weiss University Professor of Law and Sociology and the Raymond Pace\
    \ and\nSadie Tanner Mossell Alexander Professor of Civil Rights, University of\
    \ Pennsylvania\n•\nDavid Jones, A. Bernard Ackerman Professor of the Culture of\
    \ Medicine, Harvard University\n•\nJamila Michener, Associate Professor of Government,\
    \ Cornell University; Co-Director, Cornell Center for\nHealth Equity­\nPanelists\
    \ discussed the impact of new technologies on health disparities; healthcare access,\
    \ delivery, and \noutcomes; and areas ripe for research and policymaking. Panelists\
    \ discussed the increasing importance of tech-\nnology as both a vehicle to deliver\
    \ healthcare and a tool to enhance the quality of care. On the issue of \ndelivery,\
    \ various panelists pointed to a number of concerns including access to and expense\
    \ of broadband \nservice, the privacy concerns associated with telehealth systems,\
    \ the expense associated with health \nmonitoring devices, and how this can exacerbate\
    \ equity issues.  On the issue of technology enhanced care, \nsome panelists spoke\
    \ extensively about the way in which racial biases and the use of race in medicine\
    \ \nperpetuate harms and embed prior discrimination, and the importance of ensuring\
    \ that the technologies used \nin medical care were accountable to the relevant\
    \ stakeholders. Various panelists emphasized the importance \nof having the voices\
    \ of those subjected to these technologies be heard.\n59"
  - "27 \nMP-4.1-010 \nConduct appropriate diligence on training data use to assess\
    \ intellectual property, \nand privacy, risks, including to examine whether use\
    \ of proprietary or sensitive \ntraining data is consistent with applicable laws.\
    \  \nIntellectual Property; Data Privacy \nAI Actor Tasks: Governance and Oversight,\
    \ Operation and Monitoring, Procurement, Third-party entities \n \nMAP 5.1: Likelihood\
    \ and magnitude of each identiﬁed impact (both potentially beneﬁcial and harmful)\
    \ based on expected use, past \nuses of AI systems in similar contexts, public\
    \ incident reports, feedback from those external to the team that developed or\
    \ deployed \nthe AI system, or other data are identiﬁed and documented. \nAction\
    \ ID \nSuggested Action \nGAI Risks \nMP-5.1-001 Apply TEVV practices for content\
    \ provenance (e.g., probing a system's synthetic \ndata generation capabilities\
    \ for potential misuse or vulnerabilities. \nInformation Integrity; Information\
    \ \nSecurity \nMP-5.1-002 \nIdentify potential content provenance harms of GAI,\
    \ such as misinformation or \ndisinformation, deepfakes, including NCII, or tampered\
    \ content. Enumerate and \nrank risks based on their likelihood and potential\
    \ impact, and determine how well \nprovenance solutions address speciﬁc risks\
    \ and/or harms. \nInformation Integrity; Dangerous, \nViolent, or Hateful Content;\
    \ \nObscene, Degrading, and/or \nAbusive Content \nMP-5.1-003 \nConsider disclosing\
    \ use of GAI to end users in relevant contexts, while considering \nthe objective\
    \ of disclosure, the context of use, the likelihood and magnitude of the \nrisk\
    \ posed, the audience of the disclosure, as well as the frequency of the \ndisclosures.\
    \ \nHuman-AI Conﬁguration \nMP-5.1-004 Prioritize GAI structured public feedback\
    \ processes based on risk assessment \nestimates. \nInformation Integrity; CBRN\
    \ \nInformation or Capabilities; \nDangerous, Violent, or Hateful \nContent; Harmful\
    \ Bias and \nHomogenization \nMP-5.1-005 Conduct adversarial role-playing exercises,\
    \ GAI red-teaming, or chaos testing to \nidentify anomalous or unforeseen failure\
    \ modes. \nInformation Security \nMP-5.1-006 \nProﬁle threats and negative impacts\
    \ arising from GAI systems interacting with, \nmanipulating, or generating content,\
    \ and outlining known and potential \nvulnerabilities and the likelihood of their\
    \ occurrence. \nInformation Security \nAI Actor Tasks: AI Deployment, AI Design,\
    \ AI Development, AI Impact Assessment, Aﬀected Individuals and Communities, End-\n\
    Users, Operation and Monitoring"
  - "18 \nGOVERN 3.2: Policies and procedures are in place to deﬁne and diﬀerentiate\
    \ roles and responsibilities for human-AI conﬁgurations \nand oversight of AI\
    \ systems. \nAction ID \nSuggested Action \nGAI Risks \nGV-3.2-001 \nPolicies\
    \ are in place to bolster oversight of GAI systems with independent \nevaluations\
    \ or assessments of GAI models or systems where the type and \nrobustness of evaluations\
    \ are proportional to the identiﬁed risks. \nCBRN Information or Capabilities;\
    \ \nHarmful Bias and Homogenization \nGV-3.2-002 \nConsider adjustment of organizational\
    \ roles and components across lifecycle \nstages of large or complex GAI systems,\
    \ including: Test and evaluation, validation, \nand red-teaming of GAI systems;\
    \ GAI content moderation; GAI system \ndevelopment and engineering; Increased\
    \ accessibility of GAI tools, interfaces, and \nsystems, Incident response and\
    \ containment. \nHuman-AI Conﬁguration; \nInformation Security; Harmful Bias \n\
    and Homogenization \nGV-3.2-003 \nDeﬁne acceptable use policies for GAI interfaces,\
    \ modalities, and human-AI \nconﬁgurations (i.e., for chatbots and decision-making\
    \ tasks), including criteria for \nthe kinds of queries GAI applications should\
    \ refuse to respond to.  \nHuman-AI Conﬁguration \nGV-3.2-004 \nEstablish policies\
    \ for user feedback mechanisms for GAI systems which include \nthorough instructions\
    \ and any mechanisms for recourse. \nHuman-AI Conﬁguration  \nGV-3.2-005 \nEngage\
    \ in threat modeling to anticipate potential risks from GAI systems. \nCBRN Information\
    \ or Capabilities; \nInformation Security \nAI Actors: AI Design \n \nGOVERN 4.1:\
    \ Organizational policies and practices are in place to foster a critical thinking\
    \ and safety-ﬁrst mindset in the design, \ndevelopment, deployment, and uses of\
    \ AI systems to minimize potential negative impacts. \nAction ID \nSuggested Action\
    \ \nGAI Risks \nGV-4.1-001 \nEstablish policies and procedures that address continual\
    \ improvement processes \nfor GAI risk measurement. Address general risks associated\
    \ with a lack of \nexplainability and transparency in GAI systems by using ample\
    \ documentation and \ntechniques such as: application of gradient-based attributions,\
    \ occlusion/term \nreduction, counterfactual prompts and prompt engineering, and\
    \ analysis of \nembeddings; Assess and update risk measurement approaches at regular\
    \ \ncadences. \nConfabulation \nGV-4.1-002 \nEstablish policies, procedures, and\
    \ processes detailing risk measurement in \ncontext of use with standardized measurement\
    \ protocols and structured public \nfeedback exercises such as AI red-teaming\
    \ or independent external evaluations. \nCBRN Information and Capability; \nValue\
    \ Chain and Component \nIntegration"
- source_sentence: What should individuals be able to do when encountering problems
    with automated systems, according to the context provided?
  sentences:
  - "6 \n2.2. Confabulation \n“Confabulation” refers to a phenomenon in which GAI\
    \ systems generate and conﬁdently present \nerroneous or false content in response\
    \ to prompts. Confabulations also include generated outputs that \ndiverge from\
    \ the prompts or other input or that contradict previously generated statements\
    \ in the same \ncontext. These phenomena are colloquially also referred to as\
    \ “hallucinations” or “fabrications.” \nConfabulations can occur across GAI outputs\
    \ and contexts.9,10 Confabulations are a natural result of the \nway generative\
    \ models are designed: they generate outputs that approximate the statistical\
    \ distribution \nof their training data; for example, LLMs predict the next token\
    \ or word in a sentence or phrase. While \nsuch statistical prediction can produce\
    \ factually accurate and consistent outputs, it can also produce \noutputs that\
    \ are factually inaccurate or internally inconsistent. This dynamic is particularly\
    \ relevant when \nit comes to open-ended prompts for long-form responses and in\
    \ domains which require highly \ncontextual and/or domain expertise.  \nRisks\
    \ from confabulations may arise when users believe false content – often due to\
    \ the conﬁdent nature \nof the response – leading users to act upon or promote\
    \ the false information. This poses a challenge for \nmany real-world applications,\
    \ such as in healthcare, where a confabulated summary of patient \ninformation\
    \ reports could cause doctors to make incorrect diagnoses and/or recommend the\
    \ wrong \ntreatments. Risks of confabulated content may be especially important\
    \ to monitor when integrating GAI \ninto applications involving consequential\
    \ decision making. \nGAI outputs may also include confabulated logic or citations\
    \ that purport to justify or explain the \nsystem’s answer, which may further\
    \ mislead humans into inappropriately trusting the system’s output. \nFor instance,\
    \ LLMs sometimes provide logical steps for how they arrived at an answer even\
    \ when the \nanswer itself is incorrect. Similarly, an LLM could falsely assert\
    \ that it is human or has human traits, \npotentially deceiving humans into believing\
    \ they are speaking with another human. \nThe extent to which humans can be deceived\
    \ by LLMs, the mechanisms by which this may occur, and the \npotential risks from\
    \ adversarial prompting of such behavior are emerging areas of study. Given the\
    \ wide \nrange of downstream impacts of GAI, it is diﬃcult to estimate the downstream\
    \ scale and impact of \nconfabulations. \nTrustworthy AI Characteristics: Fair\
    \ with Harmful Bias Managed, Safe, Valid and Reliable, Explainable \nand Interpretable\
    \ \n2.3. Dangerous, Violent, or Hateful Content \nGAI systems can produce content\
    \ that is inciting, radicalizing, or threatening, or that gloriﬁes violence, \n\
    with greater ease and scale than other technologies. LLMs have been reported to\
    \ generate dangerous or \nviolent recommendations, and some models have generated\
    \ actionable instructions for dangerous or \n \n \n9 Confabulations of falsehoods\
    \ are most commonly a problem for text-based outputs; for audio, image, or video\
    \ \ncontent, creative generation of non-factual content can be a desired behavior.\
    \  \n10 For example, legal confabulations have been shown to be pervasive in current\
    \ state-of-the-art LLMs. See also, \ne.g.,"
  - "SECTION TITLE\nHUMAN ALTERNATIVES, CONSIDERATION, AND FALLBACK\nYou should be\
    \ able to opt out, where appropriate, and have access to a person who can quickly\
    \ \nconsider and remedy problems you encounter. You should be able to opt out\
    \ from automated systems in \nfavor of a human alternative, where appropriate.\
    \ Appropriateness should be determined based on reasonable \nexpectations in a\
    \ given context and with a focus on ensuring broad accessibility and protecting\
    \ the public from \nespecially harmful impacts. In some cases, a human or other\
    \ alternative may be required by law. You should have \naccess to timely human\
    \ consideration and remedy by a fallback and escalation process if an automated\
    \ system \nfails, it produces an error, or you would like to appeal or contest\
    \ its impacts on you. Human consideration and \nfallback should be accessible,\
    \ equitable, effective, maintained, accompanied by appropriate operator training,\
    \ and \nshould not impose an unreasonable burden on the public. Automated systems\
    \ with an intended use within sensi­\ntive domains, including, but not limited\
    \ to, criminal justice, employment, education, and health, should additional­\n\
    ly be tailored to the purpose, provide meaningful access for oversight, include\
    \ training for any people interacting \nwith the system, and incorporate human\
    \ consideration for adverse or high-risk decisions. Reporting that includes \n\
    a description of these human governance processes and assessment of their timeliness,\
    \ accessibility, outcomes, \nand effectiveness should be made public whenever\
    \ possible. \nDefinitions for key terms in The Blueprint for an AI Bill of Rights\
    \ can be found in Applying the Blueprint for an AI Bill of Rights. \nAccompanying\
    \ analysis and tools for actualizing each principle can be found in the Technical\
    \ Companion. \n7"
  - "FROM \nPRINCIPLES \nTO PRACTICE \nA TECHINCAL COMPANION TO\nTHE Blueprint for\
    \ an \nAI BILL OF RIGHTS\n12"
- source_sentence: How did the White House Office of Science and Technology Policy
    gather input from the American public regarding algorithmic and data-driven harms?
  sentences:
  - "23 \nMP-1.1-002 \nDetermine and document the expected and acceptable GAI system\
    \ context of \nuse in collaboration with socio-cultural and other domain experts,\
    \ by assessing: \nAssumptions and limitations; Direct value to the organization;\
    \ Intended \noperational environment and observed usage patterns; Potential positive\
    \ and \nnegative impacts to individuals, public safety, groups, communities, \n\
    organizations, democratic institutions, and the physical environment; Social \n\
    norms and expectations. \nHarmful Bias and Homogenization \nMP-1.1-003 \nDocument\
    \ risk measurement plans to address identiﬁed risks. Plans may \ninclude, as applicable:\
    \ Individual and group cognitive biases (e.g., conﬁrmation \nbias, funding bias,\
    \ groupthink) for AI Actors involved in the design, \nimplementation, and use\
    \ of GAI systems; Known past GAI system incidents and \nfailure modes; In-context\
    \ use and foreseeable misuse, abuse, and oﬀ-label use; \nOver reliance on quantitative\
    \ metrics and methodologies without suﬃcient \nawareness of their limitations\
    \ in the context(s) of use; Standard measurement \nand structured human feedback\
    \ approaches; Anticipated human-AI \nconﬁgurations. \nHuman-AI Conﬁguration; Harmful\
    \ \nBias and Homogenization; \nDangerous, Violent, or Hateful \nContent \nMP-1.1-004\
    \ \nIdentify and document foreseeable illegal uses or applications of the GAI\
    \ system \nthat surpass organizational risk tolerances. \nCBRN Information or\
    \ Capabilities; \nDangerous, Violent, or Hateful \nContent; Obscene, Degrading,\
    \ \nand/or Abusive Content \nAI Actor Tasks: AI Deployment \n \nMAP 1.2: Interdisciplinary\
    \ AI Actors, competencies, skills, and capacities for establishing context reﬂect\
    \ demographic diversity and \nbroad domain and user experience expertise, and\
    \ their participation is documented. Opportunities for interdisciplinary \ncollaboration\
    \ are prioritized. \nAction ID \nSuggested Action \nGAI Risks \nMP-1.2-001 \n\
    Establish and empower interdisciplinary teams that reﬂect a wide range of \ncapabilities,\
    \ competencies, demographic groups, domain expertise, educational \nbackgrounds,\
    \ lived experiences, professions, and skills across the enterprise to \ninform\
    \ and conduct risk measurement and management functions. \nHuman-AI Conﬁguration;\
    \ Harmful \nBias and Homogenization \nMP-1.2-002 \nVerify that data or benchmarks\
    \ used in risk measurement, and users, \nparticipants, or subjects involved in\
    \ structured GAI public feedback exercises \nare representative of diverse in-context\
    \ user populations. \nHuman-AI Conﬁguration; Harmful \nBias and Homogenization\
    \ \nAI Actor Tasks: AI Deployment"
  - "49 \nearly lifecycle TEVV approaches are developed and matured for GAI, organizations\
    \ may use \nrecommended “pre-deployment testing” practices to measure performance,\
    \ capabilities, limits, risks, \nand impacts. This section describes risk measurement\
    \ and estimation as part of pre-deployment TEVV, \nand examines the state of play\
    \ for pre-deployment testing methodologies.  \nLimitations of Current Pre-deployment\
    \ Test Approaches \nCurrently available pre-deployment TEVV processes used for\
    \ GAI applications may be inadequate, non-\nsystematically applied, or fail to\
    \ reﬂect or mismatched to deployment contexts. For example, the \nanecdotal testing\
    \ of GAI system capabilities through video games or standardized tests designed\
    \ for \nhumans (e.g., intelligence tests, professional licensing exams) does not\
    \ guarantee GAI system validity or \nreliability in those domains. Similarly,\
    \ jailbreaking or prompt engineering tests may not systematically \nassess validity\
    \ or reliability risks.  \nMeasurement gaps can arise from mismatches between\
    \ laboratory and real-world settings. Current \ntesting approaches often remain\
    \ focused on laboratory conditions or restricted to benchmark test \ndatasets\
    \ and in silico techniques that may not extrapolate well to—or directly assess\
    \ GAI impacts in real-\nworld conditions. For example, current measurement gaps\
    \ for GAI make it diﬃcult to precisely estimate \nits potential ecosystem-level\
    \ or longitudinal risks and related political, social, and economic impacts. \n\
    Gaps between benchmarks and real-world use of GAI systems may likely be exacerbated\
    \ due to prompt \nsensitivity and broad heterogeneity of contexts of use. \nA.1.5.\
    \ Structured Public Feedback \nStructured public feedback can be used to evaluate\
    \ whether GAI systems are performing as intended \nand to calibrate and verify\
    \ traditional measurement methods. Examples of structured feedback include, \n\
    but are not limited to: \n• \nParticipatory Engagement Methods: Methods used to\
    \ solicit feedback from civil society groups, \naﬀected communities, and users,\
    \ including focus groups, small user studies, and surveys. \n• \nField Testing:\
    \ Methods used to determine how people interact with, consume, use, and make \n\
    sense of AI-generated information, and subsequent actions and eﬀects, including\
    \ UX, usability, \nand other structured, randomized experiments.  \n• \nAI Red-teaming:\
    \ A structured testing exercise used to probe an AI system to ﬁnd ﬂaws and \n\
    vulnerabilities such as inaccurate, harmful, or discriminatory outputs, often\
    \ in a controlled \nenvironment and in collaboration with system developers. \n\
    Information gathered from structured public feedback can inform design, implementation,\
    \ deployment \napproval, maintenance, or decommissioning decisions. Results and\
    \ insights gleaned from these exercises \ncan serve multiple purposes, including\
    \ improving data quality and preprocessing, bolstering governance \ndecision making,\
    \ and enhancing system documentation and debugging practices. When implementing\
    \ \nfeedback activities, organizations should follow human subjects research requirements\
    \ and best \npractices such as informed consent and subject compensation."
  - "ABOUT THIS FRAMEWORK­­­­­\nThe Blueprint for an AI Bill of Rights is a set of\
    \ five principles and associated practices to help guide the \ndesign, use, and\
    \ deployment of automated systems to protect the rights of the American public\
    \ in the age of \nartificial intel-ligence. Developed through extensive consultation\
    \ with the American public, these principles are \na blueprint for building and\
    \ deploying automated systems that are aligned with democratic values and protect\
    \ \ncivil rights, civil liberties, and privacy. The Blueprint for an AI Bill of\
    \ Rights includes this Foreword, the five \nprinciples, notes on Applying the\
    \ The Blueprint for an AI Bill of Rights, and a Technical Companion that gives\
    \ \nconcrete steps that can be taken by many kinds of organizations—from governments\
    \ at all levels to companies of \nall sizes—to uphold these values. Experts from\
    \ across the private sector, governments, and international \nconsortia have published\
    \ principles and frameworks to guide the responsible use of automated systems;\
    \ this \nframework provides a national values statement and toolkit that is sector-agnostic\
    \ to inform building these \nprotections into policy, practice, or the technological\
    \ design process.  Where existing law or policy—such as \nsector-specific privacy\
    \ laws and oversight requirements—do not already provide guidance, the Blueprint\
    \ for an \nAI Bill of Rights should be used to inform policy decisions.\nLISTENING\
    \ TO THE AMERICAN PUBLIC\nThe White House Office of Science and Technology Policy\
    \ has led a year-long process to seek and distill input \nfrom people across the\
    \ country—from impacted communities and industry stakeholders to technology develop-\n\
    ers and other experts across fields and sectors, as well as policymakers throughout\
    \ the Federal government—on \nthe issue of algorithmic and data-driven harms and\
    \ potential remedies. Through panel discussions, public listen-\ning sessions,\
    \ meetings, a formal request for information, and input to a publicly accessible\
    \ and widely-publicized \nemail address, people throughout the United States,\
    \ public servants across Federal agencies, and members of the \ninternational\
    \ community spoke up about both the promises and potential harms of these technologies,\
    \ and \nplayed a central role in shaping the Blueprint for an AI Bill of Rights.\
    \ The core messages gleaned from these \ndiscussions include that AI has transformative\
    \ potential to improve Americans’ lives, and that preventing the \nharms of these\
    \ technologies is both necessary and achievable. The Appendix includes a full\
    \ list of public engage-\nments. \n4"
- source_sentence: What are the suggested actions for establishing transparency policies
    related to GAI applications?
  sentences:
  - "42 \nMG-2.4-002 \nEstablish and maintain procedures for escalating GAI system\
    \ incidents to the \norganizational risk management authority when speciﬁc criteria\
    \ for deactivation \nor disengagement is met for a particular context of use or\
    \ for the GAI system as a \nwhole. \nInformation Security \nMG-2.4-003 \nEstablish\
    \ and maintain procedures for the remediation of issues which trigger \nincident\
    \ response processes for the use of a GAI system, and provide stakeholders \n\
    timelines associated with the remediation plan. \nInformation Security \n \nMG-2.4-004\
    \ Establish and regularly review speciﬁc criteria that warrants the deactivation\
    \ of \nGAI systems in accordance with set risk tolerances and appetites. \nInformation\
    \ Security \n \nAI Actor Tasks: AI Deployment, Governance and Oversight, Operation\
    \ and Monitoring \n \nMANAGE 3.1: AI risks and beneﬁts from third-party resources\
    \ are regularly monitored, and risk controls are applied and \ndocumented. \n\
    Action ID \nSuggested Action \nGAI Risks \nMG-3.1-001 \nApply organizational risk\
    \ tolerances and controls (e.g., acquisition and \nprocurement processes; assessing\
    \ personnel credentials and qualiﬁcations, \nperforming background checks; ﬁltering\
    \ GAI input and outputs, grounding, ﬁne \ntuning, retrieval-augmented generation)\
    \ to third-party GAI resources: Apply \norganizational risk tolerance to the utilization\
    \ of third-party datasets and other \nGAI resources; Apply organizational risk\
    \ tolerances to ﬁne-tuned third-party \nmodels; Apply organizational risk tolerance\
    \ to existing third-party models \nadapted to a new domain; Reassess risk measurements\
    \ after ﬁne-tuning third-\nparty GAI models. \nValue Chain and Component \nIntegration;\
    \ Intellectual Property \nMG-3.1-002 \nTest GAI system value chain risks (e.g.,\
    \ data poisoning, malware, other software \nand hardware vulnerabilities; labor\
    \ practices; data privacy and localization \ncompliance; geopolitical alignment).\
    \ \nData Privacy; Information Security; \nValue Chain and Component \nIntegration;\
    \ Harmful Bias and \nHomogenization \nMG-3.1-003 \nRe-assess model risks after\
    \ ﬁne-tuning or retrieval-augmented generation \nimplementation and for any third-party\
    \ GAI models deployed for applications \nand/or use cases that were not evaluated\
    \ in initial testing. \nValue Chain and Component \nIntegration \nMG-3.1-004 \n\
    Take reasonable measures to review training data for CBRN information, and \n\
    intellectual property, and where appropriate, remove it. Implement reasonable\
    \ \nmeasures to prevent, ﬂag, or take other action in response to outputs that\
    \ \nreproduce particular training data (e.g., plagiarized, trademarked, patented,\
    \ \nlicensed content or trade secret material). \nIntellectual Property; CBRN\
    \ \nInformation or Capabilities"
  - "DATA PRIVACY \nEXTRA PROTECTIONS FOR DATA RELATED TO SENSITIVE\nDOMAINS\n•\n\
    Continuous positive airway pressure machines gather data for medical purposes,\
    \ such as diagnosing sleep\napnea, and send usage data to a patient’s insurance\
    \ company, which may subsequently deny coverage for the\ndevice based on usage\
    \ data. Patients were not aware that the data would be used in this way or monitored\n\
    by anyone other than their doctor.70 \n•\nA department store company used predictive\
    \ analytics applied to collected consumer data to determine that a\nteenage girl\
    \ was pregnant, and sent maternity clothing ads and other baby-related advertisements\
    \ to her\nhouse, revealing to her father that she was pregnant.71\n•\nSchool audio\
    \ surveillance systems monitor student conversations to detect potential \"stress\
    \ indicators\" as\na warning of potential violence.72 Online proctoring systems\
    \ claim to detect if a student is cheating on an\nexam using biometric markers.73\
    \ These systems have the potential to limit student freedom to express a range\n\
    of emotions at school and may inappropriately flag students with disabilities\
    \ who need accommodations or\nuse screen readers or dictation software as cheating.74\n\
    •\nLocation data, acquired from a data broker, can be used to identify people\
    \ who visit abortion clinics.75\n•\nCompanies collect student data such as demographic\
    \ information, free or reduced lunch status, whether\nthey've used drugs, or whether\
    \ they've expressed interest in LGBTQI+ groups, and then use that data to \nforecast\
    \ student success.76 Parents and education experts have expressed concern about\
    \ collection of such\nsensitive data without express parental consent, the lack\
    \ of transparency in how such data is being used, and\nthe potential for resulting\
    \ discriminatory impacts.\n• Many employers transfer employee data to third party\
    \ job verification services. This information is then used\nby potential future\
    \ employers, banks, or landlords. In one case, a former employee alleged that\
    \ a\ncompany supplied false data about her job title which resulted in a job offer\
    \ being revoked.77\n37"
  - "14 \nGOVERN 1.2: The characteristics of trustworthy AI are integrated into organizational\
    \ policies, processes, procedures, and practices. \nAction ID \nSuggested Action\
    \ \nGAI Risks \nGV-1.2-001 \nEstablish transparency policies and processes for\
    \ documenting the origin and \nhistory of training data and generated data for\
    \ GAI applications to advance digital \ncontent transparency, while balancing\
    \ the proprietary nature of training \napproaches. \nData Privacy; Information\
    \ \nIntegrity; Intellectual Property \nGV-1.2-002 \nEstablish policies to evaluate\
    \ risk-relevant capabilities of GAI and robustness of \nsafety measures, both\
    \ prior to deployment and on an ongoing basis, through \ninternal and external\
    \ evaluations. \nCBRN Information or Capabilities; \nInformation Security \nAI\
    \ Actor Tasks: Governance and Oversight \n \nGOVERN 1.3: Processes, procedures,\
    \ and practices are in place to determine the needed level of risk management\
    \ activities based \non the organization’s risk tolerance. \nAction ID \nSuggested\
    \ Action \nGAI Risks \nGV-1.3-001 \nConsider the following factors when updating\
    \ or deﬁning risk tiers for GAI: Abuses \nand impacts to information integrity;\
    \ Dependencies between GAI and other IT or \ndata systems; Harm to fundamental\
    \ rights or public safety; Presentation of \nobscene, objectionable, oﬀensive,\
    \ discriminatory, invalid or untruthful output; \nPsychological impacts to humans\
    \ (e.g., anthropomorphization, algorithmic \naversion, emotional entanglement);\
    \ Possibility for malicious use; Whether the \nsystem introduces signiﬁcant new\
    \ security vulnerabilities; Anticipated system \nimpact on some groups compared\
    \ to others; Unreliable decision making \ncapabilities, validity, adaptability,\
    \ and variability of GAI system performance over \ntime. \nInformation Integrity;\
    \ Obscene, \nDegrading, and/or Abusive \nContent; Value Chain and \nComponent\
    \ Integration; Harmful \nBias and Homogenization; \nDangerous, Violent, or Hateful\
    \ \nContent; CBRN Information or \nCapabilities \nGV-1.3-002 \nEstablish minimum\
    \ thresholds for performance or assurance criteria and review as \npart of deployment\
    \ approval (“go/”no-go”) policies, procedures, and processes, \nwith reviewed\
    \ processes and approval thresholds reﬂecting measurement of GAI \ncapabilities\
    \ and risks. \nCBRN Information or Capabilities; \nConfabulation; Dangerous, \n\
    Violent, or Hateful Content \nGV-1.3-003 \nEstablish a test plan and response\
    \ policy, before developing highly capable models, \nto periodically evaluate\
    \ whether the model may misuse CBRN information or \ncapabilities and/or oﬀensive\
    \ cyber capabilities. \nCBRN Information or Capabilities; \nInformation Security"
- source_sentence: What methods are suggested for recording and integrating structured
    feedback about content provenance from various stakeholders in the context of
    GAI systems?
  sentences:
  - "39 \nMS-3.3-004 \nProvide input for training materials about the capabilities\
    \ and limitations of GAI \nsystems related to digital content transparency for\
    \ AI Actors, other \nprofessionals, and the public about the societal impacts\
    \ of AI and the role of \ndiverse and inclusive content generation. \nHuman-AI\
    \ Conﬁguration; \nInformation Integrity; Harmful Bias \nand Homogenization \n\
    MS-3.3-005 \nRecord and integrate structured feedback about content provenance\
    \ from \noperators, users, and potentially impacted communities through the use\
    \ of \nmethods such as user research studies, focus groups, or community forums.\
    \ \nActively seek feedback on generated content quality and potential biases.\
    \ \nAssess the general awareness among end users and impacted communities \nabout\
    \ the availability of these feedback channels. \nHuman-AI Conﬁguration; \nInformation\
    \ Integrity; Harmful Bias \nand Homogenization \nAI Actor Tasks: AI Deployment,\
    \ Aﬀected Individuals and Communities, End-Users, Operation and Monitoring, TEVV\
    \ \n \nMEASURE 4.2: Measurement results regarding AI system trustworthiness in\
    \ deployment context(s) and across the AI lifecycle are \ninformed by input from\
    \ domain experts and relevant AI Actors to validate whether the system is performing\
    \ consistently as \nintended. Results are documented. \nAction ID \nSuggested\
    \ Action \nGAI Risks \nMS-4.2-001 \nConduct adversarial testing at a regular cadence\
    \ to map and measure GAI risks, \nincluding tests to address attempts to deceive\
    \ or manipulate the application of \nprovenance techniques or other misuses. Identify\
    \ vulnerabilities and \nunderstand potential misuse scenarios and unintended outputs.\
    \ \nInformation Integrity; Information \nSecurity \nMS-4.2-002 \nEvaluate GAI\
    \ system performance in real-world scenarios to observe its \nbehavior in practical\
    \ environments and reveal issues that might not surface in \ncontrolled and optimized\
    \ testing environments. \nHuman-AI Conﬁguration; \nConfabulation; Information\
    \ \nSecurity \nMS-4.2-003 \nImplement interpretability and explainability methods\
    \ to evaluate GAI system \ndecisions and verify alignment with intended purpose.\
    \ \nInformation Integrity; Harmful Bias \nand Homogenization \nMS-4.2-004 \nMonitor\
    \ and document instances where human operators or other systems \noverride the\
    \ GAI's decisions. Evaluate these cases to understand if the overrides \nare linked\
    \ to issues related to content provenance. \nInformation Integrity \nMS-4.2-005\
    \ \nVerify and document the incorporation of results of structured public feedback\
    \ \nexercises into design, implementation, deployment approval (“go”/“no-go” \n\
    decisions), monitoring, and decommission decisions. \nHuman-AI Conﬁguration; \n\
    Information Security \nAI Actor Tasks: AI Deployment, Domain Experts, End-Users,\
    \ Operation and Monitoring, TEVV"
  - "46 \nMG-4.3-003 \nReport GAI incidents in compliance with legal and regulatory\
    \ requirements (e.g., \nHIPAA breach reporting, e.g., OCR (2023) or NHTSA (2022)\
    \ autonomous vehicle \ncrash reporting requirements. \nInformation Security; Data\
    \ Privacy \nAI Actor Tasks: AI Deployment, Aﬀected Individuals and Communities,\
    \ Domain Experts, End-Users, Human Factors, Operation and \nMonitoring"
  - 'ENDNOTES

    107. Centers for Medicare & Medicaid Services. Biden-Harris Administration Quadruples
    the Number

    of Health Care Navigators Ahead of HealthCare.gov Open Enrollment Period. Aug.
    27, 2021.

    https://www.cms.gov/newsroom/press-releases/biden-harris-administration-quadruples-number­

    health-care-navigators-ahead-healthcaregov-open

    108. See, e.g., McKinsey & Company. The State of Customer Care in 2022. July 8,
    2022. https://

    www.mckinsey.com/business-functions/operations/our-insights/the-state-of-customer-care-in-2022;

    Sara Angeles. Customer Service Solutions for Small Businesses. Business News Daily.

    Jun. 29, 2022. https://www.businessnewsdaily.com/7575-customer-service-solutions.html

    109. Mike Hughes. Are We Getting The Best Out Of Our Bots? Co-Intelligence Between
    Robots &

    Humans. Forbes. Jul. 14, 2022.

    https://www.forbes.com/sites/mikehughes1/2022/07/14/are-we-getting-the-best-out-of-our-bots-co­

    intelligence-between-robots--humans/?sh=16a2bd207395

    110. Rachel Orey and Owen Bacskai. The Low Down on Ballot Curing. Nov. 04, 2020.
    https://

    bipartisanpolicy.org/blog/the-low-down-on-ballot-curing/; Zahavah Levine and Thea
    Raymond-

    Seidel. Mail Voting Litigation in 2020, Part IV: Verifying Mail Ballots. Oct.
    29, 2020.

    https://www.lawfareblog.com/mail-voting-litigation-2020-part-iv-verifying-mail-ballots

    111. National Conference of State Legislatures. Table 15: States With Signature
    Cure Processes. Jan. 18,

    2022.

    https://www.ncsl.org/research/elections-and-campaigns/vopp-table-15-states-that-permit-voters-to­

    correct-signature-discrepancies.aspx

    112. White House Office of Science and Technology Policy. Join the Effort to Create
    A Bill of Rights for

    an Automated Society. Nov. 10, 2021.

    https://www.whitehouse.gov/ostp/news-updates/2021/11/10/join-the-effort-to-create-a-bill-of­

    rights-for-an-automated-society/

    113. White House Office of Science and Technology Policy. Notice of Request for
    Information (RFI) on

    Public and Private Sector Uses of Biometric Technologies. Issued Oct. 8, 2021.

    https://www.federalregister.gov/documents/2021/10/08/2021-21975/notice-of-request-for­

    information-rfi-on-public-and-private-sector-uses-of-biometric-technologies

    114. National Artificial Intelligence Initiative Office. Public Input on Public
    and Private Sector Uses of

    Biometric Technologies. Accessed Apr. 19, 2022.

    https://www.ai.gov/86-fr-56300-responses/

    115. Thomas D. Olszewski, Lisa M. Van Pay, Javier F. Ortiz, Sarah E. Swiersz,
    and Laurie A. Dacus.

    Synopsis of Responses to OSTP’s Request for Information on the Use and Governance
    of Biometric

    Technologies in the Public and Private Sectors. Science and Technology Policy
    Institute. Mar. 2022.

    https://www.ida.org/-/media/feature/publications/s/sy/synopsis-of-responses-to-request-for­

    information-on-the-use-and-governance-of-biometric-technologies/ida-document-d-33070.ashx

    73'
---

# SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) <!-- at revision 8b3219a92973c328a8e22fadcfa821b5dc75636a -->
- **Maximum Sequence Length:** 256 tokens
- **Output Dimensionality:** 384 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("danicafisher/dfisher-sentence-transformer-fine-tuned")
# Run inference
sentences = [
    'What methods are suggested for recording and integrating structured feedback about content provenance from various stakeholders in the context of GAI systems?',
    "39 \nMS-3.3-004 \nProvide input for training materials about the capabilities and limitations of GAI \nsystems related to digital content transparency for AI Actors, other \nprofessionals, and the public about the societal impacts of AI and the role of \ndiverse and inclusive content generation. \nHuman-AI Conﬁguration; \nInformation Integrity; Harmful Bias \nand Homogenization \nMS-3.3-005 \nRecord and integrate structured feedback about content provenance from \noperators, users, and potentially impacted communities through the use of \nmethods such as user research studies, focus groups, or community forums. \nActively seek feedback on generated content quality and potential biases. \nAssess the general awareness among end users and impacted communities \nabout the availability of these feedback channels. \nHuman-AI Conﬁguration; \nInformation Integrity; Harmful Bias \nand Homogenization \nAI Actor Tasks: AI Deployment, Aﬀected Individuals and Communities, End-Users, Operation and Monitoring, TEVV \n \nMEASURE 4.2: Measurement results regarding AI system trustworthiness in deployment context(s) and across the AI lifecycle are \ninformed by input from domain experts and relevant AI Actors to validate whether the system is performing consistently as \nintended. Results are documented. \nAction ID \nSuggested Action \nGAI Risks \nMS-4.2-001 \nConduct adversarial testing at a regular cadence to map and measure GAI risks, \nincluding tests to address attempts to deceive or manipulate the application of \nprovenance techniques or other misuses. Identify vulnerabilities and \nunderstand potential misuse scenarios and unintended outputs. \nInformation Integrity; Information \nSecurity \nMS-4.2-002 \nEvaluate GAI system performance in real-world scenarios to observe its \nbehavior in practical environments and reveal issues that might not surface in \ncontrolled and optimized testing environments. \nHuman-AI Conﬁguration; \nConfabulation; Information \nSecurity \nMS-4.2-003 \nImplement interpretability and explainability methods to evaluate GAI system \ndecisions and verify alignment with intended purpose. \nInformation Integrity; Harmful Bias \nand Homogenization \nMS-4.2-004 \nMonitor and document instances where human operators or other systems \noverride the GAI's decisions. Evaluate these cases to understand if the overrides \nare linked to issues related to content provenance. \nInformation Integrity \nMS-4.2-005 \nVerify and document the incorporation of results of structured public feedback \nexercises into design, implementation, deployment approval (“go”/“no-go” \ndecisions), monitoring, and decommission decisions. \nHuman-AI Conﬁguration; \nInformation Security \nAI Actor Tasks: AI Deployment, Domain Experts, End-Users, Operation and Monitoring, TEVV",
    '46 \nMG-4.3-003 \nReport GAI incidents in compliance with legal and regulatory requirements (e.g., \nHIPAA breach reporting, e.g., OCR (2023) or NHTSA (2022) autonomous vehicle \ncrash reporting requirements. \nInformation Security; Data Privacy \nAI Actor Tasks: AI Deployment, Aﬀected Individuals and Communities, Domain Experts, End-Users, Human Factors, Operation and \nMonitoring',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset


* Size: 274 training samples
* Columns: <code>sentence_0</code> and <code>sentence_1</code>
* Approximate statistics based on the first 274 samples:
  |         | sentence_0                                                                         | sentence_1                                                                           |
  |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
  | type    | string                                                                             | string                                                                               |
  | details | <ul><li>min: 12 tokens</li><li>mean: 22.67 tokens</li><li>max: 38 tokens</li></ul> | <ul><li>min: 21 tokens</li><li>mean: 245.27 tokens</li><li>max: 256 tokens</li></ul> |
* Samples:
  | sentence_0                                                                                                                                                         | sentence_1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
  |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>How does the Executive Order on Advancing Racial Equity define 'equity' and 'underserved communities'?</code>                                                | <code>ENDNOTES<br>47. Darshali A. Vyas et al., Hidden in Plain Sight – Reconsidering the Use of Race Correction in Clinical<br>Algorithms, 383 N. Engl. J. Med.874, 876-78 (Aug. 27, 2020), https://www.nejm.org/doi/full/10.1056/<br>NEJMms2004740.<br>48. The definitions of 'equity' and 'underserved communities' can be found in the Definitions section of<br>this framework as well as in Section 2 of The Executive Order On Advancing Racial Equity and Support<br>for Underserved Communities Through the Federal Government. https://www.whitehouse.gov/<br>briefing-room/presidential-actions/2021/01/20/executive-order-advancing-racial-equity-and-support­<br>for-underserved-communities-through-the-federal-government/<br>49. Id.<br>50. Various organizations have offered proposals for how such assessments might be designed. See, e.g.,<br>Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, Madeleine Clare Elish, and Jacob Metcalf.<br>Assembling Accountability: Algorithmic Impact Assessment for the Public Interest. Data & Society<br>Research Institute Report. June 29, 2021. https://datasociety.net/library/assembling-accountability­<br>algorithmic-impact-assessment-for-the-public-interest/; Nicol Turner Lee, Paul Resnick, and Genie<br>Barton. Algorithmic bias detection and mitigation: Best practices and policies to reduce consumer harms.<br>Brookings Report. May 22, 2019.<br>https://www.brookings.edu/research/algorithmic-bias-detection-and-mitigation-best-practices-and­<br>policies-to-reduce-consumer-harms/; Andrew D. Selbst. An Institutional View Of Algorithmic Impact<br>Assessments. Harvard Journal of Law & Technology. June 15, 2021. https://ssrn.com/abstract=3867634;<br>Dillon Reisman, Jason Schultz, Kate Crawford, and Meredith Whittaker. Algorithmic Impact<br>Assessments: A Practical Framework for Public Agency Accountability. AI Now Institute Report. April<br>2018. https://ainowinstitute.org/aiareport2018.pdf<br>51. Department of Justice. Justice Department Announces New Initiative to Combat Redlining. Oct. 22,<br>2021. https://www.justice.gov/opa/pr/justice-department-announces-new-initiative-combat-redlining<br>52. PAVE Interagency Task Force on Property Appraisal and Valuation Equity. Action Plan to Advance<br>Property Appraisal and Valuation Equity: Closing the Racial Wealth Gap by Addressing Mis-valuations for<br>Families and Communities of Color. March 2022. https://pave.hud.gov/sites/pave.hud.gov/files/<br>documents/PAVEActionPlan.pdf<br>53. U.S. Equal Employment Opportunity Commission. The Americans with Disabilities Act and the Use of<br>Software, Algorithms, and Artificial Intelligence to Assess Job Applicants and Employees. EEOC­<br>NVTA-2022-2. May 12, 2022. https://www.eeoc.gov/laws/guidance/americans-disabilities-act-and-use­<br>software-algorithms-and-artificial-intelligence; U.S. Department of Justice. Algorithms, Artificial<br>Intelligence, and Disability Discrimination in Hiring. May 12, 2022. https://beta.ada.gov/resources/ai­<br>guidance/<br>54. Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in<br>an algorithm used to manage the health of populations. Science. Vol. 366, No. 6464. Oct. 25, 2019. https://<br>www.science.org/doi/10.1126/science.aax2342<br>55. Data & Trust Alliance. Algorithmic Bias Safeguards for Workforce: Overview. Jan. 2022. https://<br>dataandtrustalliance.org/Algorithmic_Bias_Safeguards_for_Workforce_Overview.pdf<br>56. Section 508.gov. IT Accessibility Laws and Policies. Access Board. https://www.section508.gov/<br>manage/laws-and-policies/<br>67</code> |
  | <code>What are the key expectations for automated systems as outlined in the context?</code>                                                                       | <code>HUMAN ALTERNATIVES, <br>CONSIDERATION, AND <br>FALLBACK <br>WHAT SHOULD BE EXPECTED OF AUTOMATED SYSTEMS<br>The expectations for automated systems are meant to serve as a blueprint for the development of additional <br>technical standards and practices that are tailored for particular sectors and contexts. <br>Equitable. Consideration should be given to ensuring outcomes of the fallback and escalation system are <br>equitable when compared to those of the automated system and such that the fallback and escalation <br>system provides equitable access to underserved communities.105 <br>Timely. Human consideration and fallback are only useful if they are conducted and concluded in a <br>timely manner. The determination of what is timely should be made relative to the specific automated <br>system, and the review system should be staffed and regularly assessed to ensure it is providing timely <br>consideration and fallback. In time-critical systems, this mechanism should be immediately available or, <br>where possible, available before the harm occurs. Time-critical systems include, but are not limited to, <br>voting-related systems, automated building access and other access systems, systems that form a critical <br>component of healthcare, and systems that have the ability to withhold wages or otherwise cause <br>immediate financial penalties. <br>Effective. The organizational structure surrounding processes for consideration and fallback should <br>be designed so that if the human decision-maker charged with reassessing a decision determines that it <br>should be overruled, the new decision will be effectively enacted. This includes ensuring that the new <br>decision is entered into the automated system throughout its components, any previous repercussions from <br>the old decision are also overturned, and safeguards are put in place to help ensure that future decisions do <br>not result in the same errors. <br>Maintained. The human consideration and fallback process and any associated automated processes <br>should be maintained and supported as long as the relevant automated system continues to be in use. <br>Institute training, assessment, and oversight to combat automation bias and ensure any <br>human-based components of a system are effective. <br>Training and assessment. Anyone administering, interacting with, or interpreting the outputs of an auto­<br>mated system should receive training in that system, including how to properly interpret outputs of a system <br>in light of its intended purpose and in how to mitigate the effects of automation bias. The training should reoc­<br>cur regularly to ensure it is up to date with the system and to ensure the system is used appropriately. Assess­<br>ment should be ongoing to ensure that the use of the system with human involvement provides for appropri­<br>ate results, i.e., that the involvement of people does not invalidate the system's assessment as safe and effective <br>or lead to algorithmic discrimination. <br>Oversight. Human-based systems have the potential for bias, including automation bias, as well as other <br>concerns that may limit their effectiveness. The results of assessments of the efficacy and potential bias of <br>such human-based systems should be overseen by governance structures that have the potential to update the <br>operation of the human-based system in order to mitigate these effects. <br>50</code>                                                                                                                                                       |
  | <code>What is the focus of the report titled "Assembling Accountability: Algorithmic Impact Assessment for the Public Interest" by Emanuel Moss and others?</code> | <code>ENDNOTES<br>47. Darshali A. Vyas et al., Hidden in Plain Sight – Reconsidering the Use of Race Correction in Clinical<br>Algorithms, 383 N. Engl. J. Med.874, 876-78 (Aug. 27, 2020), https://www.nejm.org/doi/full/10.1056/<br>NEJMms2004740.<br>48. The definitions of 'equity' and 'underserved communities' can be found in the Definitions section of<br>this framework as well as in Section 2 of The Executive Order On Advancing Racial Equity and Support<br>for Underserved Communities Through the Federal Government. https://www.whitehouse.gov/<br>briefing-room/presidential-actions/2021/01/20/executive-order-advancing-racial-equity-and-support­<br>for-underserved-communities-through-the-federal-government/<br>49. Id.<br>50. Various organizations have offered proposals for how such assessments might be designed. See, e.g.,<br>Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, Madeleine Clare Elish, and Jacob Metcalf.<br>Assembling Accountability: Algorithmic Impact Assessment for the Public Interest. Data & Society<br>Research Institute Report. June 29, 2021. https://datasociety.net/library/assembling-accountability­<br>algorithmic-impact-assessment-for-the-public-interest/; Nicol Turner Lee, Paul Resnick, and Genie<br>Barton. Algorithmic bias detection and mitigation: Best practices and policies to reduce consumer harms.<br>Brookings Report. May 22, 2019.<br>https://www.brookings.edu/research/algorithmic-bias-detection-and-mitigation-best-practices-and­<br>policies-to-reduce-consumer-harms/; Andrew D. Selbst. An Institutional View Of Algorithmic Impact<br>Assessments. Harvard Journal of Law & Technology. June 15, 2021. https://ssrn.com/abstract=3867634;<br>Dillon Reisman, Jason Schultz, Kate Crawford, and Meredith Whittaker. Algorithmic Impact<br>Assessments: A Practical Framework for Public Agency Accountability. AI Now Institute Report. April<br>2018. https://ainowinstitute.org/aiareport2018.pdf<br>51. Department of Justice. Justice Department Announces New Initiative to Combat Redlining. Oct. 22,<br>2021. https://www.justice.gov/opa/pr/justice-department-announces-new-initiative-combat-redlining<br>52. PAVE Interagency Task Force on Property Appraisal and Valuation Equity. Action Plan to Advance<br>Property Appraisal and Valuation Equity: Closing the Racial Wealth Gap by Addressing Mis-valuations for<br>Families and Communities of Color. March 2022. https://pave.hud.gov/sites/pave.hud.gov/files/<br>documents/PAVEActionPlan.pdf<br>53. U.S. Equal Employment Opportunity Commission. The Americans with Disabilities Act and the Use of<br>Software, Algorithms, and Artificial Intelligence to Assess Job Applicants and Employees. EEOC­<br>NVTA-2022-2. May 12, 2022. https://www.eeoc.gov/laws/guidance/americans-disabilities-act-and-use­<br>software-algorithms-and-artificial-intelligence; U.S. Department of Justice. Algorithms, Artificial<br>Intelligence, and Disability Discrimination in Hiring. May 12, 2022. https://beta.ada.gov/resources/ai­<br>guidance/<br>54. Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in<br>an algorithm used to manage the health of populations. Science. Vol. 366, No. 6464. Oct. 25, 2019. https://<br>www.science.org/doi/10.1126/science.aax2342<br>55. Data & Trust Alliance. Algorithmic Bias Safeguards for Workforce: Overview. Jan. 2022. https://<br>dataandtrustalliance.org/Algorithmic_Bias_Safeguards_for_Workforce_Overview.pdf<br>56. Section 508.gov. IT Accessibility Laws and Policies. Access Board. https://www.section508.gov/<br>manage/laws-and-policies/<br>67</code> |
* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `multi_dataset_batch_sampler`: round_robin

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: no
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 3
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `eval_use_gather_object`: False
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin

</details>

### Framework Versions
- Python: 3.11.9
- Sentence Transformers: 3.1.1
- Transformers: 4.44.2
- PyTorch: 2.4.1
- Accelerate: 0.34.2
- Datasets: 3.0.0
- Tokenizers: 0.19.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->