Here's a list of pre-implemented criteria. Note that in the absence of labels, the LLM merely predicts what it thinks the best answer is and is not grounded in actual law or context.from langchain.evaluation import Criteria# For a list of other default supported criteria, try calling `supported_default_criteria`list(Criteria)    [<Criteria.CONCISENESS: 'conciseness'>,     <Criteria.RELEVANCE: 'relevance'>,     <Criteria.CORRECTNESS: 'correctness'>,     <Criteria.COHERENCE: 'coherence'>,     <Criteria.HARMFULNESS: 'harmfulness'>,     <Criteria.MALICIOUSNESS: 'maliciousness'>,     <Criteria.HELPFULNESS: 'helpfulness'>,     <Criteria.CONTROVERSIALITY: 'controversiality'>,     <Criteria.MISOGYNY: 'misogyny'>,     <Criteria.CRIMINALITY: 'criminality'>,     <Criteria.INSENSITIVITY: 'insensitivity'>]Custom Criteria‚ÄãTo evaluate outputs against your own custom criteria, or to be more explicit the definition of any of the default criteria, pass in a dictionary of "criterion_name": "criterion_description"Note: it's recommended that you create a single evaluator per criterion. This way, separate feedback can be provided for each aspect. Additionally, if you provide antagonistic criteria, the evaluator won't be very useful, as it will be configured to predict compliance for ALL of the criteria provided.custom_criterion = {"numeric": "Does the output contain numeric or mathematical information?"}eval_chain = load_evaluator(    EvaluatorType.CRITERIA,    criteria=custom_criterion,)query = "Tell me a joke"prediction = "I ate some square pie but I don't know the square of pi."eval_result = eval_chain.evaluate_strings(prediction=prediction, input=query)print(eval_result)# If you wanted to specify multiple criteria. Generally not recommendedcustom_criteria = {    "numeric": "Does the output contain numeric information?",    "mathematical": "Does the output contain mathematical information?",    "grammatical": "Is the output grammatically correct?",    "logical": "Is the output
Open In Colab
Open In Colab ->: Here's a list of pre-implemented criteria. Note that in the absence of labels, the LLM merely predicts what it thinks the best answer is and is not grounded in actual law or context.from langchain.evaluation import Criteria# For a list of other default supported criteria, try calling `supported_default_criteria`list(Criteria)    [<Criteria.CONCISENESS: 'conciseness'>,     <Criteria.RELEVANCE: 'relevance'>,     <Criteria.CORRECTNESS: 'correctness'>,     <Criteria.COHERENCE: 'coherence'>,     <Criteria.HARMFULNESS: 'harmfulness'>,     <Criteria.MALICIOUSNESS: 'maliciousness'>,     <Criteria.HELPFULNESS: 'helpfulness'>,     <Criteria.CONTROVERSIALITY: 'controversiality'>,     <Criteria.MISOGYNY: 'misogyny'>,     <Criteria.CRIMINALITY: 'criminality'>,     <Criteria.INSENSITIVITY: 'insensitivity'>]Custom Criteria‚ÄãTo evaluate outputs against your own custom criteria, or to be more explicit the definition of any of the default criteria, pass in a dictionary of "criterion_name": "criterion_description"Note: it's recommended that you create a single evaluator per criterion. This way, separate feedback can be provided for each aspect. Additionally, if you provide antagonistic criteria, the evaluator won't be very useful, as it will be configured to predict compliance for ALL of the criteria provided.custom_criterion = {"numeric": "Does the output contain numeric or mathematical information?"}eval_chain = load_evaluator(    EvaluatorType.CRITERIA,    criteria=custom_criterion,)query = "Tell me a joke"prediction = "I ate some square pie but I don't know the square of pi."eval_result = eval_chain.evaluate_strings(prediction=prediction, input=query)print(eval_result)# If you wanted to specify multiple criteria. Generally not recommendedcustom_criteria = {    "numeric": "Does the output contain numeric information?",    "mathematical": "Does the output contain mathematical information?",    "grammatical": "Is the output grammatically correct?",    "logical": "Is the output