Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKGet startedIntroductionInstallationQuickstartLangChain Expression LanguageInterfaceHow toCookbookLangChain Expression Language (LCEL)Why use LCEL?ModulesModel I/‚ÄãORetrievalChainsMemoryAgentsCallbacksModulesSecurityGuidesAdaptersDebuggingDeploymentEvaluationString EvaluatorsCriteria EvaluationCustom String EvaluatorEmbedding DistanceExact MatchRegex MatchScoring EvaluatorString DistanceComparison EvaluatorsTrajectory EvaluatorsExamplesFallbacksLangSmithRun LLMs locallyModel comparisonPrivacyPydantic compatibilitySafetyMoreGuidesEvaluationString EvaluatorsCriteria EvaluationOn this pageCriteria EvaluationIn scenarios where you wish to assess a model's output using a specific rubric or criteria set, the criteria evaluator proves to be a handy tool. It allows you to verify if an LLM or Chain's output complies with a defined set of criteria.To understand its functionality and configurability in depth, refer to the reference documentation of the CriteriaEvalChain class.Usage without references‚ÄãIn this example, you will use the CriteriaEvalChain to check whether an output is concise. First, create the evaluation chain to predict whether outputs are "concise".from langchain.evaluation import load_evaluatorevaluator = load_evaluator("criteria", criteria="conciseness")# This is equivalent to loading using the enumfrom langchain.evaluation import EvaluatorTypeevaluator = load_evaluator(EvaluatorType.CRITERIA, criteria="conciseness")eval_result = evaluator.evaluate_strings(    prediction="What's 2+2? That's an elementary question. The answer you're looking for is that two and two is four.",    input="What's 2+2?",)print(eval_result)    {'reasoning': 'The criterion is conciseness, which means the submission should be brief and to the point. \n\nLooking at the submission, the answer to the question "What\'s 2+2?" is indeed "four". However, the respondent
Open In Colab
Open In Colab ->: Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKGet startedIntroductionInstallationQuickstartLangChain Expression LanguageInterfaceHow toCookbookLangChain Expression Language (LCEL)Why use LCEL?ModulesModel I/‚ÄãORetrievalChainsMemoryAgentsCallbacksModulesSecurityGuidesAdaptersDebuggingDeploymentEvaluationString EvaluatorsCriteria EvaluationCustom String EvaluatorEmbedding DistanceExact MatchRegex MatchScoring EvaluatorString DistanceComparison EvaluatorsTrajectory EvaluatorsExamplesFallbacksLangSmithRun LLMs locallyModel comparisonPrivacyPydantic compatibilitySafetyMoreGuidesEvaluationString EvaluatorsCriteria EvaluationOn this pageCriteria EvaluationIn scenarios where you wish to assess a model's output using a specific rubric or criteria set, the criteria evaluator proves to be a handy tool. It allows you to verify if an LLM or Chain's output complies with a defined set of criteria.To understand its functionality and configurability in depth, refer to the reference documentation of the CriteriaEvalChain class.Usage without references‚ÄãIn this example, you will use the CriteriaEvalChain to check whether an output is concise. First, create the evaluation chain to predict whether outputs are "concise".from langchain.evaluation import load_evaluatorevaluator = load_evaluator("criteria", criteria="conciseness")# This is equivalent to loading using the enumfrom langchain.evaluation import EvaluatorTypeevaluator = load_evaluator(EvaluatorType.CRITERIA, criteria="conciseness")eval_result = evaluator.evaluate_strings(    prediction="What's 2+2? That's an elementary question. The answer you're looking for is that two and two is four.",    input="What's 2+2?",)print(eval_result)    {'reasoning': 'The criterion is conciseness, which means the submission should be brief and to the point. \n\nLooking at the submission, the answer to the question "What\'s 2+2?" is indeed "four". However, the respondent