Exact Match | ü¶úÔ∏èüîó Langchain






Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKGet startedIntroductionInstallationQuickstartLangChain Expression LanguageInterfaceHow toCookbookLangChain Expression Language (LCEL)Why use LCEL?ModulesModel I/‚ÄãORetrievalChainsMemoryAgentsCallbacksModulesSecurityGuidesAdaptersDebuggingDeploymentEvaluationString EvaluatorsCriteria EvaluationCustom String EvaluatorEmbedding DistanceExact MatchRegex MatchScoring EvaluatorString DistanceComparison EvaluatorsTrajectory EvaluatorsExamplesFallbacksLangSmithRun LLMs locallyModel comparisonPrivacyPydantic compatibilitySafetyMoreGuidesEvaluationString EvaluatorsExact MatchOn this pageExact MatchProbably the simplest ways to evaluate an LLM or runnable's string output against a reference label is by a simple string equivalence.This can be accessed using the exact_match evaluator.from langchain.evaluation import ExactMatchStringEvaluatorevaluator = ExactMatchStringEvaluator()Alternatively via the loader:from langchain.evaluation import load_evaluatorevaluator = load_evaluator("exact_match")evaluator.evaluate_strings(    prediction="1 LLM.",    reference="2 llm",)    {'score': 0}evaluator.evaluate_strings(    prediction="LangChain",    reference="langchain",)    {'score': 0}Configure the ExactMatchStringEvaluator‚ÄãYou can relax the "exactness" when comparing strings.evaluator = ExactMatchStringEvaluator(    ignore_case=True,    ignore_numbers=True,    ignore_punctuation=True,)# Alternatively# evaluator = load_evaluator("exact_match", ignore_case=True, ignore_numbers=True, ignore_punctuation=True)evaluator.evaluate_strings(    prediction="1 LLM.",    reference="2 llm",)    {'score': 1}PreviousEmbedding DistanceNextRegex MatchConfigure the ExactMatchStringEvaluatorCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.
Open In Colab
Open In Colab ->: Exact Match | ü¶úÔ∏èüîó Langchain






Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKGet startedIntroductionInstallationQuickstartLangChain Expression LanguageInterfaceHow toCookbookLangChain Expression Language (LCEL)Why use LCEL?ModulesModel I/‚ÄãORetrievalChainsMemoryAgentsCallbacksModulesSecurityGuidesAdaptersDebuggingDeploymentEvaluationString EvaluatorsCriteria EvaluationCustom String EvaluatorEmbedding DistanceExact MatchRegex MatchScoring EvaluatorString DistanceComparison EvaluatorsTrajectory EvaluatorsExamplesFallbacksLangSmithRun LLMs locallyModel comparisonPrivacyPydantic compatibilitySafetyMoreGuidesEvaluationString EvaluatorsExact MatchOn this pageExact MatchProbably the simplest ways to evaluate an LLM or runnable's string output against a reference label is by a simple string equivalence.This can be accessed using the exact_match evaluator.from langchain.evaluation import ExactMatchStringEvaluatorevaluator = ExactMatchStringEvaluator()Alternatively via the loader:from langchain.evaluation import load_evaluatorevaluator = load_evaluator("exact_match")evaluator.evaluate_strings(    prediction="1 LLM.",    reference="2 llm",)    {'score': 0}evaluator.evaluate_strings(    prediction="LangChain",    reference="langchain",)    {'score': 0}Configure the ExactMatchStringEvaluator‚ÄãYou can relax the "exactness" when comparing strings.evaluator = ExactMatchStringEvaluator(    ignore_case=True,    ignore_numbers=True,    ignore_punctuation=True,)# Alternatively# evaluator = load_evaluator("exact_match", ignore_case=True, ignore_numbers=True, ignore_punctuation=True)evaluator.evaluate_strings(    prediction="1 LLM.",    reference="2 llm",)    {'score': 1}PreviousEmbedding DistanceNextRegex MatchConfigure the ExactMatchStringEvaluatorCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.