Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKGet startedIntroductionInstallationQuickstartLangChain Expression LanguageInterfaceHow toCookbookLangChain Expression Language (LCEL)Why use LCEL?ModulesModel I/‚ÄãORetrievalChainsMemoryAgentsCallbacksModulesSecurityGuidesAdaptersDebuggingDeploymentEvaluationString EvaluatorsCriteria EvaluationCustom String EvaluatorEmbedding DistanceExact MatchRegex MatchScoring EvaluatorString DistanceComparison EvaluatorsTrajectory EvaluatorsExamplesFallbacksLangSmithRun LLMs locallyModel comparisonPrivacyPydantic compatibilitySafetyMoreGuidesEvaluationString EvaluatorsString DistanceOn this pageString DistanceOne of the simplest ways to compare an LLM or chain's string output against a reference label is by using string distance measurements such as Levenshtein or postfix distance.  This can be used alongside approximate/fuzzy matching criteria for very basic unit testing.This can be accessed using the string_distance evaluator, which uses distance metric's from the rapidfuzz library.Note: The returned scores are distances, meaning lower is typically "better".For more information, check out the reference docs for the StringDistanceEvalChain for more info.# %pip install rapidfuzzfrom langchain.evaluation import load_evaluatorevaluator = load_evaluator("string_distance")evaluator.evaluate_strings(    prediction="The job is completely done.",    reference="The job is done",)    {'score': 0.11555555555555552}# The results purely character-based, so it's less useful when negation is concernedevaluator.evaluate_strings(    prediction="The job is done.",    reference="The job isn't done",)    {'score': 0.0724999999999999}Configure the String Distance Metric‚ÄãBy default, the StringDistanceEvalChain uses  levenshtein distance, but it also supports other string distance algorithms. Configure using the distance argument.from langchain.evaluation import
Open In Colab
Open In Colab ->: Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKGet startedIntroductionInstallationQuickstartLangChain Expression LanguageInterfaceHow toCookbookLangChain Expression Language (LCEL)Why use LCEL?ModulesModel I/‚ÄãORetrievalChainsMemoryAgentsCallbacksModulesSecurityGuidesAdaptersDebuggingDeploymentEvaluationString EvaluatorsCriteria EvaluationCustom String EvaluatorEmbedding DistanceExact MatchRegex MatchScoring EvaluatorString DistanceComparison EvaluatorsTrajectory EvaluatorsExamplesFallbacksLangSmithRun LLMs locallyModel comparisonPrivacyPydantic compatibilitySafetyMoreGuidesEvaluationString EvaluatorsString DistanceOn this pageString DistanceOne of the simplest ways to compare an LLM or chain's string output against a reference label is by using string distance measurements such as Levenshtein or postfix distance.  This can be used alongside approximate/fuzzy matching criteria for very basic unit testing.This can be accessed using the string_distance evaluator, which uses distance metric's from the rapidfuzz library.Note: The returned scores are distances, meaning lower is typically "better".For more information, check out the reference docs for the StringDistanceEvalChain for more info.# %pip install rapidfuzzfrom langchain.evaluation import load_evaluatorevaluator = load_evaluator("string_distance")evaluator.evaluate_strings(    prediction="The job is completely done.",    reference="The job is done",)    {'score': 0.11555555555555552}# The results purely character-based, so it's less useful when negation is concernedevaluator.evaluate_strings(    prediction="The job is done.",    reference="The job isn't done",)    {'score': 0.0724999999999999}Configure the String Distance Metric‚ÄãBy default, the StringDistanceEvalChain uses  levenshtein distance, but it also supports other string distance algorithms. Configure using the distance argument.from langchain.evaluation import