evaluation.requires_input: This property indicates whether this evaluator requires an input string.requires_reference: This property specifies whether this evaluator requires a reference label.LangSmith SupportThe run_on_dataset evaluation method is designed to evaluate only a single model at a time, and thus, doesn't support these evaluators.Detailed information about creating custom evaluators and the available built-in comparison evaluators is provided in the following sections.üìÑÔ∏è Pairwise string comparisonOpen In ColabüìÑÔ∏è Pairwise embedding distanceOpen In ColabüìÑÔ∏è Custom pairwise evaluatorOpen In ColabPreviousString DistanceNextPairwise string comparisonCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.
Comparison evaluators in LangChain help measure two different chains or LLM outputs. These evaluators are helpful for comparative analyses, such as A/B testing between two language models, or comparing different versions of the same model. They can also be useful for things like generating preference scores for ai-assisted reinforcement learning.
Comparison evaluators in LangChain help measure two different chains or LLM outputs. These evaluators are helpful for comparative analyses, such as A/B testing between two language models, or comparing different versions of the same model. They can also be useful for things like generating preference scores for ai-assisted reinforcement learning. ->: evaluation.requires_input: This property indicates whether this evaluator requires an input string.requires_reference: This property specifies whether this evaluator requires a reference label.LangSmith SupportThe run_on_dataset evaluation method is designed to evaluate only a single model at a time, and thus, doesn't support these evaluators.Detailed information about creating custom evaluators and the available built-in comparison evaluators is provided in the following sections.üìÑÔ∏è Pairwise string comparisonOpen In ColabüìÑÔ∏è Pairwise embedding distanceOpen In ColabüìÑÔ∏è Custom pairwise evaluatorOpen In ColabPreviousString DistanceNextPairwise string comparisonCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.