an input is provided, indicating that it will not be considered in the evaluation.requires_reference: Boolean attribute specifying whether the evaluator requires a reference label. If True, the evaluator will raise an error when the reference isn't provided. If False, a warning will be logged if a reference is provided, indicating that it will not be considered in the evaluation.String evaluators also implement the following methods:aevaluate_strings: Asynchronously evaluates the output of the Chain or Language Model, with support for optional input and label.evaluate_strings: Synchronously evaluates the output of the Chain or Language Model, with support for optional input and label.The following sections provide detailed information on available string evaluator implementations as well as how to create a custom string evaluator.üìÑÔ∏è Criteria EvaluationOpen In ColabüìÑÔ∏è Custom String EvaluatorOpen In ColabüìÑÔ∏è Embedding DistanceOpen In ColabüìÑÔ∏è Exact MatchOpen In ColabüìÑÔ∏è Regex MatchOpen In ColabüìÑÔ∏è Scoring EvaluatorThe Scoring Evaluator instructs a language model to assess your model's predictions on a specified scale (default is 1-10) based on your custom criteria or rubric. This feature provides a nuanced evaluation instead of a simplistic binary score, aiding in evaluating models against tailored rubrics and comparing model performance on specific tasks.üìÑÔ∏è String DistanceOpen In ColabPreviousEvaluationNextCriteria EvaluationCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.
A string evaluator is a component within LangChain designed to assess the performance of a language model by comparing its generated outputs (predictions) to a reference string or an input. This comparison is a crucial step in the evaluation of language models, providing a measure of the accuracy or quality of the generated text.
A string evaluator is a component within LangChain designed to assess the performance of a language model by comparing its generated outputs (predictions) to a reference string or an input. This comparison is a crucial step in the evaluation of language models, providing a measure of the accuracy or quality of the generated text. ->: an input is provided, indicating that it will not be considered in the evaluation.requires_reference: Boolean attribute specifying whether the evaluator requires a reference label. If True, the evaluator will raise an error when the reference isn't provided. If False, a warning will be logged if a reference is provided, indicating that it will not be considered in the evaluation.String evaluators also implement the following methods:aevaluate_strings: Asynchronously evaluates the output of the Chain or Language Model, with support for optional input and label.evaluate_strings: Synchronously evaluates the output of the Chain or Language Model, with support for optional input and label.The following sections provide detailed information on available string evaluator implementations as well as how to create a custom string evaluator.üìÑÔ∏è Criteria EvaluationOpen In ColabüìÑÔ∏è Custom String EvaluatorOpen In ColabüìÑÔ∏è Embedding DistanceOpen In ColabüìÑÔ∏è Exact MatchOpen In ColabüìÑÔ∏è Regex MatchOpen In ColabüìÑÔ∏è Scoring EvaluatorThe Scoring Evaluator instructs a language model to assess your model's predictions on a specified scale (default is 1-10) based on your custom criteria or rubric. This feature provides a nuanced evaluation instead of a simplistic binary score, aiding in evaluating models against tailored rubrics and comparing model performance on specific tasks.üìÑÔ∏è String DistanceOpen In ColabPreviousEvaluationNextCriteria EvaluationCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.