Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKGet startedIntroductionInstallationQuickstartLangChain Expression LanguageInterfaceHow toCookbookLangChain Expression Language (LCEL)Why use LCEL?ModulesModel I/‚ÄãORetrievalChainsMemoryAgentsCallbacksModulesSecurityGuidesAdaptersDebuggingDeploymentEvaluationString EvaluatorsComparison EvaluatorsTrajectory EvaluatorsCustom Trajectory EvaluatorAgent TrajectoryExamplesFallbacksLangSmithRun LLMs locallyModel comparisonPrivacyPydantic compatibilitySafetyMoreGuidesEvaluationTrajectory EvaluatorsAgent TrajectoryOn this pageAgent TrajectoryAgents can be difficult to holistically evaluate due to the breadth of actions and generation they can make. We recommend using multiple evaluation techniques appropriate to your use case. One way to evaluate an agent is to look at the whole trajectory of actions taken along with their responses.Evaluators that do this can implement the AgentTrajectoryEvaluator interface. This walkthrough will show how to use the trajectory evaluator to grade  an OpenAI functions agent.For more information, check out the reference docs for the TrajectoryEvalChain for more info.from langchain.evaluation import load_evaluatorevaluator = load_evaluator("trajectory")Methods‚ÄãThe Agent Trajectory Evaluators are used with the evaluate_agent_trajectory (and async aevaluate_agent_trajectory) methods, which accept:input (str) ‚Äì The input to the agent.prediction (str) ‚Äì The final predicted response.agent_trajectory (List[Tuple[AgentAction, str]]) ‚Äì The intermediate steps forming the agent trajectoryThey return a dictionary with the following values:score: Float from 0 to 1, where 1 would mean "most effective" and 0 would mean "least effective"reasoning: String "chain of thought reasoning" from the LLM generated prior to creating the scoreCapturing Trajectory‚ÄãThe easiest way to return an agent's trajectory (without using tracing
Open In Colab
Open In Colab ->: Skip to main contentü¶úÔ∏èüîó LangChainDocsUse casesIntegrationsAPICommunityChat our docsLangSmithJS/TS DocsSearchCTRLKGet startedIntroductionInstallationQuickstartLangChain Expression LanguageInterfaceHow toCookbookLangChain Expression Language (LCEL)Why use LCEL?ModulesModel I/‚ÄãORetrievalChainsMemoryAgentsCallbacksModulesSecurityGuidesAdaptersDebuggingDeploymentEvaluationString EvaluatorsComparison EvaluatorsTrajectory EvaluatorsCustom Trajectory EvaluatorAgent TrajectoryExamplesFallbacksLangSmithRun LLMs locallyModel comparisonPrivacyPydantic compatibilitySafetyMoreGuidesEvaluationTrajectory EvaluatorsAgent TrajectoryOn this pageAgent TrajectoryAgents can be difficult to holistically evaluate due to the breadth of actions and generation they can make. We recommend using multiple evaluation techniques appropriate to your use case. One way to evaluate an agent is to look at the whole trajectory of actions taken along with their responses.Evaluators that do this can implement the AgentTrajectoryEvaluator interface. This walkthrough will show how to use the trajectory evaluator to grade  an OpenAI functions agent.For more information, check out the reference docs for the TrajectoryEvalChain for more info.from langchain.evaluation import load_evaluatorevaluator = load_evaluator("trajectory")Methods‚ÄãThe Agent Trajectory Evaluators are used with the evaluate_agent_trajectory (and async aevaluate_agent_trajectory) methods, which accept:input (str) ‚Äì The input to the agent.prediction (str) ‚Äì The final predicted response.agent_trajectory (List[Tuple[AgentAction, str]]) ‚Äì The intermediate steps forming the agent trajectoryThey return a dictionary with the following values:score: Float from 0 to 1, where 1 would mean "most effective" and 0 would mean "least effective"reasoning: String "chain of thought reasoning" from the LLM generated prior to creating the scoreCapturing Trajectory‚ÄãThe easiest way to return an agent's trajectory (without using tracing