from synthetic data creation to testing.In this guide we will demonstrate how to test and measure LLMs in performance. We show how you can use our callback to measure performance and how you can define your own metric and log them into our dashboard.DeepEval also offers:How to generate synthetic dataHow to measure performanceA dashboard to monitor and review results over timeInstallation and Setup‚Äãpip install deepeval --upgradeGetting API Credentials‚ÄãTo get the DeepEval API credentials, follow the next steps:Go to https://app.confident-ai.comClick on "Organization"Copy the API Key.When you log in, you will also be asked to set the implementation name. The implementation name is required to describe the type of implementation. (Think of what you want to call your project. We recommend making it descriptive.)deepeval loginSetup DeepEval‚ÄãYou can, by default, use the DeepEvalCallbackHandler to set up the metrics you want to track. However, this has limited support for metrics at the moment (more to be added soon). It currently supports:Answer RelevancyBiasToxicnessfrom deepeval.metrics.answer_relevancy import AnswerRelevancy# Here we want to make sure the answer is minimally relevantanswer_relevancy_metric = AnswerRelevancy(minimum_score=0.5)Get Started‚ÄãTo use the DeepEvalCallbackHandler, we need the implementation_name. import osfrom langchain.callbacks.confident_callback import DeepEvalCallbackHandlerdeepeval_callback = DeepEvalCallbackHandler(    implementation_name="langchainQuickstart",    metrics=[answer_relevancy_metric])Scenario 1: Feeding into LLM‚ÄãYou can then feed it into your LLM with OpenAI.from langchain.llms import OpenAIllm = OpenAI(    temperature=0,    callbacks=[deepeval_callback],    verbose=True,    openai_api_key="<YOUR_API_KEY>",)output = llm.generate(    [        "What is the best evaluation tool out there? (no bias at all)",    ])    LLMResult(generations=[[Generation(text='\n\nQ: What did the fish say when he hit the wall? \nA: Dam.',
DeepEval package for unit testing LLMs.
DeepEval package for unit testing LLMs. ->: from synthetic data creation to testing.In this guide we will demonstrate how to test and measure LLMs in performance. We show how you can use our callback to measure performance and how you can define your own metric and log them into our dashboard.DeepEval also offers:How to generate synthetic dataHow to measure performanceA dashboard to monitor and review results over timeInstallation and Setup‚Äãpip install deepeval --upgradeGetting API Credentials‚ÄãTo get the DeepEval API credentials, follow the next steps:Go to https://app.confident-ai.comClick on "Organization"Copy the API Key.When you log in, you will also be asked to set the implementation name. The implementation name is required to describe the type of implementation. (Think of what you want to call your project. We recommend making it descriptive.)deepeval loginSetup DeepEval‚ÄãYou can, by default, use the DeepEvalCallbackHandler to set up the metrics you want to track. However, this has limited support for metrics at the moment (more to be added soon). It currently supports:Answer RelevancyBiasToxicnessfrom deepeval.metrics.answer_relevancy import AnswerRelevancy# Here we want to make sure the answer is minimally relevantanswer_relevancy_metric = AnswerRelevancy(minimum_score=0.5)Get Started‚ÄãTo use the DeepEvalCallbackHandler, we need the implementation_name. import osfrom langchain.callbacks.confident_callback import DeepEvalCallbackHandlerdeepeval_callback = DeepEvalCallbackHandler(    implementation_name="langchainQuickstart",    metrics=[answer_relevancy_metric])Scenario 1: Feeding into LLM‚ÄãYou can then feed it into your LLM with OpenAI.from langchain.llms import OpenAIllm = OpenAI(    temperature=0,    callbacks=[deepeval_callback],    verbose=True,    openai_api_key="<YOUR_API_KEY>",)output = llm.generate(    [        "What is the best evaluation tool out there? (no bias at all)",    ])    LLMResult(generations=[[Generation(text='\n\nQ: What did the fish say when he hit the wall? \nA: Dam.',