with out-of-the-box Model MonitoringFire Reports - Create and share rich MarkDown documents supporting embeddable online contentIn order to properly keep track of your langchain experiments and their results, you can enable the ClearML integration. We use the ClearML Experiment Manager that neatly tracks and organizes all your experiment runs.Installation and Setup‚Äãpip install clearmlpip install pandaspip install textstatpip install spacypython -m spacy download en_core_web_smGetting API Credentials‚ÄãWe'll be using quite some APIs in this notebook, here is a list and where to get them:ClearML: https://app.clear.ml/settings/workspace-configurationOpenAI: https://platform.openai.com/account/api-keysSerpAPI (google search): https://serpapi.com/dashboardimport osos.environ["CLEARML_API_ACCESS_KEY"] = ""os.environ["CLEARML_API_SECRET_KEY"] = ""os.environ["OPENAI_API_KEY"] = ""os.environ["SERPAPI_API_KEY"] = ""Callbacks‚Äãfrom langchain.callbacks import ClearMLCallbackHandlerfrom datetime import datetimefrom langchain.callbacks import StdOutCallbackHandlerfrom langchain.llms import OpenAI# Setup and use the ClearML Callbackclearml_callback = ClearMLCallbackHandler(    task_type="inference",    project_name="langchain_callback_demo",    task_name="llm",    tags=["test"],    # Change the following parameters based on the amount of detail you want tracked    visualize=True,    complexity_metrics=True,    stream_logs=True,)callbacks = [StdOutCallbackHandler(), clearml_callback]# Get the OpenAI model ready to gollm = OpenAI(temperature=0, callbacks=callbacks)    The clearml callback is currently in beta and is subject to change based on updates to `langchain`. Please report any issues to https://github.com/allegroai/clearml/issues with the tag `langchain`.Scenario 1: Just an LLM‚ÄãFirst, let's just run a single LLM a few times and capture the resulting prompt-answer conversation in ClearML# SCENARIO 1 - LLMllm_result = llm.generate(["Tell me a joke", "Tell me a poem"] *
ClearML is a ML/DL development and production suite, it contains 5 main modules:
ClearML is a ML/DL development and production suite, it contains 5 main modules: ->: with out-of-the-box Model MonitoringFire Reports - Create and share rich MarkDown documents supporting embeddable online contentIn order to properly keep track of your langchain experiments and their results, you can enable the ClearML integration. We use the ClearML Experiment Manager that neatly tracks and organizes all your experiment runs.Installation and Setup‚Äãpip install clearmlpip install pandaspip install textstatpip install spacypython -m spacy download en_core_web_smGetting API Credentials‚ÄãWe'll be using quite some APIs in this notebook, here is a list and where to get them:ClearML: https://app.clear.ml/settings/workspace-configurationOpenAI: https://platform.openai.com/account/api-keysSerpAPI (google search): https://serpapi.com/dashboardimport osos.environ["CLEARML_API_ACCESS_KEY"] = ""os.environ["CLEARML_API_SECRET_KEY"] = ""os.environ["OPENAI_API_KEY"] = ""os.environ["SERPAPI_API_KEY"] = ""Callbacks‚Äãfrom langchain.callbacks import ClearMLCallbackHandlerfrom datetime import datetimefrom langchain.callbacks import StdOutCallbackHandlerfrom langchain.llms import OpenAI# Setup and use the ClearML Callbackclearml_callback = ClearMLCallbackHandler(    task_type="inference",    project_name="langchain_callback_demo",    task_name="llm",    tags=["test"],    # Change the following parameters based on the amount of detail you want tracked    visualize=True,    complexity_metrics=True,    stream_logs=True,)callbacks = [StdOutCallbackHandler(), clearml_callback]# Get the OpenAI model ready to gollm = OpenAI(temperature=0, callbacks=callbacks)    The clearml callback is currently in beta and is subject to change based on updates to `langchain`. Please report any issues to https://github.com/allegroai/clearml/issues with the tag `langchain`.Scenario 1: Just an LLM‚ÄãFirst, let's just run a single LLM a few times and capture the resulting prompt-answer conversation in ClearML# SCENARIO 1 - LLMllm_result = llm.generate(["Tell me a joke", "Tell me a poem"] *