import LangSmithRunChatLoaderloader = LangSmithRunChatLoader(runs=llm_runs)chat_sessions = loader.lazy_load()With the chat sessions loaded, convert them into a format suitable for fine-tuning.‚Äãfrom langchain.adapters.openai import convert_messages_for_finetuningtraining_data = convert_messages_for_finetuning(chat_sessions)3. Fine-tune the model‚ÄãNow, initiate the fine-tuning process using the OpenAI library.import openaiimport timeimport jsonfrom io import BytesIOmy_file = BytesIO()for dialog in training_data:    my_file.write((json.dumps({"messages": dialog}) + "\n").encode('utf-8'))my_file.seek(0)training_file = openai.File.create(  file=my_file,  purpose='fine-tune')job = openai.FineTuningJob.create(    training_file=training_file.id,    model="gpt-3.5-turbo",)# Wait for the fine-tuning to complete (this may take some time)status = openai.FineTuningJob.retrieve(job.id).statusstart_time = time.time()while status != "succeeded":    print(f"Status=[{status}]... {time.time() - start_time:.2f}s", end="\r", flush=True)    time.sleep(5)    status = openai.FineTuningJob.retrieve(job.id).status# Now your model is fine-tuned!    Status=[running]... 346.26s. 31.70s4. Use in LangChain‚ÄãAfter fine-tuning, use the resulting model ID with the ChatOpenAI model class in your LangChain app.# Get the fine-tuned model IDjob = openai.FineTuningJob.retrieve(job.id)model_id = job.fine_tuned_model# Use the fine-tuned model in LangChainmodel = ChatOpenAI(    model=model_id,    temperature=1,)(prompt | model).invoke({"input": "What's 56/7?"})    AIMessage(content='{\n  "num1": 56,\n  "num2": 7,\n  "operation": "/"\n}')Now you have successfully fine-tuned a model using data from LangSmith LLM runs!PreviousFine-Tuning on LangSmith Chat DatasetsNextSlackPrerequisites1. Select Runs2. Prepare data3. Fine-tune the model4. Use in LangChainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.
This notebook demonstrates how to directly load data from LangSmith's LLM runs and fine-tune a model on that data.
This notebook demonstrates how to directly load data from LangSmith's LLM runs and fine-tune a model on that data. ->: import LangSmithRunChatLoaderloader = LangSmithRunChatLoader(runs=llm_runs)chat_sessions = loader.lazy_load()With the chat sessions loaded, convert them into a format suitable for fine-tuning.‚Äãfrom langchain.adapters.openai import convert_messages_for_finetuningtraining_data = convert_messages_for_finetuning(chat_sessions)3. Fine-tune the model‚ÄãNow, initiate the fine-tuning process using the OpenAI library.import openaiimport timeimport jsonfrom io import BytesIOmy_file = BytesIO()for dialog in training_data:    my_file.write((json.dumps({"messages": dialog}) + "\n").encode('utf-8'))my_file.seek(0)training_file = openai.File.create(  file=my_file,  purpose='fine-tune')job = openai.FineTuningJob.create(    training_file=training_file.id,    model="gpt-3.5-turbo",)# Wait for the fine-tuning to complete (this may take some time)status = openai.FineTuningJob.retrieve(job.id).statusstart_time = time.time()while status != "succeeded":    print(f"Status=[{status}]... {time.time() - start_time:.2f}s", end="\r", flush=True)    time.sleep(5)    status = openai.FineTuningJob.retrieve(job.id).status# Now your model is fine-tuned!    Status=[running]... 346.26s. 31.70s4. Use in LangChain‚ÄãAfter fine-tuning, use the resulting model ID with the ChatOpenAI model class in your LangChain app.# Get the fine-tuned model IDjob = openai.FineTuningJob.retrieve(job.id)model_id = job.fine_tuned_model# Use the fine-tuned model in LangChainmodel = ChatOpenAI(    model=model_id,    temperature=1,)(prompt | model).invoke({"input": "What's 56/7?"})    AIMessage(content='{\n  "num1": 56,\n  "num2": 7,\n  "operation": "/"\n}')Now you have successfully fine-tuned a model using data from LangSmith LLM runs!PreviousFine-Tuning on LangSmith Chat DatasetsNextSlackPrerequisites1. Select Runs2. Prepare data3. Fine-tune the model4. Use in LangChainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.