input_variables=["state"])model_lab_with_prompt = ModelLaboratory.from_llms(llms, prompt=prompt)model_lab_with_prompt.compare("New York")    Input:    New York        OpenAI    Params: {'model': 'text-davinci-002', 'temperature': 0.0, 'max_tokens': 256, 'top_p': 1, 'frequency_penalty': 0, 'presence_penalty': 0, 'n': 1, 'best_of': 1}            The capital of New York is Albany.        Cohere    Params: {'model': 'command-xlarge-20221108', 'max_tokens': 20, 'temperature': 0.0, 'k': 0, 'p': 1, 'frequency_penalty': 0, 'presence_penalty': 0}            The capital of New York is Albany.        HuggingFaceHub    Params: {'repo_id': 'google/flan-t5-xl', 'temperature': 1}    st john s    from langchain.chains import SelfAskWithSearchChainfrom langchain.utilities import SerpAPIWrapperopen_ai_llm = OpenAI(temperature=0)search = SerpAPIWrapper()self_ask_with_search_openai = SelfAskWithSearchChain(    llm=open_ai_llm, search_chain=search, verbose=True)cohere_llm = Cohere(temperature=0, model="command-xlarge-20221108")search = SerpAPIWrapper()self_ask_with_search_cohere = SelfAskWithSearchChain(    llm=cohere_llm, search_chain=search, verbose=True)chains = [self_ask_with_search_openai, self_ask_with_search_cohere]names = [str(open_ai_llm), str(cohere_llm)]model_lab = ModelLaboratory(chains, names=names)model_lab.compare("What is the hometown of the reigning men's U.S. Open champion?")    Input:    What is the hometown of the reigning men's U.S. Open champion?        OpenAI    Params: {'model': 'text-davinci-002', 'temperature': 0.0, 'max_tokens': 256, 'top_p': 1, 'frequency_penalty': 0, 'presence_penalty': 0, 'n': 1, 'best_of': 1}            > Entering new chain...    What is the hometown of the reigning men's U.S. Open champion?    Are follow up questions needed here: Yes.    Follow up: Who is the reigning men's U.S. Open champion?    Intermediate answer: Carlos Alcaraz.    Follow up: Where is Carlos Alcaraz from?    Intermediate answer: El Palmar, Spain.    So the final
Constructing your language model application will likely involved choosing between many different options of prompts, models, and even chains to use. When doing so, you will want to compare these different options on different inputs in an easy, flexible, and intuitive way.
Constructing your language model application will likely involved choosing between many different options of prompts, models, and even chains to use. When doing so, you will want to compare these different options on different inputs in an easy, flexible, and intuitive way. ->: input_variables=["state"])model_lab_with_prompt = ModelLaboratory.from_llms(llms, prompt=prompt)model_lab_with_prompt.compare("New York")    Input:    New York        OpenAI    Params: {'model': 'text-davinci-002', 'temperature': 0.0, 'max_tokens': 256, 'top_p': 1, 'frequency_penalty': 0, 'presence_penalty': 0, 'n': 1, 'best_of': 1}            The capital of New York is Albany.        Cohere    Params: {'model': 'command-xlarge-20221108', 'max_tokens': 20, 'temperature': 0.0, 'k': 0, 'p': 1, 'frequency_penalty': 0, 'presence_penalty': 0}            The capital of New York is Albany.        HuggingFaceHub    Params: {'repo_id': 'google/flan-t5-xl', 'temperature': 1}    st john s    from langchain.chains import SelfAskWithSearchChainfrom langchain.utilities import SerpAPIWrapperopen_ai_llm = OpenAI(temperature=0)search = SerpAPIWrapper()self_ask_with_search_openai = SelfAskWithSearchChain(    llm=open_ai_llm, search_chain=search, verbose=True)cohere_llm = Cohere(temperature=0, model="command-xlarge-20221108")search = SerpAPIWrapper()self_ask_with_search_cohere = SelfAskWithSearchChain(    llm=cohere_llm, search_chain=search, verbose=True)chains = [self_ask_with_search_openai, self_ask_with_search_cohere]names = [str(open_ai_llm), str(cohere_llm)]model_lab = ModelLaboratory(chains, names=names)model_lab.compare("What is the hometown of the reigning men's U.S. Open champion?")    Input:    What is the hometown of the reigning men's U.S. Open champion?        OpenAI    Params: {'model': 'text-davinci-002', 'temperature': 0.0, 'max_tokens': 256, 'top_p': 1, 'frequency_penalty': 0, 'presence_penalty': 0, 'n': 1, 'best_of': 1}            > Entering new chain...    What is the hometown of the reigning men's U.S. Open champion?    Are follow up questions needed here: Yes.    Follow up: Who is the reigning men's U.S. Open champion?    Intermediate answer: Carlos Alcaraz.    Follow up: Where is Carlos Alcaraz from?    Intermediate answer: El Palmar, Spain.    So the final