from here:pip install -r requirements.txtSet GOOGLE_CSE_ID and GOOGLE_API_KEY.from langchain.vectorstores import Chromafrom langchain.embeddings import OpenAIEmbeddingsfrom langchain.chat_models.openai import ChatOpenAIfrom langchain.utilities import GoogleSearchAPIWrapperfrom langchain.retrievers.web_research import WebResearchRetriever# Vectorstorevectorstore = Chroma(embedding_function=OpenAIEmbeddings(),persist_directory="./chroma_db_oai")# LLMllm = ChatOpenAI(temperature=0)# Search search = GoogleSearchAPIWrapper()Initialize retriever with the above tools to:Use an LLM to generate multiple relevant search queries (one LLM call)Execute a search for each queryChoose the top K links per query  (multiple search calls in parallel)Load the information from all chosen links (scrape pages in parallel)Index those documents into a vectorstoreFind the most relevant documents for each original generated search query# Initializeweb_research_retriever = WebResearchRetriever.from_llm(    vectorstore=vectorstore,    llm=llm,     search=search)# Runimport logginglogging.basicConfig()logging.getLogger("langchain.retrievers.web_research").setLevel(logging.INFO)from langchain.chains import RetrievalQAWithSourcesChainuser_input = "How do LLM Powered Autonomous Agents work?"qa_chain = RetrievalQAWithSourcesChain.from_chain_type(llm,retriever=web_research_retriever)result = qa_chain({"question": user_input})result    INFO:langchain.retrievers.web_research:Generating questions for Google Search ...    INFO:langchain.retrievers.web_research:Questions for Google Search (raw): {'question': 'How do LLM Powered Autonomous Agents work?', 'text': LineList(lines=['1. What is the functioning principle of LLM Powered Autonomous Agents?\n', '2. How do LLM Powered Autonomous Agents operate?\n'])}    INFO:langchain.retrievers.web_research:Questions for Google Search: ['1. What is the functioning principle of LLM Powered Autonomous Agents?\n', '2. How do LLM Powered Autonomous Agents operate?\n']
Open In Collab
Open In Collab ->: from here:pip install -r requirements.txtSet GOOGLE_CSE_ID and GOOGLE_API_KEY.from langchain.vectorstores import Chromafrom langchain.embeddings import OpenAIEmbeddingsfrom langchain.chat_models.openai import ChatOpenAIfrom langchain.utilities import GoogleSearchAPIWrapperfrom langchain.retrievers.web_research import WebResearchRetriever# Vectorstorevectorstore = Chroma(embedding_function=OpenAIEmbeddings(),persist_directory="./chroma_db_oai")# LLMllm = ChatOpenAI(temperature=0)# Search search = GoogleSearchAPIWrapper()Initialize retriever with the above tools to:Use an LLM to generate multiple relevant search queries (one LLM call)Execute a search for each queryChoose the top K links per query  (multiple search calls in parallel)Load the information from all chosen links (scrape pages in parallel)Index those documents into a vectorstoreFind the most relevant documents for each original generated search query# Initializeweb_research_retriever = WebResearchRetriever.from_llm(    vectorstore=vectorstore,    llm=llm,     search=search)# Runimport logginglogging.basicConfig()logging.getLogger("langchain.retrievers.web_research").setLevel(logging.INFO)from langchain.chains import RetrievalQAWithSourcesChainuser_input = "How do LLM Powered Autonomous Agents work?"qa_chain = RetrievalQAWithSourcesChain.from_chain_type(llm,retriever=web_research_retriever)result = qa_chain({"question": user_input})result    INFO:langchain.retrievers.web_research:Generating questions for Google Search ...    INFO:langchain.retrievers.web_research:Questions for Google Search (raw): {'question': 'How do LLM Powered Autonomous Agents work?', 'text': LineList(lines=['1. What is the functioning principle of LLM Powered Autonomous Agents?\n', '2. How do LLM Powered Autonomous Agents operate?\n'])}    INFO:langchain.retrievers.web_research:Questions for Google Search: ['1. What is the functioning principle of LLM Powered Autonomous Agents?\n', '2. How do LLM Powered Autonomous Agents operate?\n']