"Question: {question}\n\nAnswer: Let's think step by step."        prompt = PromptTemplate(template=template, input_variables=["question"])        self.chain = LLMChain(llm=llm, prompt=prompt)    def _run_chain(self, text: str):        return self.chain(text)    async def __call__(self, request: Request):        # 1. Parse the request        text = request.query_params["text"]        # 2. Run the chain        resp = self._run_chain(text)        # 3. Return the response        return resp["text"]Now we can bind the deployment.# Bind the model to deploymentdeployment = DeployLLM.bind()We can assign the port number and host when we want to run the deployment. # Example port numberPORT_NUMBER = 8282# Run the deploymentserve.api.run(deployment, port=PORT_NUMBER)Now that service is deployed on port localhost:8282 we can send a post request to get the results back.import requeststext = "What NFL team won the Super Bowl in the year Justin Beiber was born?"response = requests.post(f"http://localhost:{PORT_NUMBER}/?text={text}")print(response.content.decode())PreviousQdrantNextRebuffGoal of this notebookSetup Ray ServeGeneral SkeletonExample of deploying and OpenAI chain with custom promptsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.
Ray Serve is a scalable model serving library for building online inference APIs. Serve is particularly well suited for system composition, enabling you to build a complex inference service consisting of multiple chains and business logic all in Python code.
Ray Serve is a scalable model serving library for building online inference APIs. Serve is particularly well suited for system composition, enabling you to build a complex inference service consisting of multiple chains and business logic all in Python code. ->: "Question: {question}\n\nAnswer: Let's think step by step."        prompt = PromptTemplate(template=template, input_variables=["question"])        self.chain = LLMChain(llm=llm, prompt=prompt)    def _run_chain(self, text: str):        return self.chain(text)    async def __call__(self, request: Request):        # 1. Parse the request        text = request.query_params["text"]        # 2. Run the chain        resp = self._run_chain(text)        # 3. Return the response        return resp["text"]Now we can bind the deployment.# Bind the model to deploymentdeployment = DeployLLM.bind()We can assign the port number and host when we want to run the deployment. # Example port numberPORT_NUMBER = 8282# Run the deploymentserve.api.run(deployment, port=PORT_NUMBER)Now that service is deployed on port localhost:8282 we can send a post request to get the results back.import requeststext = "What NFL team won the Super Bowl in the year Justin Beiber was born?"response = requests.post(f"http://localhost:{PORT_NUMBER}/?text={text}")print(response.content.decode())PreviousQdrantNextRebuffGoal of this notebookSetup Ray ServeGeneral SkeletonExample of deploying and OpenAI chain with custom promptsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.