use_dummy_weights=False, download_dir=None, use_np_weights=False, tensor_parallel_size=1, seed=0)    INFO 08-06 11:37:41 llm_engine.py:196] # GPU blocks: 861, # CPU blocks: 512    Processed prompts: 100%|‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà| 1/1 [00:00<00:00,  2.00it/s]        What is the capital of France ? The capital of France is Paris.    Integrate the model in an LLMChain‚Äãfrom langchain.prompts import PromptTemplatefrom langchain.chains import LLMChaintemplate = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])llm_chain = LLMChain(prompt=prompt, llm=llm)question = "Who was the US president in the year the first Pokemon game was released?"print(llm_chain.run(question))    Processed prompts: 100%|‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà| 1/1 [00:01<00:00,  1.34s/it]            1. The first Pokemon game was released in 1996.    2. The president was Bill Clinton.    3. Clinton was president from 1993 to 2001.    4. The answer is Clinton.        Distributed Inference‚ÄãvLLM supports distributed tensor-parallel inference and serving. To run multi-GPU inference with the LLM class, set the tensor_parallel_size argument to the number of GPUs you want to use. For example, to run inference on 4 GPUsfrom langchain.llms import VLLMllm = VLLM(model="mosaicml/mpt-30b",           tensor_parallel_size=4,           trust_remote_code=True,  # mandatory for hf models)llm("What is the future of AI?")OpenAI-Compatible Server‚ÄãvLLM can be deployed as a server that mimics the OpenAI API protocol. This allows vLLM to be used as a drop-in replacement for applications using OpenAI API.This server can be queried in the same format as OpenAI API.OpenAI-Compatible Completion‚Äãfrom langchain.llms import VLLMOpenAIllm = VLLMOpenAI(    openai_api_key="EMPTY",    openai_api_base="http://localhost:8000/v1",    model_name="tiiuae/falcon-7b",    model_kwargs={"stop": ["."]})print(llm("Rome is"))     a city that is filled with
vLLM is a fast and easy-to-use library for LLM inference and serving, offering:
vLLM is a fast and easy-to-use library for LLM inference and serving, offering: ->: use_dummy_weights=False, download_dir=None, use_np_weights=False, tensor_parallel_size=1, seed=0)    INFO 08-06 11:37:41 llm_engine.py:196] # GPU blocks: 861, # CPU blocks: 512    Processed prompts: 100%|‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà| 1/1 [00:00<00:00,  2.00it/s]        What is the capital of France ? The capital of France is Paris.    Integrate the model in an LLMChain‚Äãfrom langchain.prompts import PromptTemplatefrom langchain.chains import LLMChaintemplate = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate(template=template, input_variables=["question"])llm_chain = LLMChain(prompt=prompt, llm=llm)question = "Who was the US president in the year the first Pokemon game was released?"print(llm_chain.run(question))    Processed prompts: 100%|‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà‚ñà| 1/1 [00:01<00:00,  1.34s/it]            1. The first Pokemon game was released in 1996.    2. The president was Bill Clinton.    3. Clinton was president from 1993 to 2001.    4. The answer is Clinton.        Distributed Inference‚ÄãvLLM supports distributed tensor-parallel inference and serving. To run multi-GPU inference with the LLM class, set the tensor_parallel_size argument to the number of GPUs you want to use. For example, to run inference on 4 GPUsfrom langchain.llms import VLLMllm = VLLM(model="mosaicml/mpt-30b",           tensor_parallel_size=4,           trust_remote_code=True,  # mandatory for hf models)llm("What is the future of AI?")OpenAI-Compatible Server‚ÄãvLLM can be deployed as a server that mimics the OpenAI API protocol. This allows vLLM to be used as a drop-in replacement for applications using OpenAI API.This server can be queried in the same format as OpenAI API.OpenAI-Compatible Completion‚Äãfrom langchain.llms import VLLMOpenAIllm = VLLMOpenAI(    openai_api_key="EMPTY",    openai_api_base="http://localhost:8000/v1",    model_name="tiiuae/falcon-7b",    model_kwargs={"stop": ["."]})print(llm("Rome is"))     a city that is filled with