"action_input": "2 + 5 = 7."}Human: 'What's the capital of Maryland?'AI Assistant:"""from transformers import pipelinefrom langchain.llms import HuggingFacePipelinehf_model = pipeline(    "text-generation", model="cerebras/Cerebras-GPT-590M", max_new_tokens=200)original_model = HuggingFacePipeline(pipeline=hf_model)generated = original_model.generate([prompt], stop=["Human:"])print(generated)    Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.    generations=[[Generation(text=' "What\'s the capital of Maryland?"\n', generation_info=None)]] llm_output=NoneThat's not so impressive, is it? It didn't answer the question and it didn't follow the JSON format at all! Let's try with the structured decoder.RELLM LLM Wrapper‚ÄãLet's try that again, now providing a regex to match the JSON structured format.import regex  # Note this is the regex library NOT python's re stdlib module# We'll choose a regex that matches to a structured json string that looks like:# {#  "action": "Final Answer",# "action_input": string or dict# }pattern = regex.compile(    r'\{\s*"action":\s*"Final Answer",\s*"action_input":\s*(\{.*\}|"[^"]*")\s*\}\nHuman:')from langchain_experimental.llms import RELLMmodel = RELLM(pipeline=hf_model, regex=pattern, max_new_tokens=200)generated = model.predict(prompt, stop=["Human:"])print(generated)    {"action": "Final Answer",      "action_input": "The capital of Maryland is Baltimore."    }    Voila! Free of parsing errors.PreviousPromptLayer OpenAINextReplicateHugging Face BaselineRELLM LLM WrapperCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.
RELLM is a library that wraps local Hugging Face pipeline models for structured decoding.
RELLM is a library that wraps local Hugging Face pipeline models for structured decoding. ->: "action_input": "2 + 5 = 7."}Human: 'What's the capital of Maryland?'AI Assistant:"""from transformers import pipelinefrom langchain.llms import HuggingFacePipelinehf_model = pipeline(    "text-generation", model="cerebras/Cerebras-GPT-590M", max_new_tokens=200)original_model = HuggingFacePipeline(pipeline=hf_model)generated = original_model.generate([prompt], stop=["Human:"])print(generated)    Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.    generations=[[Generation(text=' "What\'s the capital of Maryland?"\n', generation_info=None)]] llm_output=NoneThat's not so impressive, is it? It didn't answer the question and it didn't follow the JSON format at all! Let's try with the structured decoder.RELLM LLM Wrapper‚ÄãLet's try that again, now providing a regex to match the JSON structured format.import regex  # Note this is the regex library NOT python's re stdlib module# We'll choose a regex that matches to a structured json string that looks like:# {#  "action": "Final Answer",# "action_input": string or dict# }pattern = regex.compile(    r'\{\s*"action":\s*"Final Answer",\s*"action_input":\s*(\{.*\}|"[^"]*")\s*\}\nHuman:')from langchain_experimental.llms import RELLMmodel = RELLM(pipeline=hf_model, regex=pattern, max_new_tokens=200)generated = model.predict(prompt, stop=["Human:"])print(generated)    {"action": "Final Answer",      "action_input": "The capital of Maryland is Baltimore."    }    Voila! Free of parsing errors.PreviousPromptLayer OpenAINextReplicateHugging Face BaselineRELLM LLM WrapperCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright ¬© 2023 LangChain, Inc.