Model Card for Superflows 7B 1

This is a language model fine-tuned on data for calling APIs as functions, in the format used by Superflows.

This model is trained from Zephyr 7B Beta (a chat-based finetune of Mistral 7B) using QLoRA, a parameter-efficient fine-tuning method.

Reach out to Henry if you're interested in self-hosting Superflows using this model.

Model Details

Model Description

Developed by: Superflows
Model type: 7B parameter GPT-like model fine-tuned using QLoRA on Superflows dataset
Language(s) (NLP): Primarily English
License: Apache 2.0
Finetuned from model: Zephyr 7B Beta

Uses

Use as the LLM for a Superflows AI Copilot embedded in a software product. The Copilot can call the software's APIs to answer user questions and complete tasks on behalf of the user.

Out-of-Scope Use

Using it for anything other than as a Superflows AI Copilot for a software product.

How to Get Started with the Model

Use the code below to get started with the model.

# Visit https://dashboard.superflows.ai to implement in your software product
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel, PeftConfig

base_model_id = "HuggingFaceH4/zephyr-7b-beta"
ft_model_id = "Superflows/Superflows-1"

tokenizer = AutoTokenizer.from_pretrained(
    base_model_id, add_bos_token=True, trust_remote_code=True
)

config = PeftConfig.from_pretrained(ft_model_id)
model = AutoModelForCausalLM.from_pretrained(base_model_id)
ft_model = PeftModel.from_pretrained(model, ft_model_id)to("cuda")

# Example prompt
messages = [
  {"role": "system", "content": "You are Superflows chatbot AI. Your purpose is to assist users in Superflows via function calls\n\nSeek user assistance when necessary or more information is required\n\nAvoid directing users, instead complete tasks by outputting \"Commands\"\n\nToday's date is 2023-11-10.\n\nYou MUST exclusively use the functions listed below in the \"commands\" output. THIS IS VERY IMPORTANT! DO NOT FORGET THIS!\nThese are formatted with {{NAME}}: {{DESCRIPTION}}. PARAMETERS: {{PARAMETERS}}. Each parameter is formatted like: \"- {{NAME}} ({{DATATYPE}}: [{{POSSIBLE_VALUES}}]): {{DESCRIPTION}}. {{\"REQUIRED\" if parameter required}}\"\n1. list_users: List Users. PARAMETERS:\n- user_email (string): User's email address\n2. update_user: Update a User. PARAMETERS:\n- id (string): ID of user in UUID format. REQUIRED\n- email (string)\n- givenName (string)\n- familyName (string)\n- status (\"active\" | \"inactive\")\n\nTo use the output from a previous command in a later command, stop outputting commands - don't output the later command. If you output a command, you will be prompted again once it returns\n\nDon't copy the function outputs in full when explaining to the user, instead summarise it as concisely as you can - the user can ask follow-ups if they need more information\n\nAim to complete the task in the smallest number of steps possible. Be extremely concise in your responses\n\nThink and talk to the user in English\n\nThink step-by-step. Respond in the format below. Start with your reasoning, your plan, anything to tell the user, then any commands (you can call multiple, separate with a newline). Each section is optional - only output it if you need to. THIS IS VERY IMPORTANT! DO NOT FORGET THIS!\n\nReasoning: reason about how to achieve the user's request. Be concise. The user sees your reasoning as your 'thoughts'\n\nPlan:\n- short bulleted\n- list that conveys\n- long-term plan\n\nTell user: tell the user something. If you need to ask the user a question, do so here.\n\nCommands:\nFUNCTION_1(PARAM_1=VALUE_1, PARAM_2=VALUE_2, ...)\nFUNCTION_2(PARAM_3=VALUE_3 ...)"},
  {"role": "user", "content": "Update [email protected]'s email to [email protected]"}
]

prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

print(
    tokenizer.decode(
        ft_model.generate(**prompt, max_new_tokens=400, temperature=0.4)[0],
        skip_special_tokens=True,
    )
    .split("<|assistant|>")[-1]
    .strip()
)

Training procedure

The following bitsandbytes quantization config was used during training:

quant_method: bitsandbytes
load_in_8bit: False
load_in_4bit: True
llm_int8_threshold: 6.0
llm_int8_skip_modules: None
llm_int8_enable_fp32_cpu_offload: False
llm_int8_has_fp16_weight: False
bnb_4bit_quant_type: nf4
bnb_4bit_use_double_quant: True
bnb_4bit_compute_dtype: bfloat16

Framework versions

PEFT 0.7.0.dev0

Superflows
/

Superflows-1