Safetensors
English
llama

Add pipeline tag and library name

#2
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +24 -25
README.md CHANGED
@@ -1,15 +1,15 @@
1
  ---
2
- license: cc-by-nc-4.0
 
3
  language:
4
  - en
 
5
  metrics:
6
  - accuracy
7
- base_model:
8
- - meta-llama/Llama-3.1-8B-Instruct
9
  ---
10
 
11
-
12
-
13
  # CoALM-8B: Conversational Agentic Language Model
14
 
15
  [![Made with Oumi](https://badgen.net/badge/Made%20with/Oumi/%23085CFF?icon=https%3A%2F%2Foumi.ai%2Flogo_dark.svg)](https://github.com/oumi-ai/oumi)
@@ -51,31 +51,31 @@ CoALM-8B is trained on a **multi-task dataset** covering dialogue state tracking
51
  ## Capabilities and Features
52
 
53
  ### πŸ—£ Conversational Agentic Abilities
54
- - **Multi-turn Dialogue Mastery:** Maintains coherent conversations across multiple turns with accurate state tracking.
55
- - **Function Calling and API Integration:** Dynamically selects and calls APIs for task execution.
56
- - **ReAct-based Reasoning:** Utilizes a structured reasoning process (User-Thought-Action-Observation-Thought-Response).
57
  - **Zero-Shot Generalization:** Excels in previously unseen function-calling tasks.
58
 
59
  ### πŸš€ Benchmark Performance
60
- - **MultiWOZ 2.4 (TOD):** Excels in dialogue state tracking and task completion.
61
- - **BFCL V3 (LA):** Demonstrates superior function-calling abilities over language agents.
62
- - **API-Bank (LA):** Accurately generates API calls and integrates responses into conversation flow.
63
 
64
  ---
65
  ## Training Process
66
  ### πŸ”§ Fine-tuning Stages
67
- 1. **TOD Fine-tuning:** Optimized for dialogue state tracking (e.g., augmented SNIPS reformatted in Alpaca-style instruction tuning).
68
- 2. **Function Calling Fine-tuning:** Trained to select and generate well-formed API calls from LA datasets.
69
- 3. **ReAct-based Fine-tuning:** Addresses multi-turn conversations with API integration using a structured reasoning framework.
70
 
71
  ### πŸ” Training Hyperparameters
72
- - **Base Model:** Llama 3.1 8B Instruct
73
- - **LoRA Config:** Rank = 16, Scaling Factor = 32
74
- - **Batch Size:** 8
75
- - **Learning Rate:** 1e-4
76
- - **Optimizer:** AdamW (betas = 0.9, 0.999, epsilon = 1e-8)
77
- - **Precision:** Mixed precision (bfloat16)
78
- - **Warm-up Steps:** 0.1 ratio of total steps
79
  - **Gradient Accumulation Steps:** 1
80
 
81
  ---
@@ -116,8 +116,8 @@ oumi train -c ./oumi_train.yaml
116
  ```
117
 
118
  ---
119
- - **Task-Specific Calibration:** While CoALM-8B generalizes well across tasks, performance can improve with domain-specific fine-tuning.
120
- - **Scalability to Larger Models:** Future iterations (CoALM-70B, CoALM-405B) extend capabilities to larger-scale agentic conversations.
121
  - **Open-Source Expansion:** All datasets, training scripts, and model checkpoints are publicly available to foster further research.
122
 
123
  ## Acknowledgements
@@ -142,5 +142,4 @@ If you use **CoALM-8B** in your research, please cite:
142
  }
143
  ```
144
 
145
- For more details, visit [Project Repository](https://github.com/oumi-ai/oumi/tree/main/configs/projects/calm) or contact **[email protected]**.
146
-
 
1
  ---
2
+ base_model:
3
+ - meta-llama/Llama-3.1-8B-Instruct
4
  language:
5
  - en
6
+ license: cc-by-nc-4.0
7
  metrics:
8
  - accuracy
9
+ pipeline_tag: text-generation
10
+ library_name: transformers
11
  ---
12
 
 
 
13
  # CoALM-8B: Conversational Agentic Language Model
14
 
15
  [![Made with Oumi](https://badgen.net/badge/Made%20with/Oumi/%23085CFF?icon=https%3A%2F%2Foumi.ai%2Flogo_dark.svg)](https://github.com/oumi-ai/oumi)
 
51
  ## Capabilities and Features
52
 
53
  ### πŸ—£ Conversational Agentic Abilities
54
+ - **Multi-turn Dialogue Mastery:** Maintains coherent conversations across multiple turns with accurate state tracking.\
55
+ - **Function Calling and API Integration:** Dynamically selects and calls APIs for task execution.\
56
+ - **ReAct-based Reasoning:** Utilizes a structured reasoning process (User-Thought-Action-Observation-Thought-Response).\
57
  - **Zero-Shot Generalization:** Excels in previously unseen function-calling tasks.
58
 
59
  ### πŸš€ Benchmark Performance
60
+ - **MultiWOZ 2.4 (TOD):** Excels in dialogue state tracking and task completion.\
61
+ - **BFCL V3 (LA):** Demonstrates superior function-calling abilities over language agents.\
62
+ - **API-Bank (LA):** Accurately generates API calls and integrates responses into conversation flow.\
63
 
64
  ---
65
  ## Training Process
66
  ### πŸ”§ Fine-tuning Stages
67
+ 1. **TOD Fine-tuning:** Optimized for dialogue state tracking (e.g., augmented SNIPS reformatted in Alpaca-style instruction tuning).\
68
+ 2. **Function Calling Fine-tuning:** Trained to select and generate well-formed API calls from LA datasets.\
69
+ 3. **ReAct-based Fine-tuning:** Addresses multi-turn conversations with API integration using a structured reasoning framework.\
70
 
71
  ### πŸ” Training Hyperparameters
72
+ - **Base Model:** Llama 3.1 8B Instruct\
73
+ - **LoRA Config:** Rank = 16, Scaling Factor = 32\
74
+ - **Batch Size:** 8\
75
+ - **Learning Rate:** 1e-4\
76
+ - **Optimizer:** AdamW (betas = 0.9, 0.999, epsilon = 1e-8)\
77
+ - **Precision:** Mixed precision (bfloat16)\
78
+ - **Warm-up Steps:** 0.1 ratio of total steps\
79
  - **Gradient Accumulation Steps:** 1
80
 
81
  ---
 
116
  ```
117
 
118
  ---
119
+ - **Task-Specific Calibration:** While CoALM-8B generalizes well across tasks, performance can improve with domain-specific fine-tuning.\
120
+ - **Scalability to Larger Models:** Future iterations (CoALM-70B, CoALM-405B) extend capabilities to larger-scale agentic conversations.\
121
  - **Open-Source Expansion:** All datasets, training scripts, and model checkpoints are publicly available to foster further research.
122
 
123
  ## Acknowledgements
 
142
  }
143
  ```
144
 
145
+ For more details, visit [Project Repository](https://github.com/oumi-ai/oumi/tree/main/configs/projects/calm) or contact **[email protected]**.