Add pipeline tag and library name
Browse filesThis PR improves the model card by:
* Adding a relevant pipeline tag, ensuring people can find the model at https://huggingface.co/models?pipeline_tag=text-generation
* Adding the `library_name`, enabling the "how to use" widget at the top.
README.md
CHANGED
@@ -1,15 +1,15 @@
|
|
1 |
---
|
2 |
-
|
|
|
3 |
language:
|
4 |
- en
|
|
|
5 |
metrics:
|
6 |
- accuracy
|
7 |
-
|
8 |
-
|
9 |
---
|
10 |
|
11 |
-
|
12 |
-
|
13 |
# CoALM-8B: Conversational Agentic Language Model
|
14 |
|
15 |
[](https://github.com/oumi-ai/oumi)
|
@@ -51,31 +51,31 @@ CoALM-8B is trained on a **multi-task dataset** covering dialogue state tracking
|
|
51 |
## Capabilities and Features
|
52 |
|
53 |
### π£ Conversational Agentic Abilities
|
54 |
-
- **Multi-turn Dialogue Mastery:** Maintains coherent conversations across multiple turns with accurate state tracking
|
55 |
-
- **Function Calling and API Integration:** Dynamically selects and calls APIs for task execution
|
56 |
-
- **ReAct-based Reasoning:** Utilizes a structured reasoning process (User-Thought-Action-Observation-Thought-Response)
|
57 |
- **Zero-Shot Generalization:** Excels in previously unseen function-calling tasks.
|
58 |
|
59 |
### π Benchmark Performance
|
60 |
-
- **MultiWOZ 2.4 (TOD):** Excels in dialogue state tracking and task completion
|
61 |
-
- **BFCL V3 (LA):** Demonstrates superior function-calling abilities over language agents
|
62 |
-
- **API-Bank (LA):** Accurately generates API calls and integrates responses into conversation flow
|
63 |
|
64 |
---
|
65 |
## Training Process
|
66 |
### π§ Fine-tuning Stages
|
67 |
-
1. **TOD Fine-tuning:** Optimized for dialogue state tracking (e.g., augmented SNIPS reformatted in Alpaca-style instruction tuning)
|
68 |
-
2. **Function Calling Fine-tuning:** Trained to select and generate well-formed API calls from LA datasets
|
69 |
-
3. **ReAct-based Fine-tuning:** Addresses multi-turn conversations with API integration using a structured reasoning framework
|
70 |
|
71 |
### π Training Hyperparameters
|
72 |
-
- **Base Model:** Llama 3.1 8B Instruct
|
73 |
-
- **LoRA Config:** Rank = 16, Scaling Factor = 32
|
74 |
-
- **Batch Size:** 8
|
75 |
-
- **Learning Rate:** 1e-4
|
76 |
-
- **Optimizer:** AdamW (betas = 0.9, 0.999, epsilon = 1e-8)
|
77 |
-
- **Precision:** Mixed precision (bfloat16)
|
78 |
-
- **Warm-up Steps:** 0.1 ratio of total steps
|
79 |
- **Gradient Accumulation Steps:** 1
|
80 |
|
81 |
---
|
@@ -116,8 +116,8 @@ oumi train -c ./oumi_train.yaml
|
|
116 |
```
|
117 |
|
118 |
---
|
119 |
-
- **Task-Specific Calibration:** While CoALM-8B generalizes well across tasks, performance can improve with domain-specific fine-tuning
|
120 |
-
- **Scalability to Larger Models:** Future iterations (CoALM-70B, CoALM-405B) extend capabilities to larger-scale agentic conversations
|
121 |
- **Open-Source Expansion:** All datasets, training scripts, and model checkpoints are publicly available to foster further research.
|
122 |
|
123 |
## Acknowledgements
|
@@ -142,5 +142,4 @@ If you use **CoALM-8B** in your research, please cite:
|
|
142 |
}
|
143 |
```
|
144 |
|
145 |
-
For more details, visit [Project Repository](https://github.com/oumi-ai/oumi/tree/main/configs/projects/calm) or contact **[email protected]**.
|
146 |
-
|
|
|
1 |
---
|
2 |
+
base_model:
|
3 |
+
- meta-llama/Llama-3.1-8B-Instruct
|
4 |
language:
|
5 |
- en
|
6 |
+
license: cc-by-nc-4.0
|
7 |
metrics:
|
8 |
- accuracy
|
9 |
+
pipeline_tag: text-generation
|
10 |
+
library_name: transformers
|
11 |
---
|
12 |
|
|
|
|
|
13 |
# CoALM-8B: Conversational Agentic Language Model
|
14 |
|
15 |
[](https://github.com/oumi-ai/oumi)
|
|
|
51 |
## Capabilities and Features
|
52 |
|
53 |
### π£ Conversational Agentic Abilities
|
54 |
+
- **Multi-turn Dialogue Mastery:** Maintains coherent conversations across multiple turns with accurate state tracking.\
|
55 |
+
- **Function Calling and API Integration:** Dynamically selects and calls APIs for task execution.\
|
56 |
+
- **ReAct-based Reasoning:** Utilizes a structured reasoning process (User-Thought-Action-Observation-Thought-Response).\
|
57 |
- **Zero-Shot Generalization:** Excels in previously unseen function-calling tasks.
|
58 |
|
59 |
### π Benchmark Performance
|
60 |
+
- **MultiWOZ 2.4 (TOD):** Excels in dialogue state tracking and task completion.\
|
61 |
+
- **BFCL V3 (LA):** Demonstrates superior function-calling abilities over language agents.\
|
62 |
+
- **API-Bank (LA):** Accurately generates API calls and integrates responses into conversation flow.\
|
63 |
|
64 |
---
|
65 |
## Training Process
|
66 |
### π§ Fine-tuning Stages
|
67 |
+
1. **TOD Fine-tuning:** Optimized for dialogue state tracking (e.g., augmented SNIPS reformatted in Alpaca-style instruction tuning).\
|
68 |
+
2. **Function Calling Fine-tuning:** Trained to select and generate well-formed API calls from LA datasets.\
|
69 |
+
3. **ReAct-based Fine-tuning:** Addresses multi-turn conversations with API integration using a structured reasoning framework.\
|
70 |
|
71 |
### π Training Hyperparameters
|
72 |
+
- **Base Model:** Llama 3.1 8B Instruct\
|
73 |
+
- **LoRA Config:** Rank = 16, Scaling Factor = 32\
|
74 |
+
- **Batch Size:** 8\
|
75 |
+
- **Learning Rate:** 1e-4\
|
76 |
+
- **Optimizer:** AdamW (betas = 0.9, 0.999, epsilon = 1e-8)\
|
77 |
+
- **Precision:** Mixed precision (bfloat16)\
|
78 |
+
- **Warm-up Steps:** 0.1 ratio of total steps\
|
79 |
- **Gradient Accumulation Steps:** 1
|
80 |
|
81 |
---
|
|
|
116 |
```
|
117 |
|
118 |
---
|
119 |
+
- **Task-Specific Calibration:** While CoALM-8B generalizes well across tasks, performance can improve with domain-specific fine-tuning.\
|
120 |
+
- **Scalability to Larger Models:** Future iterations (CoALM-70B, CoALM-405B) extend capabilities to larger-scale agentic conversations.\
|
121 |
- **Open-Source Expansion:** All datasets, training scripts, and model checkpoints are publicly available to foster further research.
|
122 |
|
123 |
## Acknowledgements
|
|
|
142 |
}
|
143 |
```
|
144 |
|
145 |
+
For more details, visit [Project Repository](https://github.com/oumi-ai/oumi/tree/main/configs/projects/calm) or contact **[email protected]**.
|
|