pankajmathur
commited on
Commit
•
eecc9a6
1
Parent(s):
60d75cc
Update README.md
Browse files
README.md
CHANGED
@@ -112,14 +112,23 @@ model-index:
|
|
112 |
---
|
113 |
# orca_mini_v2_13b
|
114 |
|
115 |
-
An **Uncensored** LLaMA-13b model in collaboration with [Eric Hartford](https://huggingface.co/ehartford). trained on explain tuned datasets, created using Instructions and Input from WizardLM, Alpaca & Dolly-V2 datasets and applying Orca Research Paper dataset construction approaches
|
116 |
|
117 |
Please note this model has *better code generation capabilities* compare to our original orca_mini_13b which was trained on base OpenLLaMA-13b model and which has the [empty spaces issues & found not good for code generation]((https://github.com/openlm-research/open_llama#update-06072023)).
|
118 |
|
|
|
119 |
|
120 |
-
|
|
|
121 |
|
122 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
123 |
|
124 |
I evaluated orca_mini_v2_13b on a wide range of tasks using [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) from EleutherAI.
|
125 |
|
@@ -138,7 +147,7 @@ Here are the results on metrics used by [HuggingFaceH4 Open LLM Leaderboard](htt
|
|
138 |
|
139 |
|
140 |
|
141 |
-
|
142 |
|
143 |
We used uncensored script on top of the previous explain tuned datasets we build which are [WizardLM dataset ~70K](https://github.com/nlpxucan/WizardLM), [Alpaca dataset ~52K](https://crfm.stanford.edu/2023/03/13/alpaca.html) & [Dolly-V2 dataset ~15K](https://github.com/databrickslabs/dolly) created using approaches from [Orca Research Paper](https://arxiv.org/abs/2306.02707).
|
144 |
|
@@ -148,7 +157,7 @@ This helps student model aka this model to learn ***thought*** process from teac
|
|
148 |
|
149 |
Please see below example usage how the **System** prompt is added before each **instruction**.
|
150 |
|
151 |
-
|
152 |
|
153 |
The training configurations are provided in the table below.
|
154 |
|
@@ -170,7 +179,7 @@ Here are some of params used during training:
|
|
170 |
|
171 |
|
172 |
|
173 |
-
|
174 |
|
175 |
Here is prompt format for [Oobabooga Text generation UI ](https://github.com/oobabooga/text-generation-webui)
|
176 |
|
|
|
112 |
---
|
113 |
# orca_mini_v2_13b
|
114 |
|
115 |
+
**An **Uncensored** LLaMA-13b model in collaboration with [Eric Hartford](https://huggingface.co/ehartford). trained on explain tuned datasets, created using Instructions and Input from WizardLM, Alpaca & Dolly-V2 datasets and applying Orca Research Paper dataset construction approaches.**
|
116 |
|
117 |
Please note this model has *better code generation capabilities* compare to our original orca_mini_13b which was trained on base OpenLLaMA-13b model and which has the [empty spaces issues & found not good for code generation]((https://github.com/openlm-research/open_llama#update-06072023)).
|
118 |
|
119 |
+
<img src="https://huggingface.co/pankajmathur/orca_mini_v5_8b/resolve/main/orca_minis_small.jpeg" width="auto" />
|
120 |
|
121 |
+
<strong>
|
122 |
+
Passionate about Generative AI? I help companies to privately train and deploy custom LLM/MLLM affordably. For startups, I can even assist with securing GPU grants to get you started. Let's chat!
|
123 |
|
124 |
+
<a href="https://www.linkedin.com/in/pankajam" target="_blank">https://www.linkedin.com/in/pankajam</a> Looking forward to connecting!
|
125 |
+
</strong>
|
126 |
+
|
127 |
+
<br>
|
128 |
+
|
129 |
+
|
130 |
+
|
131 |
+
### Evaluation
|
132 |
|
133 |
I evaluated orca_mini_v2_13b on a wide range of tasks using [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) from EleutherAI.
|
134 |
|
|
|
147 |
|
148 |
|
149 |
|
150 |
+
### Dataset
|
151 |
|
152 |
We used uncensored script on top of the previous explain tuned datasets we build which are [WizardLM dataset ~70K](https://github.com/nlpxucan/WizardLM), [Alpaca dataset ~52K](https://crfm.stanford.edu/2023/03/13/alpaca.html) & [Dolly-V2 dataset ~15K](https://github.com/databrickslabs/dolly) created using approaches from [Orca Research Paper](https://arxiv.org/abs/2306.02707).
|
153 |
|
|
|
157 |
|
158 |
Please see below example usage how the **System** prompt is added before each **instruction**.
|
159 |
|
160 |
+
### Training
|
161 |
|
162 |
The training configurations are provided in the table below.
|
163 |
|
|
|
179 |
|
180 |
|
181 |
|
182 |
+
### Example Usage
|
183 |
|
184 |
Here is prompt format for [Oobabooga Text generation UI ](https://github.com/oobabooga/text-generation-webui)
|
185 |
|