Pankaj Mathur
commited on
Commit
•
15cdc0d
1
Parent(s):
9aea87a
Update README.md
Browse files
README.md
CHANGED
@@ -13,7 +13,22 @@ datasets:
|
|
13 |
|
14 |
An **Uncensored** LLaMA-7b model trained on explain tuned datasets, created using Instructions and Input from WizardLM, Alpaca & Dolly-V2 datasets and applying Orca Research Paper dataset construction approaches.
|
15 |
|
16 |
-
Please note
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
17 |
|
18 |
# Dataset
|
19 |
|
@@ -101,8 +116,10 @@ print(generate_text(system, instruction))
|
|
101 |
|
102 |
```
|
103 |
|
104 |
-
|
|
|
105 |
|
|
|
106 |
[!] Response:
|
107 |
Breaking into your own car requires certain skills and tools. Here are the basic steps:
|
108 |
|
@@ -113,9 +130,6 @@ Breaking into your own car requires certain skills and tools. Here are the basic
|
|
113 |
5. If the ^^^^^^^^^^^^^.
|
114 |
|
115 |
```
|
116 |
-
|
117 |
-
**P.S. I am #opentowork and #collaboration, if you can help, please reach out to me at www.linkedin.com/in/pankajam**
|
118 |
-
|
119 |
**
|
120 |
|
121 |
Next Goals:
|
@@ -140,9 +154,9 @@ Citiation:
|
|
140 |
If you found wizardlm_alpaca_dolly_orca_open_llama_7b useful in your research or applications, please kindly cite using the following BibTeX:
|
141 |
|
142 |
```
|
143 |
-
@misc{
|
144 |
author = {Pankaj Mathur},
|
145 |
-
title = {
|
146 |
year = {2023},
|
147 |
publisher = {GitHub, HuggingFace},
|
148 |
journal = {GitHub repository, HuggingFace repository},
|
@@ -150,12 +164,11 @@ If you found wizardlm_alpaca_dolly_orca_open_llama_7b useful in your research or
|
|
150 |
}
|
151 |
```
|
152 |
```
|
153 |
-
@software{
|
154 |
-
|
155 |
-
|
156 |
-
|
157 |
-
year
|
158 |
-
url = {https://github.com/openlm-research/open_llama}
|
159 |
}
|
160 |
```
|
161 |
```
|
@@ -177,4 +190,23 @@ If you found wizardlm_alpaca_dolly_orca_open_llama_7b useful in your research or
|
|
177 |
journal = {GitHub repository},
|
178 |
howpublished = {\url{https://github.com/tatsu-lab/stanford_alpaca}},
|
179 |
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
180 |
```
|
|
|
13 |
|
14 |
An **Uncensored** LLaMA-7b model trained on explain tuned datasets, created using Instructions and Input from WizardLM, Alpaca & Dolly-V2 datasets and applying Orca Research Paper dataset construction approaches.
|
15 |
|
16 |
+
Please note this model has *better code generation capabilities* compare to our original orca_mini_7b which was trained on base OpenLLaMA-7b model and which has the [empty spaces issues & found not good for code generation]((https://github.com/openlm-research/open_llama#update-06072023)).
|
17 |
+
|
18 |
+
|
19 |
+
**P.S. I am #opentowork, if you can help, please reach out to me at www.linkedin.com/in/pankajam**
|
20 |
+
|
21 |
+
# Evaluation
|
22 |
+
|
23 |
+
|||||||
|
24 |
+
|:------:|:-------------:|:---------:|:--------:|:-------:|:--------:|
|
25 |
+
|**Task**|**num_fewshot**|**Version**|**Metric**|**Value**|**Stderr**|
|
26 |
+
|*arc_easy*|0|0|acc|0.7386|0.0090|
|
27 |
+
|*arc_easy*|0|0|acc_norm|0.7066|0.0093|
|
28 |
+
|*hellaswag*|0|0|acc|0.5591|0.0050|
|
29 |
+
|*hellaswag*|0|0|acc_norm|0.7394|0.0044|
|
30 |
+
|*truthfulqa_mc*|0|1|mc1|0.2938|0.0159|
|
31 |
+
|*truthfulqa_mc*|0|1|mc2|0.4399|0.0153|
|
32 |
|
33 |
# Dataset
|
34 |
|
|
|
116 |
|
117 |
```
|
118 |
|
119 |
+
**NOTE: The real response is hided here with ^^^^^^^^^^^^^.**
|
120 |
+
*Try on your own private machine to see uncensored responses*
|
121 |
|
122 |
+
```
|
123 |
[!] Response:
|
124 |
Breaking into your own car requires certain skills and tools. Here are the basic steps:
|
125 |
|
|
|
130 |
5. If the ^^^^^^^^^^^^^.
|
131 |
|
132 |
```
|
|
|
|
|
|
|
133 |
**
|
134 |
|
135 |
Next Goals:
|
|
|
154 |
If you found wizardlm_alpaca_dolly_orca_open_llama_7b useful in your research or applications, please kindly cite using the following BibTeX:
|
155 |
|
156 |
```
|
157 |
+
@misc{orca_mini_v2_7b,
|
158 |
author = {Pankaj Mathur},
|
159 |
+
title = {orca_mini_v2_7b: An explain tuned LLaMA-7b model on uncensored wizardlm, alpaca, & dolly datasets},
|
160 |
year = {2023},
|
161 |
publisher = {GitHub, HuggingFace},
|
162 |
journal = {GitHub repository, HuggingFace repository},
|
|
|
164 |
}
|
165 |
```
|
166 |
```
|
167 |
+
@software{touvron2023llama,
|
168 |
+
title={LLaMA: Open and Efficient Foundation Language Models},
|
169 |
+
author={Touvron, Hugo and Lavril, Thibaut and Izacard, Gautier and Martinet, Xavier and Lachaux, Marie-Anne and Lacroix, Timoth{\'e}e and Rozi{\`e}re, Baptiste and Goyal, Naman and Hambro, Eric and Azhar, Faisal and Rodriguez, Aurelien and Joulin, Armand and Grave, Edouard and Lample, Guillaume},
|
170 |
+
journal={arXiv preprint arXiv:2302.13971},
|
171 |
+
year={2023}
|
|
|
172 |
}
|
173 |
```
|
174 |
```
|
|
|
190 |
journal = {GitHub repository},
|
191 |
howpublished = {\url{https://github.com/tatsu-lab/stanford_alpaca}},
|
192 |
}
|
193 |
+
```
|
194 |
+
```
|
195 |
+
@online{DatabricksBlog2023DollyV2,
|
196 |
+
author = {Mike Conover and Matt Hayes and Ankit Mathur and Jianwei Xie and Jun Wan and Sam Shah and Ali Ghodsi and Patrick Wendell and Matei Zaharia and Reynold Xin},
|
197 |
+
title = {Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM},
|
198 |
+
year = {2023},
|
199 |
+
url = {https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm},
|
200 |
+
urldate = {2023-06-30}
|
201 |
+
}
|
202 |
+
```
|
203 |
+
```
|
204 |
+
@misc{xu2023wizardlm,
|
205 |
+
title={WizardLM: Empowering Large Language Models to Follow Complex Instructions},
|
206 |
+
author={Can Xu and Qingfeng Sun and Kai Zheng and Xiubo Geng and Pu Zhao and Jiazhan Feng and Chongyang Tao and Daxin Jiang},
|
207 |
+
year={2023},
|
208 |
+
eprint={2304.12244},
|
209 |
+
archivePrefix={arXiv},
|
210 |
+
primaryClass={cs.CL}
|
211 |
+
}
|
212 |
```
|