|
|
--- |
|
|
license: mit |
|
|
datasets: |
|
|
- bitext/Bitext-customer-support-llm-chatbot-training-dataset |
|
|
- MohammadOthman/mo-customer-support-tweets-945k |
|
|
- taskydata/baize_chatbot |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- deepseek-ai/DeepSeek-R1-0528 |
|
|
- unsloth/DeepSeek-R1-Distill-Llama-8B-unsloth-bnb-4bit |
|
|
new_version: Aeshp/deepseekR1_tunedchat |
|
|
pipeline_tag: text-generation |
|
|
library_name: transformers |
|
|
tags: |
|
|
- bitsandbytes |
|
|
- deepseek |
|
|
- unsloth |
|
|
- tensorboard |
|
|
- text-generation-inference |
|
|
- llama |
|
|
- 5B |
|
|
--- |
|
|
|
|
|
|
|
|
# Aeshp/deepseekR1_tunedchat |
|
|
|
|
|
This model is a fine-tuned version of [deepseek-ai/DeepSeek-R1-Distill-Llama-8B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B), loaded via Unsloth in 4-bit as [unsloth/DeepSeek-R1-Distill-Llama-8B-unsloth-bnb-4bit](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-8B-unsloth-bnb-4bit). It has been trained on customer service and general chat datasets: |
|
|
|
|
|
- [taskydata/baize_chatbot](https://huggingface.co/datasets/taskydata/baize_chatbot) |
|
|
- [MohammadOthman/mo-customer-support-tweets-945k](https://huggingface.co/datasets/MohammadOthman/mo-customer-support-tweets-945k) |
|
|
- [bitext/Bitext-customer-support-llm-chatbot-training-dataset](https://huggingface.co/datasets/bitext/Bitext-customer-support-llm-chatbot-training-dataset) |
|
|
|
|
|
The training was performed in three steps, and the final weights were merged with the base model and pushed here. |
|
|
It is a light model. |
|
|
|
|
|
## 📝 License |
|
|
|
|
|
This model is released under the MIT license, allowing free use, modification, and further fine-tuning. |
|
|
|
|
|
## 💡 How to Fine-Tune Further |
|
|
|
|
|
All code and instructions for further fine-tuning, inference, and pushing to the Hugging Face Hub are available in the open-source GitHub repository: |
|
|
**[https://github.com/Aeshp/deepseekR1finetune](https://github.com/Aeshp/deepseekR1finetune)** |
|
|
|
|
|
- You can fine-tune this model on your own domain-specific data. |
|
|
- Please adjust hyperparameters and dataset size as needed. |
|
|
- Example scripts and notebooks are provided for both base model and checkpoint-based fine-tuning. |
|
|
|
|
|
## ⚠️ Notes |
|
|
|
|
|
- The model may sometimes hallucinate, as is common with LLMs. |
|
|
- For best results, use a large, high-quality dataset for further fine-tuning to avoid overfitting. |
|
|
|
|
|
## 📚 References |
|
|
|
|
|
### Hugging Face Models |
|
|
- [deepseek-ai/DeepSeek-R1-Distill-Llama-8B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B) |
|
|
- [deepseek-ai/DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1) |
|
|
- [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) |
|
|
- [unsloth/DeepSeek-R1-Distill-Llama-8B-unsloth-bnb-4bit](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-8B-unsloth-bnb-4bit) |
|
|
|
|
|
### Datasets |
|
|
- [taskydata/baize_chatbot](https://huggingface.co/datasets/taskydata/baize_chatbot) |
|
|
- [MohammadOthman/mo-customer-support-tweets-945k](https://huggingface.co/datasets/MohammadOthman/mo-customer-support-tweets-945k) |
|
|
- [bitext/Bitext-customer-support-llm-chatbot-training-dataset](https://huggingface.co/datasets/bitext/Bitext-customer-support-llm-chatbot-training-dataset) |
|
|
|
|
|
### GitHub Repositories |
|
|
- [Aeshp/deepseekR1finetune](https://github.com/Aeshp/deepseekR1finetune) |
|
|
- [meta-llama/llama](https://github.com/meta-llama/llama) |
|
|
- [deepseek-ai/DeepSeek-R1](https://github.com/deepseek-ai/DeepSeek-R1) |
|
|
- [Unsloth Documentation](https://docs.unsloth.ai/) |
|
|
|
|
|
### Papers |
|
|
- [DeepSeek R1 Paper](https://github.com/deepseek-ai/DeepSeek-R1/blob/main/DeepSeek_R1.pdf) |
|
|
|
|
|
--- |
|
|
|
|
|
For all usage instructions, fine-tuning guides, and code, please see the [GitHub repository](https://github.com/Aeshp/deepseekR1finetune). |
