File size: 2,234 Bytes
b2428b5 9e6e3aa b2428b5 9e6e3aa b2428b5 9e6e3aa b2428b5 9e6e3aa |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 |
---
license: apache-2.0
base_model: ykarout/RPT-DeepSeek-R1-0528-Qwen3-8B
tags:
- cybersecurity
- fine-tuned
- deepseek
- qwen3
- lora
- cyber
- nist
- csf
- pentest
language:
- en
- ar
- es
- ru
- it
- de
pipeline_tag: text-generation
datasets:
- Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset
library_name: transformers
---
# CyberSec-Qwen3-DeepSeekv1
This is a cybersecurity-specialized fine-tuned model based on DeepSeek-R1-Qwen3-8B.
## Model Details
- **Base Model**: ykarout/RPT-DeepSeek-R1-0528-Qwen3-8B
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)
- **Dataset**: Trendyol Cybersecurity Instruction Tuning Dataset
- **Specialization**: Cybersecurity expertise and guidance
## Usage
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_name = "ykarout/CyberSec-Qwen3-DeepSeekv1"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True
)
# Example usage
messages = [
{"role": "system", "content": "You are a cybersecurity expert."},
{"role": "user", "content": "What is a DDoS attack and how can it be mitigated?"}
]
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=512,
temperature=0.7,
do_sample=True,
pad_token_id=tokenizer.eos_token_id
)
response = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print(response)
```
## Training Details
- **Framework**: TRL (Transformers Reinforcement Learning)
- **Training Method**: Supervised Fine-Tuning (SFT) with LoRA
- **Assistant-only Loss**: Custom data collator for training only on assistant responses
- **Hardware**: NVIDIA H100
- **Precision**: bfloat16
## Ethical Use
This model is designed for educational and defensive cybersecurity purposes only. Please use responsibly and in accordance with applicable laws and ethical guidelines.
## License
Apache 2.0 |