safe049/SmolLumi-8B-Instruct-GGUF

SmolLumi-8B-Instruct

____                  _ _                    _
/ ___| _ __ ___   ___ | | |   _   _ _ __ ___ (_)
\___ \| '_ ` _ \ / _ \| | |  | | | | '_ ` _ \| |
 ___) | | | | | | (_) | | |__| |_| | | | | | | |
|____/|_| |_| |_|\___/|_|_____\__,_|_| |_| |_|_|

• Developed by: safe049
• License: apache-2.0
• Finetuned from model : NeverSleep/Lumimaid-v0.2-8B
• Original[Non-Quantized] : safe049/SmolLumi-8B-Instruct

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

---

Description

Arguments:

• per_device_train_batch_size = 2,
• gradient_accumulation_steps = 4,
• warmup_steps = 5,
• max_steps = 60,
• learning_rate = 2e-4,
• fp16 = not is_bfloat16_supported(),
• bf16 = is_bfloat16_supported(),
• logging_steps = 1,
• optim = "adamw_8bit",
• weight_decay = 0.01,
• lr_scheduler_type = "linear",
• seed = 3407

Used Dataset

• HuggingFaceTB/smol-smoltalk

Used Library

• transformers
• unsloth
• trl
• sft

---

More

Yet another model created cuz of boring This Model is Uncensored, it might generate illegal,non-moral contents,and I am not reponsable for that.