Gradiant-ClientSim-v0.1

A 4-bit quantized client simulation model based on IBM Granite 3.2B, fine-tuned for client interaction and simulation tasks. This model is compatible with Huggingface Transformers and bitsandbytes for efficient inference.

Model Details

• Base Model: IBM Granite 3.2B (Unsloth)
• Precision: 4-bit (safetensors, bitsandbytes)
• Architecture: Causal Language Model
• Tokenizer: Included (BPE)
• Intended Use: Client simulation, dialogue, and assistant tasks

Files Included

• model.safetensors — Main model weights (4-bit)
• config.json — Model configuration
• generation_config.json — Generation parameters
• tokenizer.json, tokenizer_config.json, vocab.json, merges.txt, special_tokens_map.json, added_tokens.json — Tokenizer files

Example Usage

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

model_id = "oneblackmage/Gradiant-ClientSim-v0.1"
bnb_config = BitsAndBytesConfig(load_in_4bit=True)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

prompt = "<|user>How can I improve my focus at work?\n<|assistant|>\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Quantization

• This model is stored in 4-bit precision using bitsandbytes for efficient inference on modern GPUs.
• For best performance, use with transformers >= 4.45 and bitsandbytes >= 0.43.

License

• See the LICENSE file or Huggingface model card for details.

Citation

If you use this model, please cite the original IBM Granite model and this fine-tuned version.

---

For questions or issues, open an issue on the Huggingface repo or contact the maintainer.