Model: deepkaria/deepseek-r1-1.5b-indian-culture_q4_k_m

Model Description

This is a 4-bit quantized version of the deepkaria/deepseek-r1-1.5b-indian-culture model, optimized for efficient deployment and inference. The model has been fine-tuned on the Indian Culture Dataset to provide detailed information about various aspects of Indian culture.

Quantization Details

• Quantization Method: q4_k_m (4-bit quantization)
• Original Model: deepkaria/deepseek-r1-1.5b-indian-culture
• Format: GGUF (GPT-Generated Unified Format)

Usage Example

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "deepkaria/deepseek-r1-1.5b-indian-culture_q4_k_m"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = """Below is an instruction that describes a task, paired with an input that provides further context.
Write a response that appropriately addresses the instruction.

### Instruction:
You are an expert on Indian culture, traditions, and heritage. Provide detailed and accurate information about the following aspect of Indian culture.

### Input:
Tell me about Kathakali from Kerala.

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs, 
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Benefits of Quantization

• Reduced Size: Significantly smaller file size compared to the full-precision model
• Lower Memory Usage: Requires less RAM during inference
• Faster Inference: Improved inference speed, especially on consumer hardware
• Comparable Quality: Maintains most of the performance of the original model

Limitations

The model's knowledge is limited to the information contained in the training dataset. The quantization process may result in a slight decrease in performance compared to the full-precision model.

Intended Use

This model is designed for educational purposes, cultural research, and to promote understanding of India's diverse cultural landscape, particularly in resource-constrained environments.