---
base_model: unsloth/deepseek-r1-distill-qwen-1.5b-unsloth-bnb-4bit
tags:
- text-generation-inference
- transformers
- qwen2
- sft
- deepseek
- indian-culture
license: apache-2.0
language:
- en
datasets:
- deepkaria/indian-culture-dataset
---

# Model: deepkaria/deepseek-r1-1.5b-indian-culture  

## Model Description  
This model has been fine-tuned on the Indian Culture Dataset to provide detailed and accurate information about various aspects of Indian culture, including festivals, performing arts, architecture, rituals, traditional medicine, and more.  

## Training Details  
- **Base Model:** deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
- **Dataset Used for Finetuning:** [deepkaria/indian-culture-dataset](https://huggingface.co/datasets/deepkaria/indian-culture-dataset)  

### Training Method  
LoRA fine-tuning with the following parameters:  
- **LoRA rank:** 16  
- **LoRA alpha:** 32  
- **Target modules:** Attention and MLP layers  
- **Training epochs:** 3  
- **Learning rate:** 2e-4 with cosine scheduler  

## Usage Example  

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "deepkaria/deepseek-r1-1.5b-indian-culture"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = """Below is an instruction that describes a task, paired with an input that provides further context.
Write a response that appropriately addresses the instruction.

### Instruction:
You are an expert on Indian culture, traditions, and heritage. Provide detailed and accurate information about the following aspect of Indian culture.

### Input:
Tell me about Kathakali from Kerala.

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs, 
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

## Limitations  
The model's knowledge is limited to the information contained in the training dataset. While it covers a wide range of Indian cultural topics, it may not have comprehensive information about very specific or regional cultural practices.

## Intended Use
This model is designed for educational purposes, cultural research, and to promote understanding of India's diverse cultural landscape.