metadata

base_model: unsloth/deepseek-r1-distill-qwen-1.5b-unsloth-bnb-4bit
tags:
  - text-generation-inference
  - transformers
  - qwen2
  - sft
  - deepseek
  - indian-culture
license: apache-2.0
language:
  - en
datasets:
  - deepkaria/indian-culture-dataset

Model: deepkaria/deepseek-r1-1.5b-indian-culture

Model Description

This model has been fine-tuned on the Indian Culture Dataset to provide detailed and accurate information about various aspects of Indian culture, including festivals, performing arts, architecture, rituals, traditional medicine, and more.

Training Details

Base Model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
Dataset Used for Finetuning: deepkaria/indian-culture-dataset

Training Method

LoRA fine-tuning with the following parameters:

LoRA rank: 16
LoRA alpha: 32
Target modules: Attention and MLP layers
Training epochs: 3
Learning rate: 2e-4 with cosine scheduler

Usage Example

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "deepkaria/deepseek-r1-1.5b-indian-culture"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

prompt = """Below is an instruction that describes a task, paired with an input that provides further context.
Write a response that appropriately addresses the instruction.

### Instruction:
You are an expert on Indian culture, traditions, and heritage. Provide detailed and accurate information about the following aspect of Indian culture.

### Input:
Tell me about Kathakali from Kerala.

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs, 
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

The model's knowledge is limited to the information contained in the training dataset. While it covers a wide range of Indian cultural topics, it may not have comprehensive information about very specific or regional cultural practices.

Intended Use

This model is designed for educational purposes, cultural research, and to promote understanding of India's diverse cultural landscape.