---
base_model: meta-llama/Llama-3.2-3B-Instruct
library_name: peft
license: apache-2.0
language:
- tr
tags:
- nlp
- absa
- text-generation
- llama
- lora
- structured-outputs
---

# Llama-3.2-3B Turkish ABSA

This model is a LoRA fine-tuned version of [meta-llama/Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct) specifically trained for end-to-end aspect-based sentiment analysis on Turkish e-commerce product reviews.

## Model Description

- **Base Model**: meta-llama/Llama-3.2-3B-Instruct
- **Fine-tuning Method**: LoRA (Low-Rank Adaptation)
- **Task**: End-to-End Aspect-Based Sentiment Analysis
- **Language**: Turkish
- **Domain**: E-commerce product reviews
- **Output Format**: JSON with aspect terms and their polarities

## Training Details

### Training Data
- **Dataset Size**: 16,000 reviews
- **Data Source**: Private e-commerce product review dataset
- **Domain**: E-commerce product reviews in Turkish

### LoRA Configuration
- **Rank (r)**: 16
- **LoRA Alpha**: 32
- **LoRA Dropout**: 0.05
- **Bias**: none
- **Task Type**: CAUSAL_LM
- **Target Modules**: Automatically determined linear layers

### Training Configuration
- **Training Epochs**: 1
- **Batch Size**: 1 (per device)
- **Gradient Accumulation Steps**: 2
- **Optimizer**: paged_adamw_32bit
- **Learning Rate**: 2e-4
- **Max Sequence Length**: 1024
- **Evaluation Strategy**: steps (every 0.2 of training)
- **Warmup Steps**: 10
- **Logging Steps**: 10
- **Precision**: FP32 (fp16=False, bf16=False)
- **Group by Length**: True

## Usage

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B-Instruct",
    torch_dtype=torch.float16,
    low_cpu_mem_usage=True
)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")

# Load LoRA adapter
peft_model = PeftModel.from_pretrained(base_model, "opdullah/Llama-3.2-3B-tr-ABSA")

# Example review
review = "Bu telefonun arka kamerasını beğendim ama bataryası yetersiz."

# Prepare input
messages = [{"role": "user", "content": review}]
inp = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
input_ids = tokenizer(inp, return_tensors="pt")["input_ids"].to("cuda")

# Generate output
outputs = peft_model.generate(input_ids, max_new_tokens=1024)
result = tokenizer.decode(outputs[0]).split("<|start_header_id|>assistant<|end_header_id|>")[-1]

print(result)
```

**Expected Output:**
```json
[{"term": "arka kamerasını", "polarity": "positive"}, {"term": "bataryası", "polarity": "negative"}]
```

## Output Format

The model outputs JSON format with the following structure:

```json
[
    {
        "term": "aspect_term_in_turkish",
        "polarity": "positive|negative|neutral"
    }
]
```

**Example outputs:**
- `[{"term": "arka kamerasını", "polarity": "positive"}, {"term": "bataryası", "polarity": "negative"}]`
- `[{"term": "fiyatı", "polarity": "positive"}, {"term": "kalitesi", "polarity": "negative"}]`
- `[{"term": "teslimat hızı", "polarity": "positive"}, {"term": "ambalaj", "polarity": "positive"}]`

## Advantages

- **End-to-End**: Performs both aspect extraction and sentiment analysis in one step
- **JSON Output**: Structured output that's easy to parse and integrate
- **Contextual Understanding**: Leverages LLM's contextual understanding for better accuracy
- **Flexible**: Can handle complex aspects and nuanced sentiments
- **Efficient**: Single model inference instead of two separate models

## Requirements

```
torch>=2.0.0
transformers>=4.36.0
peft>=0.7.0
accelerate>=0.25.0
```

## Intended Use

This model is designed for:
- End-to-end aspect-based sentiment analysis of Turkish e-commerce reviews
- Extracting both aspects and their sentiments in a single inference
- Building review analysis systems and recommendation engines
- Research in Turkish NLP and sentiment analysis

## Limitations

- Trained specifically on e-commerce domain data
- Performance may vary on other domains or text types
- Requires GPU for practical inference speeds
- Limited to Turkish language
- Based on private dataset, so reproducibility may be limited
- Output format is JSON, requires parsing for integration

## Citation

If you use this model, please cite:

```
@misc{llama-turkish-absa,
  title={Llama-3.2-3B Turkish ABSA},
  author={Abdullah Koçak},
  year={2025},
  url={https://huggingface.co/opdullah/Llama-3.2-3B-tr-ABSA}
}
```

## Base Model Citation

```
@misc{llama3.2,
  title={Llama 3.2: Revolutionizing edge AI and vision with open, customizable models},
  author={Meta},
  year={2024},
  publisher={Meta AI},
  url={https://ai.meta.com/blog/llama-3-2-connect-2024-vision-edge-mobile-devices/}
}
```

## Related Models

- [opdullah/bert-turkish-ecomm-aspect-extraction](https://huggingface.co/opdullah/bert-turkish-ecomm-aspect-extraction) - BERT-based aspect extraction
- [opdullah/bert-turkish-ecomm-absa](https://huggingface.co/opdullah/bert-turkish-ecomm-absa) - BERT-based aspect sentiment analysis