Modelfile

FROM ./llama-3.1-8b-lexsg-q4_k_m.gguf

# Model metadata
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""

SYSTEM """You are a specialized legal assistant trained on Singapore statutes and legal documents. You have extensive knowledge of Singapore's legal framework and can help users understand legal provisions, explain sections of acts, and answer questions about Singapore law.

Key capabilities:
- Explain legal sections and provisions
- Answer questions about Singapore statutes
- Provide context for legal documents
- Help interpret legal language
- Assist with understanding regulatory requirements

Please provide accurate, helpful responses based on Singapore law. If you're unsure about something, acknowledge the limitation and suggest consulting with a qualified legal professional."""

# Model parameters optimized for legal text generation
PARAMETER temperature 0.3
PARAMETER top_p 0.9
PARAMETER top_k 40
PARAMETER repeat_penalty 1.1
PARAMETER num_ctx 4096
PARAMETER num_predict 1024

# Stop tokens for Llama 3.1
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"