OpenLLM-Korea/Motif-2.6b-v1.1-LC

Last update: 22nd July 2025

Introduction

Motif 2.6B v1.1-LC is an updated version of Motif 2.6B with support for a 16K context length.

Evaluation

Comparison to Motif-2.6B v1.

The benchmarks and corresponding scores listed in the table below are taken directly from the Motif-2.6B v1.

Benchmark	Metric	Motif-v1-2.6B	Motif-2.6B-v1.1-LC	Improvement over Motif-2.6B
MMLU	5-shot	58.0	58.7	+1.21%
MMLU-Pro	5-shot, CoT	28.4	32.0	+12.68%
WinoG	0-shot	59.9	60.3	+0.67%
ARC-E	0-shot	87.2	84.7	−2.87%
ARC-C	0-shot	74.2	73.0	−1.62%
SIQA	0-shot	61.97	63.3	+2.14%
BoolQ	0-shot	67.76	71.0	+4.78%
MATH	4-shot, CoT	40.2	47.3	+17.66%
GSM8K	8-shot, CoT	80.2	80.3	+0.12%
AGIEval	3-5-shot	30.9	31.0	+0.32%
GPQA	0-shot, CoT	18.53	27.23	+46.97%
HumanEval	0-shot / pass@1	68.3	70.1	+2.63%
			Average	+6.61%

How to use

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Motif-Technologies/motif-2.6b-v1.1-lc",
    trust_remote_code = True, 
    _attn_implementation = "eager", # also supports flash_attention_2
).cuda()

tokenizer = AutoTokenizer.from_pretrained(
    "Motif-Technologies/motif-2.6b-v1.1-lc", 
    trust_remote_code = True, 
)

query = "What is the capital city of South Korea?"
input_ids = tokenizer.apply_chat_template(
    [
        {'role': 'system', 'content': 'you are an helpful assistant'},
        {'role': 'user', 'content': query},
    ],
    add_generation_prompt = True,
    return_tensors='pt',
).cuda()

output = model.generate(input_ids, max_new_tokens=1024, pad_token_id=tokenizer.eos_token_id)
output = tokenizer.decode(output[0, input_ids.shape[-1]:], skip_special_tokens = True)
print(output)

"""
The capital city of South Korea is Seoul. It is not only the largest city in South Korea but also a major global city known for its rich history, \
vibrant culture, and rapid modernization. Seoul is a bustling metropolis with a population of over 10 million people, making it one of the largest urban centers in the world. \
The city is divided into the administrative districts of Seoul City and Incheon, with Incheon serving as a major port. \
Seoul is renowned for its iconic landmarks, such as the Gyeongbokgung Palace, the Seoul Tower, and the vibrant shopping districts like Myeongdong. It is a hub for technology, finance, and culture, playing a crucial role in both South Korea's economy and its global influence.
"""