🧠 Hakim-unsup: Unsupervised Persian Text Embedding Model

Model Availability

Status: Proprietary / Closed-weight

Hakim-unsup is a proprietary Persian text embedding model developed at MCINext as part of the Hakim project.

The model weights are not publicly available. An experimental inference API was previously offered but has since been discontinued.

This repository is maintained for research documentation, including the model's training methodology and evaluation results. For further details, please refer to our technical report.

Overview

Hakim-unsup is the approximately 124-million-parameter intermediate model from the Hakim training pipeline, obtained after pretraining on Persian text and an unsupervised contrastive learning stage. Unlike Hakim and Hakim-small, it does not undergo the subsequent supervised instruction-tuning stage.

It was developed to learn general-purpose semantic representations of Persian text from unlabeled or automatically constructed text pairs. The model represents a research checkpoint in the training process, rather than a publicly downloadable model.

In the FaMTEB evaluation reported in May 2025, Hakim-unsup achieved an average score of 64.56, with an STS score of 79.36. These figures describe the reported evaluation, not current leaderboard positions.

📌 Model Highlights

• 🧱 Unsupervised semantic representations: Learns Persian text embeddings without the later labeled, task-specific fine-tuning phase.
• 🔄 Contrastive training: Uses approximately 5M pairs from the Pairsia-unsup dataset.
• ⚙️ Architecture size: Approximately 124M parameters, the same approximate size as the main Hakim model.
• 🌱 Intermediate research checkpoint: Represents the model before the supervised instruction-tuning stage used in the final Hakim variants.

🏗️ Training Datasets

Hakim-unsup is described as having two training stages:

📚 Pretraining

• Corpesia: 11B tokens from 46 Persian websites spanning 21 domains (including news, health, religion, and technology).
• hmBlogs: 6.8B tokens from approximately 20M Persian blog posts.
• Queries: 8.5M anonymized search queries.

🔄 Unsupervised Stage (Pairsia-unsup)

• Pairsia-unsup: Approximately 5M Persian text pairs, including:
  • Document–title, FAQ, question–answer, and paper title–abstract pairs.
  • Machine-translated datasets such as MS MARCO, SAMSum, and AdversarialQA.
• A contrastive learning objective was used to learn semantic representations from these pairs.

No subsequent supervised fine-tuning: Hakim-unsup was not trained on Pairsia-sup in the later supervised instruction-tuning phase.

For details on data construction and the training procedure, refer to the technical report.

🧪 Benchmark Results (FaMTEB, May 2025)

The following scores are from the evaluation reported in the Hakim technical report. They are not a continuously updated leaderboard.

Model	Avg. Score	Classification	Clustering	PairClass.	Reranking	Retrieval	STS	Summarization
Hakim	73.81	84.56	70.46	89.75	69.46	40.43	76.62	85.41
Hakim-small	70.45	80.19	66.31	87.41	67.30	38.05	75.53	78.40
Hakim-unsup	64.56	60.65	58.89	86.41	67.56	37.71	79.36	61.34
BGE-m3	65.29	58.75	57.73	85.21	74.56	43.38	76.35	61.07
Jina-embeddings-v3	64.53	59.93	59.15	83.71	61.26	43.51	78.65	65.50
multilingual-e5-large	64.40	59.86	57.19	84.42	74.34	42.98	75.38	56.61
GTE-multilingual-base	63.64	56.07	57.28	84.58	69.72	41.22	75.75	60.88
multilingual-e5-base	62.93	57.62	56.52	84.04	72.07	41.20	74.45	54.58
Tooka-SBERT	60.65	59.40	56.45	87.04	58.29	27.86	76.42	59.06

Related Models

• Hakim — the main, approximately 124M-parameter model after supervised fine-tuning.
• Hakim-small — the compact, approximately 38M-parameter supervised model.

Citation

If you refer to this model or the Hakim project in research, please cite the technical report:

@article{sarmadi2025hakim,
  title={Hakim: Farsi Text Embedding Model},
  author={Sarmadi, Mehran and Alikhani, Morteza and Zinvandi, Erfan and Pourbahman, Zahra},
  journal={arXiv preprint arXiv:2505.08435},
  year={2025}
}