Initial commit.

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+scatter_plot_performance_size.png filter=lfs diff=lfs merge=lfs -text

1_Dense/config.json

@@ -0,0 +1 @@
+{"in_features": 768, "out_features": 128, "bias": false, "activation_function": "torch.nn.modules.linear.Identity"}

1_Dense/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95058fd17ab5d528168c929071af61bfa5284e656894e2f7191ed4822b9eceb8
+size 393304

README.md

@@ -0,0 +1,399 @@
+---
+tags:
+- ColBERT
+- PyLate
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- multilingual
+- late-interaction
+- retrieval
+- bright
+- loss:Distillation
+pipeline_tag: sentence-similarity
+library_name: PyLate
+license: apache-2.0
+base_model:
+- DavidGF/SauerkrautLM-Multi-ModernColBERT
+---
+<img src="https://vago-solutions.ai/wp-content/uploads/2025/08/SauerkrautLM-Multi-Reason-ModernColBERT.png" width="500" height="auto">
+
+# SauerkrautLM-Multi-Reason-ModernColBERT
+
+This model is the first publicly available Late Interaction retriever that integrates:
+
+Knowledge Distillation from strong synthetic data (200k samples generated with Qwen/Qwen3-32B-AWQ and scored by a high-performing reranker).
+LaserRMT compression, making it the first known ColBERT-style retriever to benefit from low-rank approximation.
+
+### 🎯 Core Features and Innovations:
+
+- **Next-Generation Knowledge Distillation**: By utilizing 200,000 synthetically generated, high-quality training examples (created with `Qwen/Qwen3-32B-AWQ` and scored by a state-of-the-art reranker), our model learns complex reasoning patterns from models **54× its size**.
+
+- **Groundbreaking LaserRMT Compression**: As the first known **ColBERT-style retriever to benefit from low-rank approximation**
+
+### 💪 David vs. Goliath: Small but Mighty
+
+With only **149 million parameters** – that's **less than 1/45th the size** of some competing models – SauerkrautLM achieves or exceeds the performance of:
+- Models with **over 7 billion parameters** (47× larger than ours)
+- Proprietary API-based solutions from major tech companies
+- Specialized reasoning models like ReasonIR-8B (54× larger)
+
+This exceptional efficiency makes it the ideal choice for production environments where resource consumption and latency are critical factors.
+
+
+
+## Model Overview
+
+**Model:** `VAGOsolutions/SauerkrautLM-Multi-Reason-ModernColBERT`\
+**Base:** Fine-tuned from [VAGOsolutions/SauerkrautLM-Multi-ModernColBERT](https://huggingface.co/VAGOsolutions/SauerkrautLM-Multi-ModernColBERT) using knowledge distillation and LaserRMT\
+**Architecture:** PyLate / ColBERT (Late Interaction)\
+**Languages:** Multilingual (optimized for 7 European languages: German, English, Spanish, French, Italian, Dutch, Portuguese)\
+**License:** Apache 2.0\
+**Model Size:** 149M parameters 
+**Efficiency Ratio:** Up to **54× smaller** than comparable performing models
+
+### Model Description
+- **Model Type:** PyLate model with innovative Late Interaction architecture
+- **Document Length:** 8192 tokens (32× longer than traditional BERT models)
+- **Query Length:** 256 tokens (optimized for complex, multi-part queries)
+- **Output Dimensionality:** 128 tokens (efficient vector representation)
+- **Similarity Function:** MaxSim (enables precise token-level matching)
+- **Training Loss:** Knowledge Distillation (PyLate)
+
+### Architecture
+
+```
+ColBERT(
+  (0): Transformer(ModernBertModel)
+  (1): Dense(768 -> 128 dim, no bias)
+)
+```
+
+## 🔬 Technical Innovations in Detail
+
+### Knowledge Distillation: The Student Surpassing the Master
+
+
+1. **Synthetic Data Generation**: 200,000 high-quality query-document pairs generated using the `Qwen/Qwen3-32B-AWQ` model (32 billion parameters) based on the [ReasonIR approach](https://huggingface.co/datasets/reasonir/reasonir-data)
+2. **Quality Assurance**: Each pair evaluated and filtered by a state-of-the-art reranker
+3. **Distillation Process**: The compact ModernColBERT model learns to replicate the ranking patterns of large models
+
+
+### LaserRMT: Revolution in Model Compression
+
+As the **first ColBERT-based retrieval model with Low-Rank approximation**, SauerkrautLM sets new standards:
+
+
+This technology combines the advantages of Late Interaction Retrieval (precise token-level matching) with the efficiency of compact models.
+
+---
+
+## 🔬 Benchmarks: David vs. Goliath Performance
+
+Our comprehensive evaluation demonstrates that model size is not destiny. Despite being **47-54× smaller** than competing models, SauerkrautLM consistently delivers superior or comparable performance across challenging reasoning and multilingual retrieval tasks.
+
+### BRIGHT Benchmark (English, reasoning‑focused retrieval)
+
+The [BRIGHT benchmark](https://huggingface.co/datasets/xlangai/BRIGHT) is designed to evaluate **reasoning‑intensive retrieval**. All scores are nDCG\@10. SauerkrautLM (≈149 M parameters) is compared with dense and proprietary baselines as well as the original and re‑evaluated Reason‑ModernColBERT model.
+
+| Model / Metric                           | Biology   | Earth     | Economics | Psychology | Robotics | Stackoverflow | Sustainable | Leetcode  | Pony      | AoPS      | Theorem‑Q | Theorem‑T | Mean StackEx | Mean coding | Mean theorem | Full Mean |
+| ---------------------------------------- | --------- | --------- | --------- | ---------- | -------- | ------------- | ----------- | --------- | --------- | --------- | --------- | --------- | ------------ | ----------- | ------------ | --------- |
+| **BM25**                                 | 18.90     | 27.20     | 14.90     | 12.50      | 13.60    | 18.40         | 15.00       | 24.40     | 7.90      | 6.20      | 10.40     | 4.90      | 17.21        | 16.15       | 7.17         | 14.53     |
+| **< 1 B OS**                             |           |           |           |            |          |               |             |           |           |           |           |           |              |             |              |           |
+| BGE                                      | 11.70     | 24.60     | 16.60     | 17.50      | 11.70    | 10.80         | 13.30       | 26.70     | 5.70      | 6.00      | 13.00     | 6.90      | 15.17        | 16.20       | 8.63         | 13.71     |
+| Inst‑L                                   | 15.20     | 21.20     | 14.70     | 22.30      | 11.40    | 13.30         | 13.50       | 19.50     | 1.30      | 8.10      | 20.90     | 9.10      | 15.94        | 10.40       | 12.70        | 14.21     |
+| SBERT                                    | 15.10     | 20.40     | 16.60     | 22.70      | 8.20     | 11.00         | 15.30       | 26.40     | 7.00      | 5.30      | 20.00     | 10.80     | 15.61        | 16.70       | 12.03        | 14.90     |
+| **> 1 B OS**                             |           |           |           |            |          |               |             |           |           |           |           |           |              |             |              |           |
+| E5                                       | 18.60     | 26.00     | 15.50     | 15.80      | 16.30    | 11.20         | 18.10       | 28.70     | 4.90      | 7.10      | 26.10     | 26.80     | 17.36        | 16.80       | 20.00        | 17.93     |
+| SFR                                      | 19.10     | 26.70     | 17.80     | 19.00      | 16.30    | 14.40         | 19.20       | 27.40     | 2.00      | 7.40      | 24.30     | 26.00     | 18.93        | 14.70       | 19.23        | 18.30     |
+| Inst‑XL                                  | 21.60     | 34.30     | 22.40     | 27.40      | 18.20    | 21.20         | 19.10       | 27.50     | 5.00      | 8.50      | 15.60     | 5.90      | 23.46        | 16.25       | 10.00        | 18.89     |
+| GritLM                                   | 24.80     | 32.30     | 18.90     | 19.80      | 17.10    | 13.60         | 17.80       | 29.90     | 22.00     | 8.80      | 25.20     | 21.20     | 20.61        | 25.95       | 18.40        | 20.95     |
+| Qwen                                     | 30.60     | 36.40     | 17.80     | 24.60      | 13.20    | 22.20         | 14.80       | 25.50     | 9.90      | 14.40     | 27.80     | 32.90     | 22.80        | 17.70       | 25.03        | **22.51**     |
+| **Proprietary**                          |           |           |           |            |          |               |             |           |           |           |           |           |              |             |              |           |
+| Cohere                                   | 18.70     | 28.40     | 20.40     | 21.60      | 16.30    | 18.30         | 17.60       | 26.80     | 1.90      | 6.30      | 15.70     | 7.20      | 20.19        | 14.35       | 9.73         | 16.60     |
+| OpenAI                                   | 23.30     | 26.70     | 19.50     | 27.60      | 12.80    | 14.30         | 20.50       | 23.60     | 2.40      | 8.50      | 23.50     | 11.70     | 20.67        | 13.00       | 14.57        | 17.87     |
+| Voyage                                   | 23.10     | 25.40     | 19.90     | 24.90      | 10.80    | 16.80         | 15.40       | 30.60     | 1.50      | 7.50      | 27.40     | 11.60     | 19.47        | 16.05       | 15.50        | 17.91     |
+| Google                                   | 22.70     | 34.80     | 19.60     | 27.80      | 15.70    | 20.10         | 17.10       | 29.60     | 3.60      | 9.30      | 23.80     | 15.90     | 22.54        | 16.60       | 16.33        | 20.00     |
+| **ReasonIR data**                        |           |           |           |            |          |               |             |           |           |           |           |           |              |             |              |           |
+| ReasonIR‑8B                              | 26.20     | 31.40     | 23.30     | 30.00      | 18.00    | 23.90         | 20.50       | 35.00     | 10.50     | 14.70     | 31.90     | 27.20     | 24.76        | 22.75       | 24.60        | **24.38**     |
+| Reason‑ModernColBERT (149 M) reported    | 33.25     | 41.02     | 24.93     | 30.73      | 21.12    | 20.62         | 20.31       | 31.07     | 8.51      | 9.17      | 19.51     | 11.24     | 27.43        | 19.79       | 15.38        | **22.62**     |
+| Reason‑ModernColBERT (149 M) our eval\*\*     | 34.28     | 41.53     | 19.96     | 27.02      | 21.15    | 23.62         | 17.21       | 26.61     | 1.32      | 7.30      | 19.79     | 9.70      | 27.93        | 13.97       | 12.26        | 20.79     |
+| **SauerkrautLM Reasoning data**          |           |           |           |            |          |               |             |           |           |           |           |           |              |             |              |           |
+| **SauerkrautLM-Multi-Reason-ModernColBERT (149 M)**            | **36.92** | **45.53** | 19.47     | **27.04**  | 19.35    | **25.31**     | **20.78**   | **29.74** | **12.54** | **10.52** | 14.62     | 7.65      | **28.94**    | **21.14**   | 10.93        | **22.45** |
+| SauerkrautLM‑Reason‑EuroColBERT (210 M)  | 38.16     | 39.43     | 16.99     | 24.49      | 17.50    | 17.60         | 20.72       | 29.10     | 13.57     | 12.04     | 10.43     | 4.95      | 25.70        | 21.33       | 9.14         | 20.42     |
+| SauerkrautLM‑Reason‑Multi‑ColBERT (15 M) | 23.33     | 23.78     | 10.53     | 9.03       | 10.28    | 10.88         | 13.13       | 18.10     | 15.86     | 1.75      | 4.29      | 0.81      | 14.64        | 16.98       | 2.28         | 11.81     |
+
+**Evaluation note:** our re‑evaluation of Reason‑ModernColBERT uses the **same query‑length settings** from the original Lighton repo; the instructions for the originally reported scores are not public.
+
+
+#### ⚖️ Relative Efficiency
+
+With **149 M parameters**, SauerkrautLM surpasses several ≥7 B dense and proprietary retrievers on reasoning‑centric tasks.
+
+### BRIGHT Benchmark (German, reasoning‑focused retrieval)
+
+All scores are nDCG\@10.
+
+| Model / Metric                                      | Biology   | Earth     | Economics | Psychology | Robotics  | Stackoverflow | Sustainable | Leetcode  | Pony      | AoPS     | Theorem‑Q | Theorem‑T | Mean StackEx | Mean coding | Mean theorem | Full Mean |
+| --------------------------------------------------- | --------- | --------- | --------- | ---------- | --------- | ------------- | ----------- | --------- | --------- | -------- | --------- | --------- | ------------ | ----------- | ------------ | --------- |
+| **SauerkrautLM‑Multi‑Reason‑ModernColBERT (149 M)** | 28.00 | **34.71** | **12.90** | 17.98  | **13.67** | **19.64**     | 17.70   | 11.66     | **15.49** | 7.27     | 6.76      | 1.32      | **21.15**    | 13.57       | 5.11         | **15.59** |
+| SauerkrautLM‑Reason‑EuroColBERT (210 M)             | **31.09**     | 31.48     | 11.95     | **18.39**      | 11.25     | 14.43         | **20.26**       | **25.67** | 12.15     | **9.58** | **8.15**  | **2.76**  | 19.76        | **18.91**   | **6.83**     | **16.43** |
+| SauerkrautLM‑Reason‑Multi‑ColBERT (15 M)            | 15.37     | 20.11     | 7.36      | 7.07       | 4.24      | 4.71          | 7.67        | 0.77      | 6.31      | 3.81     | 0.76      | 0.00      | 9.81         | 3.54        | 1.52         | 6.51      |
+
+> **Observation:** Our 149 M flagship dominates most German domains (Biology, Earth, Sustainable, Mean StackExchange) while the 210 M EuroColBERT secures the **highest Full‑Mean (16.43)**, especially on coding and theorem sub‑tasks.
+
+---
+
+### NanoBEIR Europe (multilingual retrieval)
+
+Average nDCG\@10 across the seven languages we evaluated:
+
+| Language | nDCG\@10 |
+| -------- | -------- |
+| de       | 50.74    |
+| en       | 67.32    |
+| es       | 53.82    |
+| fr       | 53.94    |
+| it       | 53.19    |
+| nl       | 51.49    |
+| pt       | 53.07    |
+
+
+---
+
+### Why SauerkrautLM Matters for Production
+
+- **Outperforms proprietary APIs**: beats Cohere, OpenAI, Voyage and Google on BRIGHT Full Mean while remaining fully open‑source under a permissive **Apache 2.0** license.
+- **Highest *****Mean StackExchange***** score** of all evaluated models (28.94) — crucial for reasoning‑heavy Q&A communities.
+- **Full parameter range**: from the tiny **15 M** Multi‑ColBERT (competitive with SBERT‑scale encoders) to the robust 210 M EuroColBERT variant.
+- **Matches or exceeds** models 10–50× larger (e.g. ReasonIR‑8B, GritLM, Qwen).
+- **Strong multilingual coverage** across seven European languages without language‑specific fine‑tuning.
+
+We translated both **BRIGHT** and **NanoBEIR** into seven European languages to rigorously evaluate multilingual retrieval capabilities.
+
+Below is a **scatter plot** that visualises model size (millions of parameters) against BRIGHT Full‑Mean nDCG\@10. SauerkrautLM models occupy the best trade‑off region—smallest models with top‑tier reasoning performance.
+<img src="https://vago-solutions.ai/wp-content/uploads/2025/08/Image-graph-2.jpeg">
+
+
+### Real-World Impact
+
+The efficiency gains translate to tangible benefits:
+
+1. **Democratized AI**: Run state-of-the-art retrieval on consumer hardware
+2. **Edge Deployment**: Enable on-device search for privacy-sensitive applications  
+3. **Massive Scale**: Index billions of documents at a fraction of traditional costs
+
+## 📈 Summary: The New Efficiency Paradigm
+
+SauerkrautLM-Multi-Reason-ModernColBERT represents a paradigm shift in retrieval model design. By combining cutting-edge knowledge distillation with innovative LaserRMT compression, we've created a model that:
+
+- **Delivers 99.7% of the performance** of 7B parameter models while being **47× smaller**
+- **Outperforms all major proprietary APIs** (OpenAI, Cohere, Google, Voyage) on reasoning tasks
+- **Runs on consumer hardware** (4GB GPU) instead of requiring enterprise infrastructure (80GB+)
+- **Reduces deployment costs by 50×**
+- **Achieves the highest StackExchange score** (28.94) of any evaluated model
+
+This breakthrough demonstrates that with the right techniques, compact models can match or exceed the capabilities of models orders of magnitude larger, democratizing access to state-of-the-art retrieval technology.
+
+---
+
+# PyLate
+
+This is a [PyLate](https://github.com/lightonai/pylate) model trained. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.
+
+
+## Usage
+First install the PyLate library:
+
+```bash
+pip install -U pylate
+```
+
+### Retrieval
+
+PyLate provides a streamlined interface to index and retrieve documents using ColBERT models. The index leverages the Voyager HNSW index to efficiently handle document embeddings and enable fast retrieval.
+
+#### Indexing documents
+
+First, load the ColBERT model and initialize the Voyager index, then encode and index your documents:
+
+```python
+from pylate import indexes, models, retrieve
+
+# Step 1: Load the ColBERT model
+model = models.ColBERT(
+    model_name_or_path=pylate_model_id,
+)
+
+# Step 2: Initialize the Voyager index
+index = indexes.Voyager(
+    index_folder="pylate-index",
+    index_name="index",
+    override=True,  # This overwrites the existing index if any
+)
+
+# Step 3: Encode the documents
+documents_ids = ["1", "2", "3"]
+documents = ["document 1 text", "document 2 text", "document 3 text"]
+
+documents_embeddings = model.encode(
+    documents,
+    batch_size=32,
+    is_query=False,  # Ensure that it is set to False to indicate that these are documents, not queries
+    show_progress_bar=True,
+)
+
+# Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
+index.add_documents(
+    documents_ids=documents_ids,
+    documents_embeddings=documents_embeddings,
+)
+```
+
+Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:
+
+```python
+# To load an index, simply instantiate it with the correct folder/name and without overriding it
+index = indexes.Voyager(
+    index_folder="pylate-index",
+    index_name="index",
+)
+```
+
+#### Retrieving top-k documents for queries
+
+Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries.
+To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:
+
+```python
+# Step 1: Initialize the ColBERT retriever
+retriever = retrieve.ColBERT(index=index)
+
+# Step 2: Encode the queries
+queries_embeddings = model.encode(
+    ["query for document 3", "query for document 1"],
+    batch_size=32,
+    is_query=True,  #  # Ensure that it is set to False to indicate that these are queries
+    show_progress_bar=True,
+)
+
+# Step 3: Retrieve top-k documents
+scores = retriever.retrieve(
+    queries_embeddings=queries_embeddings,
+    k=10,  # Retrieve the top 10 matches for each query
+)
+```
+
+### Reranking
+If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank:
+
+```python
+from pylate import rank, models
+
+queries = [
+    "query A",
+    "query B",
+]
+
+documents = [
+    ["document A", "document B"],
+    ["document 1", "document C", "document B"],
+]
+
+documents_ids = [
+    [1, 2],
+    [1, 3, 2],
+]
+
+model = models.ColBERT(
+    model_name_or_path=pylate_model_id,
+)
+
+queries_embeddings = model.encode(
+    queries,
+    is_query=True,
+)
+
+documents_embeddings = model.encode(
+    documents,
+    is_query=False,
+)
+
+reranked_documents = rank.rerank(
+    documents_ids=documents_ids,
+    queries_embeddings=queries_embeddings,
+    documents_embeddings=documents_embeddings,
+)
+```
+## Citation
+
+### BibTeX
+
+#### SauerkrautLM‑Multi‑Reason‑ModernColBERT
+
+```bibtex
+@misc{SauerkrautLM-Multi-Reason-ModernColBERT,
+  title={SauerkrautLM-Multi-Reason-ModernColBERT},
+  author={David Golchinfar},
+  url={https://huggingface.co/VAGOsolutions/SauerkrautLM-Multi-Reason-ModernColBERT},
+  year={2025}
+}
+```
+
+#### GTE‑ModernColBERT
+
+```bibtex
+@misc{GTE-ModernColBERT,
+  title={GTE-ModernColBERT},
+  author={Chaffin, Antoine},
+  url={https://huggingface.co/lightonai/GTE-ModernColBERT-v1},
+  year={2025}
+}
+```
+
+#### Sentence Transformers
+
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+  title = {Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks},
+  author = {Reimers, Nils and Gurevych, Iryna},
+  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing},
+  month = {11},
+  year = {2019},
+  publisher = {Association for Computational Linguistics},
+  url = {https://arxiv.org/abs/1908.10084}
+}
+```
+
+#### PyLate
+
+```bibtex
+@misc{PyLate,
+  title={PyLate: Flexible Training and Retrieval for Late Interaction Models},
+  author={Chaffin, Antoine and Sourty, Raphaël},
+  url={https://github.com/lightonai/pylate},
+  year={2024}
+}
+```
+
+
+## Acknowledgements
+We thank Antoine Chaffin (LightOn AI) for helpful discussions and for clarifying evaluation settings for Reason‑ModernColBERT, and the PyLate team for providing the training framework that made this work possible.
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,45 @@
+{
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50281,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 50281,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "embedding_dropout": 0.0,
+  "eos_token_id": 50282,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 10000.0,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 50283,
+  "position_embedding_type": "absolute",
+  "repad_logits_with_grad": false,
+  "sep_token_id": 50282,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.51.0",
+  "vocab_size": 50370
+}

config_sentence_transformers.json

@@ -0,0 +1,49 @@
+{
+  "__version__": {
+    "sentence_transformers": "4.0.2",
+    "transformers": "4.51.0",
+    "pytorch": "2.7.0+cu126"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "MaxSim",
+  "query_prefix": "[Q] ",
+  "document_prefix": "[D] ",
+  "query_length": 256,
+  "document_length": 2048,
+  "attend_to_expansion_tokens": false,
+  "skiplist_words": [
+    "!",
+    "\"",
+    "#",
+    "$",
+    "%",
+    "&",
+    "'",
+    "(",
+    ")",
+    "*",
+    "+",
+    ",",
+    "-",
+    ".",
+    "/",
+    ":",
+    ";",
+    "<",
+    "=",
+    ">",
+    "?",
+    "@",
+    "[",
+    "\\",
+    "]",
+    "^",
+    "_",
+    "`",
+    "{",
+    "|",
+    "}",
+    "~"
+  ]
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:865debed2b4fc95f09368363aa9aafecc8153fc03485d0091e5435477bd8144a
+size 596076280

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Dense",
+    "type": "pylate.models.Dense.Dense"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 255,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "[MASK]",
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,968 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50280": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50281": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50282": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50283": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50284": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50285": {
+      "content": "[unused0]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50286": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50287": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50288": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50289": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50290": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50291": {
+      "content": "[unused6]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50292": {
+      "content": "[unused7]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50293": {
+      "content": "[unused8]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50294": {
+      "content": "[unused9]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50295": {
+      "content": "[unused10]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50296": {
+      "content": "[unused11]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50297": {
+      "content": "[unused12]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50298": {
+      "content": "[unused13]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50299": {
+      "content": "[unused14]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50300": {
+      "content": "[unused15]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50301": {
+      "content": "[unused16]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50302": {
+      "content": "[unused17]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50303": {
+      "content": "[unused18]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50304": {
+      "content": "[unused19]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50305": {
+      "content": "[unused20]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50306": {
+      "content": "[unused21]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50307": {
+      "content": "[unused22]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50308": {
+      "content": "[unused23]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50309": {
+      "content": "[unused24]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50310": {
+      "content": "[unused25]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50311": {
+      "content": "[unused26]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50312": {
+      "content": "[unused27]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50313": {
+      "content": "[unused28]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50314": {
+      "content": "[unused29]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50315": {
+      "content": "[unused30]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50316": {
+      "content": "[unused31]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50317": {
+      "content": "[unused32]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50318": {
+      "content": "[unused33]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50319": {
+      "content": "[unused34]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50320": {
+      "content": "[unused35]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50321": {
+      "content": "[unused36]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50322": {
+      "content": "[unused37]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50323": {
+      "content": "[unused38]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50324": {
+      "content": "[unused39]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50325": {
+      "content": "[unused40]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50326": {
+      "content": "[unused41]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50327": {
+      "content": "[unused42]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50328": {
+      "content": "[unused43]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50329": {
+      "content": "[unused44]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50330": {
+      "content": "[unused45]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50331": {
+      "content": "[unused46]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50332": {
+      "content": "[unused47]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50333": {
+      "content": "[unused48]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50334": {
+      "content": "[unused49]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50335": {
+      "content": "[unused50]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50336": {
+      "content": "[unused51]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50337": {
+      "content": "[unused52]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50338": {
+      "content": "[unused53]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50339": {
+      "content": "[unused54]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50340": {
+      "content": "[unused55]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50341": {
+      "content": "[unused56]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50342": {
+      "content": "[unused57]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50343": {
+      "content": "[unused58]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50344": {
+      "content": "[unused59]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50345": {
+      "content": "[unused60]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50346": {
+      "content": "[unused61]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50347": {
+      "content": "[unused62]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50348": {
+      "content": "[unused63]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50349": {
+      "content": "[unused64]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50350": {
+      "content": "[unused65]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50351": {
+      "content": "[unused66]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50352": {
+      "content": "[unused67]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50353": {
+      "content": "[unused68]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50354": {
+      "content": "[unused69]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50355": {
+      "content": "[unused70]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50356": {
+      "content": "[unused71]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50357": {
+      "content": "[unused72]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50358": {
+      "content": "[unused73]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50359": {
+      "content": "[unused74]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50360": {
+      "content": "[unused75]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50361": {
+      "content": "[unused76]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50362": {
+      "content": "[unused77]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50363": {
+      "content": "[unused78]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50364": {
+      "content": "[unused79]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50365": {
+      "content": "[unused80]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50366": {
+      "content": "[unused81]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50367": {
+      "content": "[unused82]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50368": {
+      "content": "[Q] ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50369": {
+      "content": "[D] ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_length": 299,
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 255,
+  "pad_to_multiple_of": null,
+  "pad_token": "[MASK]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "tokenizer_class": "PreTrainedTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}