Upload folder using huggingface_hub

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": true,
+  "pooling_mode_mean_tokens": false,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,64 @@
+
+---
+library_name: sentence-transformers
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- autotrain
+base_model: Alibaba-NLP/gte-modernbert-base
+widget:
+- source_sentence: 'search_query: i love autotrain'
+  sentences:
+  - 'search_query: huggingface auto train'
+  - 'search_query: hugging face auto train'
+  - 'search_query: i love autotrain'
+pipeline_tag: sentence-similarity
+---
+
+# Model Trained Using AutoTrain
+
+- Problem type: Sentence Transformers
+
+## Validation Metrics
+loss: 0.4268312156200409
+
+cosine_accuracy: 0.9693415637860082
+
+runtime: 40.9588
+
+samples_per_second: 118.656
+
+steps_per_second: 7.422
+
+: 1.0
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the Hugging Face Hub
+model = SentenceTransformer("sentence_transformers_model_id")
+# Run inference
+sentences = [
+    'search_query: autotrain',
+    'search_query: auto train',
+    'search_query: i love autotrain',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+```

checkpoint-11543/1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 768,
+  "pooling_mode_cls_token": true,
+  "pooling_mode_mean_tokens": false,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

checkpoint-11543/README.md

@@ -0,0 +1,988 @@
+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:92337
+- loss:MultipleNegativesRankingLoss
+base_model: Alibaba-NLP/gte-modernbert-base
+widget:
+- source_sentence: 'Philosophy Professor Running for US Senate! Richard Dien Winfield
+    is a philosophy professor at UGA and is a distinguished Hegel scholar. I''m sure
+    many of you are familiar with his many published books on Hegel and his audio
+    lectures, available here: is currently running for US Senate for the state of
+    Georgia and is just beginning his campaign. Check out his campaign launch video:
+    his website: can follow him on Twitter: is advocating for a new social bill of
+    rights. Here is where he is on just some of the issues:  Federal Job Guarantee  Fair
+    Minimum Wage ($20/hr)  Green New Deal  Super Medicare for All  Legal Care for
+    All It''s great to see a philosophy professor running for office!'
+  sentences:
+  - Context and Context-Dependence
+  - 'There are some contests about this, but: “Atheism” is typically defined in terms
+    of “theism”. Theism, in turn, is best understood as a proposition—something that
+    is either true or false. It is often defined as “the belief that God exists”,
+    but here “belief” means “something believed”. It refers to the propositional content
+    of belief, not to the attitude or psychological state of believing. This is why
+    it makes sense to say that theism is true or false and to argue for or against
+    theism. If, however, “atheism” is defined in terms of theism and theism is the
+    proposition that God exists and not the psychological condition of believing that
+    there is a God, then it follows that atheism is not the absence of the psychological
+    condition of believing that God exists (more on this below). The “a-” in “atheism”
+    must be understood as negation instead of absence, as “not” instead of “without”.
+    Therefore, in philosophy at least, atheism should be construed as the proposition
+    that God does not exist (or, more broadly, the proposition that there are no gods).
+    rf: sometimes you hear the other thing - that atheism is the position wherein
+    some believer "lacks the belief" that there are gods. This position is popular
+    outside of philosophy. There are lots of good reasons to reject this alternative
+    situation, as it tends to confuse other positions. You can read why here: https://old.reddit.com/r/askphilosophy/comments/2za4ez/vacuous_truths_and_shoe_atheism/cuyn8nm/'
+  - 'If we are being honest, he has a rather slim chance to win the primary (the establishmenz
+    seems to have anointed their candidate, and it''s not a bad one), and an uphill
+    general election (last Senate election was R+12). And he doesn''t seem to say
+    the right things in public.... Yeah. Edit: In my view, philosophers (and anyone)
+    should very much think about whether running for office, in this situation, is
+    the right call. Given the issues Winfield wants to push, it seems like funding
+    an NGO or a think tank would be much better suited to forward the cause. Especially
+    since the job guarantee is already authorized (but not mandated) by law, which
+    is a fun quirk. But then again, just looking at the website, you can tell that
+    this campaign isn''t run by professionals? It just sounds like a waste of time
+    and resources. If you ever Plan to run for office, hire me to give you advice
+    (I''ll do it for free if you''re not a Hegelian)'
+- source_sentence: 'Human embryos are fully human in a biological sense, they''re
+    just at a particular stage of human development (a developing human, as opposed
+    to non-humans becoming human), but maybe that''s what you meant. You''re effectively
+    asking whether there are any pro-life philosophers, and there are. Here are some:
+    Francis J. Beckwith, Stephen D. Schwarz, Christopher Kaczor, Don Marquis, Patrick
+    Lee, and Jack Mulder. Most tend to be Catholics, but none assume or employ religious
+    doctrines in making the case against abortion. I myself happen to be non-religious
+    but pro-life, for example. If you have access to a university library, you can
+    find plenty more authors who''ve written articles effectively defending the view
+    you mentioned. Most of the authors I listed above have written books and articles
+    that are more accessible, though.'
+  sentences:
+  - 'Just to add a few more papers to this: Manninen, B.A. (2007) “Revisiting the
+    Argument from Foetal Potential”. Philosophy, Ethics and Humanities in Medicine.
+    Volume 2, Number 1. Available at: P. George, R.P. (2005) “The Wrong of Abortion”.
+    Contemporary Debates in Applied Ethics, pp.13-26. Cohen, A.I.; Wellman, C.H. (eds.)
+    Malden, MA: Blackwell Pub.'
+  - I myself happen to be non-religious but pro-life I am too lol, we're quite a rare
+    breed. Are any of those authors openly non religious? I'll check some of them
+    out anyway.
+  - That last bit sounds right up the alley of what I'm looking at. Do you happen
+    to know of any papers that elaborate any of these contentions?
+- source_sentence: Is there something morally wrong with cultural appropriation in
+    the arts? I argue that the little philosophical work on this topic has been overly
+    dismissive of moral objections to cultural appropriation. Nevertheless, I argue
+    that philosophers working on epistemic injustice have developed powerful conceptual
+    tools that can aid in our understanding of objections that have been levied by
+    other scholars and artists. I then consider the relationship between these objections
+    and the harms of cultural essentialism. I argue that focusing on (...) the systematic
+    nature of appropriative harms may allow us to sidestep the problem of essentialism,
+    but not without cost. (shrink)
+  sentences:
+  - Then, I(g, ( A Af -, y) -, (c -,+ (f -, y))) = o ensures that there is a w where
+    I(w, c A AP -+ y) = 1 and I(w, a --+ (f -+ y)) = 0. This gives us x, y where Rwxy,
+    I(x, a) = 1 and I(y, P -+ y) = 0, which in turn means that there are z, t where
+    Ryzt, I(z, P) = 1 and I(t, y) = 0.
+  - Applied Ethics, Misc
+  - Defenses of Toleration
+- source_sentence: Whereas it may be said that in Britain freedom is regarded, in
+    a sense, as a privilege bestowed from above, from the upper orders of society,
+    in the United States it is regarded as a 'natural right'; an argument not really
+    vitiated by pointing out that the natural rights doctrine is nonsense, for Americans
+    throughout a great part of their history have believed that they do enjoy these
+    rights.
+  sentences:
+  - '. Art is something unique, the spirit is unique in its source. Art is symbolic,
+    since it always bears within itself a symbol, i.e., that which is eternal, and
+    rejects that which is transitory. Art is free, since it arises from inspiration.
+    "16 Bryusov and Berdyaev protested that they could only recognize socialist ideas
+    to the extent that socialism respects the basic principle of their world view:
+    the unconditional independence of the artist. The opinions of the Russian symbolists
+    were a reflection of French Symbolism and Parnassism. At its base is the Romantic
+    theory of art, derived from Novalis and F. Schlegel, according to which the artist
+    is a Brahmin. (At the same time, a trend in Romanticism stressed the social obligations
+    of the artist, e.g., Shelley in Defence of Poetry.) This apolitical tendency in
+    Romanticism was inherited by the representatives of the idea of "art for art''s
+    sake." Baudelaire wrote, in the Hymne: "Que tu viennes du Ciel ou de l''Enfer,
+    qu''importe, o Beauté." Flaubert thought: "Aimonsnous en l''art comme les mystiques
+    s''aiment en Dieu" (CorresponRevolution on the Development of Russian Esthetic
+    Thinking," in Uchonie zapiski L.G.U., S istoricheskikh nauk, 1956, No. 220. Vol.
+    II, 1893, p. 286) . The Goncourts wrote in their Journal in 1886 that only "pure
+    literature" is a matter of life and death. And we are continually coming across
+    statements in their writings concerning the eternality of the "truly beautiful,"
+    the independence of the artist, his superiority and disdain towards his clients
+    and customers. Baudelaire wrote about Poe, and Gautier about Baudelaire, that
+    the glory of the poet is his holding himself aloof from Utopians, philanthropists,
+    socialists, etc.'
+  - Maybe the guy in the room can not be said to understand Chinese but the room,
+    itself, could. That's not to say that the room is conscious, though.
+  - His next major contribution to the literature of educational reform appeared in
+    1845, in a series of articles in The Scotsman, entitled 'National Education and
+    the Common Schools of Massachusetts'. In I847 these came out in pamphlet form
+    as Remarks on National Education4 This was largely a re-statement of the argument
+    advanced in his Edinburgh Review article, prefaced by an assurance to his readers
+    that they had nothing to fear from the Government's being entrusted to run the
+    nation's schools. 'In every free country the state is merely the representative
+    of the general power (physical, moral and intellectual) of the country. It is
+    not a distant and independent being, that can exist and A Note On 'Secular5 Education
+    In The Nineteenth Century In Spite The Will Of Its Members. This was addressed
+    to those who saw in Governmental interference in education the spectre of despotism
+    rearing its head, a not unfounded fear since it was upon the Prussian example
+    that Mann had erected his Massachusetts plan. Society's claim on the individual
+    was not, however, a total one, for 'the individual has a right to unbounded liberty
+    of self-determination as to what he shall learn and what he shall not learn. '2
+    Combe made no attempt to reconcile these two propositions.
+- source_sentence: I found Kant and the Problem of Metaphysics to be a good intro
+    to his style (assuming you know Kant well)
+  sentences:
+  - I'm going on a hunch here and guessing that your professor was referring to Rorty's
+    pragmatism, in which case this would seek to eliminate, or at least redesign,
+    much of metaphysics and epistemology (primarily) but not philosophy in its entirety.
+  - I would second this recommendation; Polt is a wonderful reader of Heidegger. I
+    think he’s right to say that his Introduction is best read alongside B T.
+  - Good suggestion, I'm currently working on his rather complicated Auseinandersetzung
+    with Kant. It's baffling how insightful and at the same time how rash he can be!
+    ;)
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- cosine_accuracy
+model-index:
+- name: SentenceTransformer based on Alibaba-NLP/gte-modernbert-base
+  results:
+  - task:
+      type: triplet
+      name: Triplet
+    dataset:
+      name: Unknown
+      type: unknown
+    metrics:
+    - type: cosine_accuracy
+      value: 0.9693415637860082
+      name: Cosine Accuracy
+---
+
+# SentenceTransformer based on Alibaba-NLP/gte-modernbert-base
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Alibaba-NLP/gte-modernbert-base](https://huggingface.co/Alibaba-NLP/gte-modernbert-base). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [Alibaba-NLP/gte-modernbert-base](https://huggingface.co/Alibaba-NLP/gte-modernbert-base) <!-- at revision bc02f0a92d1b6dd82108036f6cb4b7b423fb7434 -->
+- **Maximum Sequence Length:** 8192 tokens
+- **Output Dimensionality:** 768 dimensions
+- **Similarity Function:** Cosine Similarity
+<!-- - **Training Dataset:** Unknown -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: ModernBertModel 
+  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("sentence_transformers_model_id")
+# Run inference
+sentences = [
+    'I found Kant and the Problem of Metaphysics to be a good intro to his style (assuming you know Kant well)',
+    "Good suggestion, I'm currently working on his rather complicated Auseinandersetzung with Kant. It's baffling how insightful and at the same time how rash he can be! ;)",
+    'I would second this recommendation; Polt is a wonderful reader of Heidegger. I think he’s right to say that his Introduction is best read alongside B T.',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 768]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Triplet
+
+* Evaluated with [<code>TripletEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.TripletEvaluator)
+
+| Metric              | Value      |
+|:--------------------|:-----------|
+| **cosine_accuracy** | **0.9693** |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### Unnamed Dataset
+
+
+* Size: 92,337 training samples
+* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | anchor                                                                                | positive                                                                             | negative                                                                             |
+  |:--------|:--------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
+  | type    | string                                                                                | string                                                                               | string                                                                               |
+  | details | <ul><li>min: 11 tokens</li><li>mean: 229.59 tokens</li><li>max: 1278 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 108.47 tokens</li><li>max: 1127 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 124.73 tokens</li><li>max: 1127 tokens</li></ul> |
+* Samples:
+  | anchor                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | negative                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+  |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>This article presents Derrida as a philosopher of history by reinterpreting his De la Grammatologie. In particular, it provides a schematic reconstruction of Part II of that book from the perspective of the problem of history. My account extends work on historicity in Derrida by privileging the themes of ‘history’ and ‘diagram’ in the Rousseau part. I thereby establish a Derridean concept of history which aims at accounting for the continuities and discontinuities of the past. This is in contrast to (...) some criticism that Derrida leaves behind, or inadequately accounts for history. Derrida describes a necessarily contorted condition of relating any historical event or development to itself or to another. This historicity informs other well-known aspects of Derrida's work, like the ‘quasi-transcendental’ terms he developed. I conclude that ‘history’ is a critical element in any understanding of deconstruction, and that deconstruction entails new kinds of history, but that some axioms...</code> | <code>Derrida and Other Philosophers</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | <code>Derrida: Animals</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
+  | <code>It is a struggle against the French policy of oppression, of economic impoverishment, and of social, cultural and political destruction of the Algerian people. Sartre, as a citizen of the oppressive state, chose to say Yes to the violent struggle of the oppressed, and No to the French violence. He explains that the aim of French violence is not only: the keeping of these enslaved men at arm's length; it seeks to dehumanize them. Everything will be done to wipe out their traditions, to substitute our language for theirs and to destroy their culture without giving them ours. Sheer physical fatigue will stupefy them. Starved and ill, if they have any spirit left, fear will finish the job; guns are levelled at the peasant; civilians come to take over his land and force him, by dint of flogging, to till the land for them. If he shows fight, the soldiers fire and he is no longer a man at all.2 Thus, the Algerians were forced to choose between two possibilities: slavery or freedom. They chose...</code> | <code>To avoid any political recognition of the freedom of the oppressed, and of their national independence, the oppressors will appeal to the status quo. Yet the colonialist prefers to evoke possibilities of social improvement because he knows that the demands of the natives are primarily political. And they are primarily political because the natives are aware that 'polities', in the colonies, is quite simply the installation and the regular functioning of an enormous repressive apparatus which alone permits super-exploitation. (CDR note 721) A Response To Hannah Arendt Arendt does not mention Sartre's political approach. It seems that she views the violent struggle of the national liberation movements as an antipolitical act, a destruction of the political realm, which she called the only true public realm. She repeatedly explains that the public realm is created by an exchange of opinions and dialogue between people, and by constituting laws that ensure stability and permanence. Violenc...</code> | <code>system and her [CIO] system diffel only in this respect. Thus, the "arbitrary 'bite' " [Mayo, p. 278], which Mayo finds objectionable in the [CIalt.] system is due to the satisfaction of a condition proposed by Neyman, a condition [CIO] stands in violation of.10 Lastly, on pp. 58-63 of my book, I offer a rebuttal to the objection discussed here, the objection that estimates labeled "best" by N-P standards may be deficient with respect to the legitimate concern to avoid conflicts between confidence levels and known (precise) probabilities. I base the rebuttal on a novel criterion: confidence equivalence. Perhaps others will find that defense adequate to excuse the triviality of (some) N-P "best" procedures. I do not. Nor do I find Mayo's proposals sufficient for the question at hand. 8This is Neyman's condition (ii) (Neyman 1937, p. 267). He uses it to eliminate a candidate estimation system, his #(1), pp. 269-70. 9I have recently discovered that R. von Mises observed this same difficul...</code> |
+  | <code>This paper explores the managerial aspects of the relationship with stakeholders, under the assumption that transfer of knowledge is being made from relationship marketing and market orientation perspectives. These marketing tools may prove useful to manage the relationship with other stakeholders, as has been the case with customers. This study focuses on a sample of Spanish companies representing 43% of listed companies with the largest market capitalization. Given that this is the first time that corporate relationship with stakeholders is analyzed (...) in Spain, a qualitative technique (case analysis) was used. The main conclusion of the study is that most of the participant companies have a reactive position vis-à-vis stakeholders management systems. This attitude is reflected in their concern exclusively about ethical indexes managers. (shrink)</code>                                                                                                                                             | <code>Business Ethics</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | <code>Specific Freedoms, Misc</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim"
+  }
+  ```
+
+### Evaluation Dataset
+
+#### Unnamed Dataset
+
+
+* Size: 4,860 evaluation samples
+* Columns: <code>anchor</code>, <code>positive</code>, and <code>negative</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | anchor                                                                               | positive                                                                             | negative                                                                             |
+  |:--------|:-------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
+  | type    | string                                                                               | string                                                                               | string                                                                               |
+  | details | <ul><li>min: 5 tokens</li><li>mean: 234.77 tokens</li><li>max: 1438 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 123.62 tokens</li><li>max: 8192 tokens</li></ul> | <ul><li>min: 3 tokens</li><li>mean: 121.25 tokens</li><li>max: 1090 tokens</li></ul> |
+* Samples:
+  | anchor                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | positive                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | negative                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+  |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>He next proceeds to definitions of logically necessary propositions and of general propositions. He then examines with some care the notion of an empirical law of nature, and concludes that such a law is neither an entailment proposition nor a collective general proposition. His own account is that an empirical law is a conjunction of a logical statement to the effect that the relation between two ostensive concepts, say 'P' and 'Q,' is one of inclusion-or-overlap (or, in the case of '-Q,' of exclusion-oroverlap), and of a factual statement that as a matter of fact everything is either not a P or else a Q. In spite of the ingenuity of this view two criticisms occur at once. How could the factual statement be known? Even if known, how would it justify the BOOK REVIEW 51 counterfactual statement, "If anything were a P, it would be a Q" In discussing the use of logical propositions in deductive reasoning and definitions, K6rner admits that we commonly make use of excessive entailments, e....</code> | <code>It applies at most to only a subset of beautiful things. K6rner also states an embryonic theory of moral action. He holds that the proposition, "a person, X, states that an action b, which has naturalistic characteristics 'P,' is moral," is equivalent to the proposition " X applies 'P' to b and accepts 'P' practically." To accept a concept practically is to desire an action which satisfies the concept and to desire that everyone else desire like actions in like circumstances. His view here has the advantage of recognizing the implicit universalizability of moral statefnents, but otherwise it seems to be only a slightly modified emotive or attitudinal theory. Thus, for example, he says that a person "believes in" a purely teleological ethics if he practically accepts only teleological concepts and rules. In a final group of four chapters K6rner turns his attention to metaphysical "directives. " These are not propositions but BOOK REVIEW 53 rules. They include in each case a rule for the...</code> | <code>The soul is invisible, because it is that which does the seeing. Its externally given and appearing image is the physical, which may be studied in abstraction as if it were real in and for itself. Philosophy of science is the logical reflection on and of the meaning of existence qua scientifically knowing in correlation with the objects of the sciences. In opposition to irrationalism, for which logic falsifies immediate experience, scientific method criticizes immediate experience for being logically absurd : The sun rises in the East, wanders through the sky, and settles in the West ; this is true in immediate experience, but false in astronomy. The scientific worldview sees the world as object for a knowing subject ; perceptual data are thought in the logical form of concepts related in propositions. The principle of scientific reason assumes a partial identity between the logical forms in the mind and the same logical forms as determining objects of knowledge ; that which makes the ...</code> |
+  | <code>Educating the gaze is easily understood as becoming conscious about what is 'really' happening in the world and becoming aware of the way our gaze is itself bound to a perspective and particular position. However, the paper explores a different idea. It understands educating the gaze not in the sense of 'educare' (teaching) but of 'e-ducere' as leading out, reaching out. E-ducating the gaze is not about getting at a liberated or critical view, but about liberating or displacing our view. (...) It is not about becoming conscious or aware , but about becoming attentive , about paying attention . E-ducating the gaze, then, is not depending on method, but relying on discipline; it does not require a rich methodology, but asks for a poor pedagogy, i.e. for practices which allow to expose ourselves. One example of such practice is that of walking. Consequently e-ducating the gaze could be about an invitation to go walking. This idea is explored b way of a comment on two quotations, one by Wa...</code> | <code>Applied Ethics</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | <code>Modal and Intensional Logic, Misc</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+  | <code>There is still no consensus. Furthermore, it is also possible that Boltzmann held one view in his meth odology of science and another in what might be called ontology or his theory of nature (Blackmore 1972, 1982). But starting in 1990 a vast amount of previously unsuspected informa tion began to appear for the first time, which may have initially seemed to allow for the possibility that basic agreement on what Boltzmann's real' philosophy was might finally be attained. Ilse M. Fasol-Boltzmann, Boltzmann's granddaughter published a book in that year which included a great deal of philosophical material from him which she had translated from his original shorthand (Fasol-Boltzmann 1990). She also published eighteen of his lectures on natural philosophy plus Boltzmann's notes for them as well as several other philosophical fragments. We have included INTRODUCTION 3 a translation of three lectures plus an analysis of the remaining fifteen lectures in this anthology But the ideas revealed ...</code> | <code>Nevertheless, his apparent opposition to efficient causes makes one wonder just how realistic or practical his apparent or nascent world view was at that time (Fasol-Boltzmann 1990, p. 273). In conclusion, the new material and lectures clearly increase our under standing about Boltzmann the philosopher, methodologist, and mathem atician, even if a few contradictory remarks may also add to our confusion 8 JOHN BLACKMORE in some ways. But all in all the new data has added a great deal toward putting the philosophical perspective of this great man back together again. Like Humpty Dumpty who had a great fall, Boltzmann's philosophical heritage had been broken into many pieces, but all the king's scholars are rejoining enough together to gradually recreate an original and valuable outlook. Soon, many more people will appreciate that Ludwig Boltzmann was a profound if very troubled thinker and that there is a coherent system within the range of ideas which he was considering, even if he coul...</code> | <code>16-24 , and a more popular exposition  de l'esprit,  . 18-27.</code>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+* Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
+  ```json
+  {
+      "scale": 20.0,
+      "similarity_fct": "cos_sim"
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: epoch
+- `per_device_eval_batch_size`: 16
+- `learning_rate`: 3e-05
+- `num_train_epochs`: 1
+- `warmup_ratio`: 0.1
+- `fp16`: True
+- `load_best_model_at_end`: True
+- `ddp_find_unused_parameters`: False
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: epoch
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 8
+- `per_device_eval_batch_size`: 16
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 3e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: True
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: False
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+<details><summary>Click to expand</summary>
+
+| Epoch  | Step  | Training Loss | Validation Loss | cosine_accuracy |
+|:------:|:-----:|:-------------:|:---------------:|:---------------:|
+| 0.0022 | 25    | 1.2002        | -               | -               |
+| 0.0043 | 50    | 1.1623        | -               | -               |
+| 0.0065 | 75    | 1.2012        | -               | -               |
+| 0.0087 | 100   | 1.1853        | -               | -               |
+| 0.0108 | 125   | 0.9767        | -               | -               |
+| 0.0130 | 150   | 0.865         | -               | -               |
+| 0.0152 | 175   | 0.7733        | -               | -               |
+| 0.0173 | 200   | 0.9545        | -               | -               |
+| 0.0195 | 225   | 0.8309        | -               | -               |
+| 0.0217 | 250   | 0.7514        | -               | -               |
+| 0.0238 | 275   | 0.5555        | -               | -               |
+| 0.0260 | 300   | 0.563         | -               | -               |
+| 0.0282 | 325   | 0.618         | -               | -               |
+| 0.0303 | 350   | 0.6538        | -               | -               |
+| 0.0325 | 375   | 0.5802        | -               | -               |
+| 0.0347 | 400   | 0.6568        | -               | -               |
+| 0.0368 | 425   | 0.4934        | -               | -               |
+| 0.0390 | 450   | 0.597         | -               | -               |
+| 0.0412 | 475   | 0.3812        | -               | -               |
+| 0.0433 | 500   | 0.482         | -               | -               |
+| 0.0455 | 525   | 0.5347        | -               | -               |
+| 0.0476 | 550   | 0.5012        | -               | -               |
+| 0.0498 | 575   | 0.5765        | -               | -               |
+| 0.0520 | 600   | 0.4286        | -               | -               |
+| 0.0541 | 625   | 0.5167        | -               | -               |
+| 0.0563 | 650   | 0.4791        | -               | -               |
+| 0.0585 | 675   | 0.5022        | -               | -               |
+| 0.0606 | 700   | 0.438         | -               | -               |
+| 0.0628 | 725   | 0.3995        | -               | -               |
+| 0.0650 | 750   | 0.2924        | -               | -               |
+| 0.0671 | 775   | 0.4391        | -               | -               |
+| 0.0693 | 800   | 0.4328        | -               | -               |
+| 0.0715 | 825   | 0.5658        | -               | -               |
+| 0.0736 | 850   | 0.4541        | -               | -               |
+| 0.0758 | 875   | 0.5381        | -               | -               |
+| 0.0780 | 900   | 0.4523        | -               | -               |
+| 0.0801 | 925   | 0.3522        | -               | -               |
+| 0.0823 | 950   | 0.4475        | -               | -               |
+| 0.0845 | 975   | 0.4448        | -               | -               |
+| 0.0866 | 1000  | 0.407         | -               | -               |
+| 0.0888 | 1025  | 0.4616        | -               | -               |
+| 0.0910 | 1050  | 0.4213        | -               | -               |
+| 0.0931 | 1075  | 0.465         | -               | -               |
+| 0.0953 | 1100  | 0.2964        | -               | -               |
+| 0.0975 | 1125  | 0.4414        | -               | -               |
+| 0.0996 | 1150  | 0.3508        | -               | -               |
+| 0.1018 | 1175  | 0.3362        | -               | -               |
+| 0.1040 | 1200  | 0.4953        | -               | -               |
+| 0.1061 | 1225  | 0.4041        | -               | -               |
+| 0.1083 | 1250  | 0.3773        | -               | -               |
+| 0.1105 | 1275  | 0.3574        | -               | -               |
+| 0.1126 | 1300  | 0.642         | -               | -               |
+| 0.1148 | 1325  | 0.3783        | -               | -               |
+| 0.1170 | 1350  | 0.4905        | -               | -               |
+| 0.1191 | 1375  | 0.3937        | -               | -               |
+| 0.1213 | 1400  | 0.4245        | -               | -               |
+| 0.1235 | 1425  | 0.4139        | -               | -               |
+| 0.1256 | 1450  | 0.4305        | -               | -               |
+| 0.1278 | 1475  | 0.675         | -               | -               |
+| 0.1299 | 1500  | 0.55          | -               | -               |
+| 0.1321 | 1525  | 0.4033        | -               | -               |
+| 0.1343 | 1550  | 0.4167        | -               | -               |
+| 0.1364 | 1575  | 0.3814        | -               | -               |
+| 0.1386 | 1600  | 0.5183        | -               | -               |
+| 0.1408 | 1625  | 0.3343        | -               | -               |
+| 0.1429 | 1650  | 0.4212        | -               | -               |
+| 0.1451 | 1675  | 0.4737        | -               | -               |
+| 0.1473 | 1700  | 0.4563        | -               | -               |
+| 0.1494 | 1725  | 0.4251        | -               | -               |
+| 0.1516 | 1750  | 0.3497        | -               | -               |
+| 0.1538 | 1775  | 0.3753        | -               | -               |
+| 0.1559 | 1800  | 0.4031        | -               | -               |
+| 0.1581 | 1825  | 0.4037        | -               | -               |
+| 0.1603 | 1850  | 0.4114        | -               | -               |
+| 0.1624 | 1875  | 0.3848        | -               | -               |
+| 0.1646 | 1900  | 0.5088        | -               | -               |
+| 0.1668 | 1925  | 0.4032        | -               | -               |
+| 0.1689 | 1950  | 0.3354        | -               | -               |
+| 0.1711 | 1975  | 0.4163        | -               | -               |
+| 0.1733 | 2000  | 0.3715        | -               | -               |
+| 0.1754 | 2025  | 0.3424        | -               | -               |
+| 0.1776 | 2050  | 0.3311        | -               | -               |
+| 0.1798 | 2075  | 0.4362        | -               | -               |
+| 0.1819 | 2100  | 0.4441        | -               | -               |
+| 0.1841 | 2125  | 0.3122        | -               | -               |
+| 0.1863 | 2150  | 0.3717        | -               | -               |
+| 0.1884 | 2175  | 0.3461        | -               | -               |
+| 0.1906 | 2200  | 0.4816        | -               | -               |
+| 0.1928 | 2225  | 0.4784        | -               | -               |
+| 0.1949 | 2250  | 0.4334        | -               | -               |
+| 0.1971 | 2275  | 0.3437        | -               | -               |
+| 0.1993 | 2300  | 0.4333        | -               | -               |
+| 0.2014 | 2325  | 0.3609        | -               | -               |
+| 0.2036 | 2350  | 0.3437        | -               | -               |
+| 0.2058 | 2375  | 0.4911        | -               | -               |
+| 0.2079 | 2400  | 0.3872        | -               | -               |
+| 0.2101 | 2425  | 0.276         | -               | -               |
+| 0.2122 | 2450  | 0.3318        | -               | -               |
+| 0.2144 | 2475  | 0.4833        | -               | -               |
+| 0.2166 | 2500  | 0.4656        | -               | -               |
+| 0.2187 | 2525  | 0.4232        | -               | -               |
+| 0.2209 | 2550  | 0.434         | -               | -               |
+| 0.2231 | 2575  | 0.2479        | -               | -               |
+| 0.2252 | 2600  | 0.4656        | -               | -               |
+| 0.2274 | 2625  | 0.3881        | -               | -               |
+| 0.2296 | 2650  | 0.3637        | -               | -               |
+| 0.2317 | 2675  | 0.3099        | -               | -               |
+| 0.2339 | 2700  | 0.3933        | -               | -               |
+| 0.2361 | 2725  | 0.3789        | -               | -               |
+| 0.2382 | 2750  | 0.4056        | -               | -               |
+| 0.2404 | 2775  | 0.4132        | -               | -               |
+| 0.2426 | 2800  | 0.375         | -               | -               |
+| 0.2447 | 2825  | 0.3026        | -               | -               |
+| 0.2469 | 2850  | 0.5372        | -               | -               |
+| 0.2491 | 2875  | 0.4233        | -               | -               |
+| 0.2512 | 2900  | 0.2945        | -               | -               |
+| 0.2534 | 2925  | 0.2916        | -               | -               |
+| 0.2556 | 2950  | 0.3536        | -               | -               |
+| 0.2577 | 2975  | 0.3246        | -               | -               |
+| 0.2599 | 3000  | 0.4236        | -               | -               |
+| 0.2621 | 3025  | 0.4088        | -               | -               |
+| 0.2642 | 3050  | 0.4522        | -               | -               |
+| 0.2664 | 3075  | 0.3445        | -               | -               |
+| 0.2686 | 3100  | 0.3575        | -               | -               |
+| 0.2707 | 3125  | 0.3809        | -               | -               |
+| 0.2729 | 3150  | 0.3364        | -               | -               |
+| 0.2751 | 3175  | 0.4103        | -               | -               |
+| 0.2772 | 3200  | 0.3502        | -               | -               |
+| 0.2794 | 3225  | 0.2632        | -               | -               |
+| 0.2816 | 3250  | 0.406         | -               | -               |
+| 0.2837 | 3275  | 0.4363        | -               | -               |
+| 0.2859 | 3300  | 0.2819        | -               | -               |
+| 0.2881 | 3325  | 0.3421        | -               | -               |
+| 0.2902 | 3350  | 0.269         | -               | -               |
+| 0.2924 | 3375  | 0.2902        | -               | -               |
+| 0.2946 | 3400  | 0.3548        | -               | -               |
+| 0.2967 | 3425  | 0.4575        | -               | -               |
+| 0.2989 | 3450  | 0.3942        | -               | -               |
+| 0.3010 | 3475  | 0.3537        | -               | -               |
+| 0.3032 | 3500  | 0.3672        | -               | -               |
+| 0.3054 | 3525  | 0.3502        | -               | -               |
+| 0.3075 | 3550  | 0.2545        | -               | -               |
+| 0.3097 | 3575  | 0.2544        | -               | -               |
+| 0.3119 | 3600  | 0.3443        | -               | -               |
+| 0.3140 | 3625  | 0.3784        | -               | -               |
+| 0.3162 | 3650  | 0.3828        | -               | -               |
+| 0.3184 | 3675  | 0.4032        | -               | -               |
+| 0.3205 | 3700  | 0.2556        | -               | -               |
+| 0.3227 | 3725  | 0.3352        | -               | -               |
+| 0.3249 | 3750  | 0.4054        | -               | -               |
+| 0.3270 | 3775  | 0.3049        | -               | -               |
+| 0.3292 | 3800  | 0.2223        | -               | -               |
+| 0.3314 | 3825  | 0.4878        | -               | -               |
+| 0.3335 | 3850  | 0.3015        | -               | -               |
+| 0.3357 | 3875  | 0.3816        | -               | -               |
+| 0.3379 | 3900  | 0.3334        | -               | -               |
+| 0.3400 | 3925  | 0.3724        | -               | -               |
+| 0.3422 | 3950  | 0.4217        | -               | -               |
+| 0.3444 | 3975  | 0.4339        | -               | -               |
+| 0.3465 | 4000  | 0.3642        | -               | -               |
+| 0.3487 | 4025  | 0.3819        | -               | -               |
+| 0.3509 | 4050  | 0.2796        | -               | -               |
+| 0.3530 | 4075  | 0.4277        | -               | -               |
+| 0.3552 | 4100  | 0.3407        | -               | -               |
+| 0.3574 | 4125  | 0.2781        | -               | -               |
+| 0.3595 | 4150  | 0.4274        | -               | -               |
+| 0.3617 | 4175  | 0.3609        | -               | -               |
+| 0.3639 | 4200  | 0.3476        | -               | -               |
+| 0.3660 | 4225  | 0.41          | -               | -               |
+| 0.3682 | 4250  | 0.4003        | -               | -               |
+| 0.3704 | 4275  | 0.306         | -               | -               |
+| 0.3725 | 4300  | 0.2335        | -               | -               |
+| 0.3747 | 4325  | 0.2733        | -               | -               |
+| 0.3769 | 4350  | 0.3007        | -               | -               |
+| 0.3790 | 4375  | 0.3086        | -               | -               |
+| 0.3812 | 4400  | 0.365         | -               | -               |
+| 0.3833 | 4425  | 0.3255        | -               | -               |
+| 0.3855 | 4450  | 0.3765        | -               | -               |
+| 0.3877 | 4475  | 0.2946        | -               | -               |
+| 0.3898 | 4500  | 0.3298        | -               | -               |
+| 0.3920 | 4525  | 0.3645        | -               | -               |
+| 0.3942 | 4550  | 0.2403        | -               | -               |
+| 0.3963 | 4575  | 0.28          | -               | -               |
+| 0.3985 | 4600  | 0.3814        | -               | -               |
+| 0.4007 | 4625  | 0.3419        | -               | -               |
+| 0.4028 | 4650  | 0.3374        | -               | -               |
+| 0.4050 | 4675  | 0.3511        | -               | -               |
+| 0.4072 | 4700  | 0.4339        | -               | -               |
+| 0.4093 | 4725  | 0.3441        | -               | -               |
+| 0.4115 | 4750  | 0.346         | -               | -               |
+| 0.4137 | 4775  | 0.3723        | -               | -               |
+| 0.4158 | 4800  | 0.2075        | -               | -               |
+| 0.4180 | 4825  | 0.2431        | -               | -               |
+| 0.4202 | 4850  | 0.2642        | -               | -               |
+| 0.4223 | 4875  | 0.1763        | -               | -               |
+| 0.4245 | 4900  | 0.3862        | -               | -               |
+| 0.4267 | 4925  | 0.3053        | -               | -               |
+| 0.4288 | 4950  | 0.3162        | -               | -               |
+| 0.4310 | 4975  | 0.3178        | -               | -               |
+| 0.4332 | 5000  | 0.2789        | -               | -               |
+| 0.4353 | 5025  | 0.1777        | -               | -               |
+| 0.4375 | 5050  | 0.4155        | -               | -               |
+| 0.4397 | 5075  | 0.2983        | -               | -               |
+| 0.4418 | 5100  | 0.3687        | -               | -               |
+| 0.4440 | 5125  | 0.2428        | -               | -               |
+| 0.4462 | 5150  | 0.3071        | -               | -               |
+| 0.4483 | 5175  | 0.2911        | -               | -               |
+| 0.4505 | 5200  | 0.3152        | -               | -               |
+| 0.4527 | 5225  | 0.2776        | -               | -               |
+| 0.4548 | 5250  | 0.2674        | -               | -               |
+| 0.4570 | 5275  | 0.3035        | -               | -               |
+| 0.4592 | 5300  | 0.3352        | -               | -               |
+| 0.4613 | 5325  | 0.3879        | -               | -               |
+| 0.4635 | 5350  | 0.3828        | -               | -               |
+| 0.4657 | 5375  | 0.2797        | -               | -               |
+| 0.4678 | 5400  | 0.3492        | -               | -               |
+| 0.4700 | 5425  | 0.5           | -               | -               |
+| 0.4721 | 5450  | 0.2317        | -               | -               |
+| 0.4743 | 5475  | 0.2411        | -               | -               |
+| 0.4765 | 5500  | 0.277         | -               | -               |
+| 0.4786 | 5525  | 0.4112        | -               | -               |
+| 0.4808 | 5550  | 0.5116        | -               | -               |
+| 0.4830 | 5575  | 0.3264        | -               | -               |
+| 0.4851 | 5600  | 0.3688        | -               | -               |
+| 0.4873 | 5625  | 0.3224        | -               | -               |
+| 0.4895 | 5650  | 0.3778        | -               | -               |
+| 0.4916 | 5675  | 0.3671        | -               | -               |
+| 0.4938 | 5700  | 0.3331        | -               | -               |
+| 0.4960 | 5725  | 0.3426        | -               | -               |
+| 0.4981 | 5750  | 0.2863        | -               | -               |
+| 0.5003 | 5775  | 0.5822        | -               | -               |
+| 0.5025 | 5800  | 0.2687        | -               | -               |
+| 0.5046 | 5825  | 0.3365        | -               | -               |
+| 0.5068 | 5850  | 0.4609        | -               | -               |
+| 0.5090 | 5875  | 0.3127        | -               | -               |
+| 0.5111 | 5900  | 0.2705        | -               | -               |
+| 0.5133 | 5925  | 0.3089        | -               | -               |
+| 0.5155 | 5950  | 0.3386        | -               | -               |
+| 0.5176 | 5975  | 0.3796        | -               | -               |
+| 0.5198 | 6000  | 0.4231        | -               | -               |
+| 0.5220 | 6025  | 0.3922        | -               | -               |
+| 0.5241 | 6050  | 0.3138        | -               | -               |
+| 0.5263 | 6075  | 0.3106        | -               | -               |
+| 0.5285 | 6100  | 0.188         | -               | -               |
+| 0.5306 | 6125  | 0.209         | -               | -               |
+| 0.5328 | 6150  | 0.2617        | -               | -               |
+| 0.5350 | 6175  | 0.3059        | -               | -               |
+| 0.5371 | 6200  | 0.2764        | -               | -               |
+| 0.5393 | 6225  | 0.2801        | -               | -               |
+| 0.5415 | 6250  | 0.3744        | -               | -               |
+| 0.5436 | 6275  | 0.3067        | -               | -               |
+| 0.5458 | 6300  | 0.3305        | -               | -               |
+| 0.5480 | 6325  | 0.2827        | -               | -               |
+| 0.5501 | 6350  | 0.2712        | -               | -               |
+| 0.5523 | 6375  | 0.2677        | -               | -               |
+| 0.5544 | 6400  | 0.4269        | -               | -               |
+| 0.5566 | 6425  | 0.3834        | -               | -               |
+| 0.5588 | 6450  | 0.4177        | -               | -               |
+| 0.5609 | 6475  | 0.2457        | -               | -               |
+| 0.5631 | 6500  | 0.348         | -               | -               |
+| 0.5653 | 6525  | 0.3035        | -               | -               |
+| 0.5674 | 6550  | 0.39          | -               | -               |
+| 0.5696 | 6575  | 0.366         | -               | -               |
+| 0.5718 | 6600  | 0.2299        | -               | -               |
+| 0.5739 | 6625  | 0.1737        | -               | -               |
+| 0.5761 | 6650  | 0.3773        | -               | -               |
+| 0.5783 | 6675  | 0.3409        | -               | -               |
+| 0.5804 | 6700  | 0.1739        | -               | -               |
+| 0.5826 | 6725  | 0.3462        | -               | -               |
+| 0.5848 | 6750  | 0.2976        | -               | -               |
+| 0.5869 | 6775  | 0.3246        | -               | -               |
+| 0.5891 | 6800  | 0.3808        | -               | -               |
+| 0.5913 | 6825  | 0.2926        | -               | -               |
+| 0.5934 | 6850  | 0.2709        | -               | -               |
+| 0.5956 | 6875  | 0.3777        | -               | -               |
+| 0.5978 | 6900  | 0.2834        | -               | -               |
+| 0.5999 | 6925  | 0.2965        | -               | -               |
+| 0.6021 | 6950  | 0.2399        | -               | -               |
+| 0.6043 | 6975  | 0.2936        | -               | -               |
+| 0.6064 | 7000  | 0.2674        | -               | -               |
+| 0.6086 | 7025  | 0.265         | -               | -               |
+| 0.6108 | 7050  | 0.3257        | -               | -               |
+| 0.6129 | 7075  | 0.3504        | -               | -               |
+| 0.6151 | 7100  | 0.1485        | -               | -               |
+| 0.6173 | 7125  | 0.2598        | -               | -               |
+| 0.6194 | 7150  | 0.2838        | -               | -               |
+| 0.6216 | 7175  | 0.3391        | -               | -               |
+| 0.6238 | 7200  | 0.3568        | -               | -               |
+| 0.6259 | 7225  | 0.3001        | -               | -               |
+| 0.6281 | 7250  | 0.2613        | -               | -               |
+| 0.6303 | 7275  | 0.3379        | -               | -               |
+| 0.6324 | 7300  | 0.3347        | -               | -               |
+| 0.6346 | 7325  | 0.242         | -               | -               |
+| 0.6367 | 7350  | 0.3076        | -               | -               |
+| 0.6389 | 7375  | 0.3055        | -               | -               |
+| 0.6411 | 7400  | 0.4014        | -               | -               |
+| 0.6432 | 7425  | 0.3723        | -               | -               |
+| 0.6454 | 7450  | 0.3421        | -               | -               |
+| 0.6476 | 7475  | 0.4306        | -               | -               |
+| 0.6497 | 7500  | 0.2536        | -               | -               |
+| 0.6519 | 7525  | 0.264         | -               | -               |
+| 0.6541 | 7550  | 0.1767        | -               | -               |
+| 0.6562 | 7575  | 0.259         | -               | -               |
+| 0.6584 | 7600  | 0.2761        | -               | -               |
+| 0.6606 | 7625  | 0.2934        | -               | -               |
+| 0.6627 | 7650  | 0.3055        | -               | -               |
+| 0.6649 | 7675  | 0.2532        | -               | -               |
+| 0.6671 | 7700  | 0.2942        | -               | -               |
+| 0.6692 | 7725  | 0.2048        | -               | -               |
+| 0.6714 | 7750  | 0.2884        | -               | -               |
+| 0.6736 | 7775  | 0.3598        | -               | -               |
+| 0.6757 | 7800  | 0.3318        | -               | -               |
+| 0.6779 | 7825  | 0.3058        | -               | -               |
+| 0.6801 | 7850  | 0.3395        | -               | -               |
+| 0.6822 | 7875  | 0.2973        | -               | -               |
+| 0.6844 | 7900  | 0.2741        | -               | -               |
+| 0.6866 | 7925  | 0.2493        | -               | -               |
+| 0.6887 | 7950  | 0.2966        | -               | -               |
+| 0.6909 | 7975  | 0.3207        | -               | -               |
+| 0.6931 | 8000  | 0.2501        | -               | -               |
+| 0.6952 | 8025  | 0.4028        | -               | -               |
+| 0.6974 | 8050  | 0.3549        | -               | -               |
+| 0.6996 | 8075  | 0.3805        | -               | -               |
+| 0.7017 | 8100  | 0.353         | -               | -               |
+| 0.7039 | 8125  | 0.3569        | -               | -               |
+| 0.7061 | 8150  | 0.2588        | -               | -               |
+| 0.7082 | 8175  | 0.2252        | -               | -               |
+| 0.7104 | 8200  | 0.2747        | -               | -               |
+| 0.7126 | 8225  | 0.3239        | -               | -               |
+| 0.7147 | 8250  | 0.2954        | -               | -               |
+| 0.7169 | 8275  | 0.3749        | -               | -               |
+| 0.7191 | 8300  | 0.2757        | -               | -               |
+| 0.7212 | 8325  | 0.3012        | -               | -               |
+| 0.7234 | 8350  | 0.2985        | -               | -               |
+| 0.7255 | 8375  | 0.2656        | -               | -               |
+| 0.7277 | 8400  | 0.2007        | -               | -               |
+| 0.7299 | 8425  | 0.2402        | -               | -               |
+| 0.7320 | 8450  | 0.3434        | -               | -               |
+| 0.7342 | 8475  | 0.2628        | -               | -               |
+| 0.7364 | 8500  | 0.265         | -               | -               |
+| 0.7385 | 8525  | 0.3748        | -               | -               |
+| 0.7407 | 8550  | 0.249         | -               | -               |
+| 0.7429 | 8575  | 0.3375        | -               | -               |
+| 0.7450 | 8600  | 0.3336        | -               | -               |
+| 0.7472 | 8625  | 0.3702        | -               | -               |
+| 0.7494 | 8650  | 0.3494        | -               | -               |
+| 0.7515 | 8675  | 0.2996        | -               | -               |
+| 0.7537 | 8700  | 0.2433        | -               | -               |
+| 0.7559 | 8725  | 0.3027        | -               | -               |
+| 0.7580 | 8750  | 0.382         | -               | -               |
+| 0.7602 | 8775  | 0.2874        | -               | -               |
+| 0.7624 | 8800  | 0.2737        | -               | -               |
+| 0.7645 | 8825  | 0.3212        | -               | -               |
+| 0.7667 | 8850  | 0.3475        | -               | -               |
+| 0.7689 | 8875  | 0.221         | -               | -               |
+| 0.7710 | 8900  | 0.2587        | -               | -               |
+| 0.7732 | 8925  | 0.2852        | -               | -               |
+| 0.7754 | 8950  | 0.3837        | -               | -               |
+| 0.7775 | 8975  | 0.2333        | -               | -               |
+| 0.7797 | 9000  | 0.3036        | -               | -               |
+| 0.7819 | 9025  | 0.3287        | -               | -               |
+| 0.7840 | 9050  | 0.3248        | -               | -               |
+| 0.7862 | 9075  | 0.2395        | -               | -               |
+| 0.7884 | 9100  | 0.2647        | -               | -               |
+| 0.7905 | 9125  | 0.3345        | -               | -               |
+| 0.7927 | 9150  | 0.3421        | -               | -               |
+| 0.7949 | 9175  | 0.3496        | -               | -               |
+| 0.7970 | 9200  | 0.253         | -               | -               |
+| 0.7992 | 9225  | 0.3462        | -               | -               |
+| 0.8014 | 9250  | 0.2688        | -               | -               |
+| 0.8035 | 9275  | 0.3301        | -               | -               |
+| 0.8057 | 9300  | 0.3382        | -               | -               |
+| 0.8078 | 9325  | 0.2219        | -               | -               |
+| 0.8100 | 9350  | 0.278         | -               | -               |
+| 0.8122 | 9375  | 0.2338        | -               | -               |
+| 0.8143 | 9400  | 0.2732        | -               | -               |
+| 0.8165 | 9425  | 0.2973        | -               | -               |
+| 0.8187 | 9450  | 0.2783        | -               | -               |
+| 0.8208 | 9475  | 0.2418        | -               | -               |
+| 0.8230 | 9500  | 0.2603        | -               | -               |
+| 0.8252 | 9525  | 0.1888        | -               | -               |
+| 0.8273 | 9550  | 0.2581        | -               | -               |
+| 0.8295 | 9575  | 0.2742        | -               | -               |
+| 0.8317 | 9600  | 0.2156        | -               | -               |
+| 0.8338 | 9625  | 0.3317        | -               | -               |
+| 0.8360 | 9650  | 0.1967        | -               | -               |
+| 0.8382 | 9675  | 0.1701        | -               | -               |
+| 0.8403 | 9700  | 0.3064        | -               | -               |
+| 0.8425 | 9725  | 0.3511        | -               | -               |
+| 0.8447 | 9750  | 0.2461        | -               | -               |
+| 0.8468 | 9775  | 0.3047        | -               | -               |
+| 0.8490 | 9800  | 0.3234        | -               | -               |
+| 0.8512 | 9825  | 0.2843        | -               | -               |
+| 0.8533 | 9850  | 0.3365        | -               | -               |
+| 0.8555 | 9875  | 0.3802        | -               | -               |
+| 0.8577 | 9900  | 0.2587        | -               | -               |
+| 0.8598 | 9925  | 0.2367        | -               | -               |
+| 0.8620 | 9950  | 0.2971        | -               | -               |
+| 0.8642 | 9975  | 0.2884        | -               | -               |
+| 0.8663 | 10000 | 0.2296        | -               | -               |
+| 0.8685 | 10025 | 0.3145        | -               | -               |
+| 0.8707 | 10050 | 0.178         | -               | -               |
+| 0.8728 | 10075 | 0.2681        | -               | -               |
+| 0.8750 | 10100 | 0.3191        | -               | -               |
+| 0.8772 | 10125 | 0.2544        | -               | -               |
+| 0.8793 | 10150 | 0.2965        | -               | -               |
+| 0.8815 | 10175 | 0.317         | -               | -               |
+| 0.8837 | 10200 | 0.2149        | -               | -               |
+| 0.8858 | 10225 | 0.4876        | -               | -               |
+| 0.8880 | 10250 | 0.2984        | -               | -               |
+| 0.8901 | 10275 | 0.3024        | -               | -               |
+| 0.8923 | 10300 | 0.2447        | -               | -               |
+| 0.8945 | 10325 | 0.2684        | -               | -               |
+| 0.8966 | 10350 | 0.1714        | -               | -               |
+| 0.8988 | 10375 | 0.2776        | -               | -               |
+| 0.9010 | 10400 | 0.2745        | -               | -               |
+| 0.9031 | 10425 | 0.3299        | -               | -               |
+| 0.9053 | 10450 | 0.2629        | -               | -               |
+| 0.9075 | 10475 | 0.3627        | -               | -               |
+| 0.9096 | 10500 | 0.2236        | -               | -               |
+| 0.9118 | 10525 | 0.2819        | -               | -               |
+| 0.9140 | 10550 | 0.3129        | -               | -               |
+| 0.9161 | 10575 | 0.3051        | -               | -               |
+| 0.9183 | 10600 | 0.3955        | -               | -               |
+| 0.9205 | 10625 | 0.2493        | -               | -               |
+| 0.9226 | 10650 | 0.2543        | -               | -               |
+| 0.9248 | 10675 | 0.2222        | -               | -               |
+| 0.9270 | 10700 | 0.2823        | -               | -               |
+| 0.9291 | 10725 | 0.3098        | -               | -               |
+| 0.9313 | 10750 | 0.3009        | -               | -               |
+| 0.9335 | 10775 | 0.2623        | -               | -               |
+| 0.9356 | 10800 | 0.1952        | -               | -               |
+| 0.9378 | 10825 | 0.4527        | -               | -               |
+| 0.9400 | 10850 | 0.2323        | -               | -               |
+| 0.9421 | 10875 | 0.3109        | -               | -               |
+| 0.9443 | 10900 | 0.3335        | -               | -               |
+| 0.9465 | 10925 | 0.2862        | -               | -               |
+| 0.9486 | 10950 | 0.4005        | -               | -               |
+| 0.9508 | 10975 | 0.2815        | -               | -               |
+| 0.9530 | 11000 | 0.2157        | -               | -               |
+| 0.9551 | 11025 | 0.3733        | -               | -               |
+| 0.9573 | 11050 | 0.2843        | -               | -               |
+| 0.9595 | 11075 | 0.1963        | -               | -               |
+| 0.9616 | 11100 | 0.3081        | -               | -               |
+| 0.9638 | 11125 | 0.2317        | -               | -               |
+| 0.9660 | 11150 | 0.3027        | -               | -               |
+| 0.9681 | 11175 | 0.3581        | -               | -               |
+| 0.9703 | 11200 | 0.3           | -               | -               |
+| 0.9725 | 11225 | 0.2797        | -               | -               |
+| 0.9746 | 11250 | 0.2918        | -               | -               |
+| 0.9768 | 11275 | 0.2519        | -               | -               |
+| 0.9789 | 11300 | 0.2183        | -               | -               |
+| 0.9811 | 11325 | 0.2764        | -               | -               |
+| 0.9833 | 11350 | 0.4107        | -               | -               |
+| 0.9854 | 11375 | 0.3135        | -               | -               |
+| 0.9876 | 11400 | 0.2138        | -               | -               |
+| 0.9898 | 11425 | 0.2984        | -               | -               |
+| 0.9919 | 11450 | 0.2407        | -               | -               |
+| 0.9941 | 11475 | 0.2449        | -               | -               |
+| 0.9963 | 11500 | 0.2629        | -               | -               |
+| 0.9984 | 11525 | 0.3488        | -               | -               |
+| 1.0    | 11543 | -             | 0.4268          | 0.9693          |
+
+</details>
+
+### Framework Versions
+- Python: 3.10.16
+- Sentence Transformers: 3.3.1
+- Transformers: 4.48.0
+- PyTorch: 2.4.0
+- Accelerate: 1.2.1
+- Datasets: 3.2.0
+- Tokenizers: 0.21.0
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### MultipleNegativesRankingLoss
+```bibtex
+@misc{henderson2017efficient,
+    title={Efficient Natural Language Response Suggestion for Smart Reply},
+    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
+    year={2017},
+    eprint={1705.00652},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

checkpoint-11543/config.json

@@ -0,0 +1,47 @@
+{
+  "_name_or_path": "Alibaba-NLP/gte-modernbert-base",
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50281,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 50281,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "embedding_dropout": 0.0,
+  "eos_token_id": 50282,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 10000.0,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 50283,
+  "position_embedding_type": "absolute",
+  "reference_compile": true,
+  "repad_logits_with_grad": false,
+  "sep_token_id": 50282,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.0",
+  "vocab_size": 50368
+}

checkpoint-11543/config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.3.1",
+    "transformers": "4.48.0",
+    "pytorch": "2.4.0"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

checkpoint-11543/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96616f5c428a86aaed4423cdda82e8a7d4becc0682e33fca47374d01cd7df333
+size 596070136

checkpoint-11543/modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

checkpoint-11543/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:07da211967f6be0e79f951282f9074faf49054e411505bc79bb7770f47d4dc86
+size 1192227066

checkpoint-11543/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3a32083653aa879d14fe5992c263430b4fa744e9c53bba06661aaf71a629a7e0
+size 14244

checkpoint-11543/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:80c3e6b9978ee7a9cc33e3c0c447fc88f0abe81f0665d86d6d894841efc0e05d
+size 1064

checkpoint-11543/sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 8192,
+  "do_lower_case": false
+}

checkpoint-11543/special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-11543/tokenizer_config.json

@@ -0,0 +1,945 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50280": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50281": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50282": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50283": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50284": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50285": {
+      "content": "[unused0]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50286": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50287": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50288": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50289": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50290": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50291": {
+      "content": "[unused6]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50292": {
+      "content": "[unused7]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50293": {
+      "content": "[unused8]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50294": {
+      "content": "[unused9]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50295": {
+      "content": "[unused10]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50296": {
+      "content": "[unused11]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50297": {
+      "content": "[unused12]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50298": {
+      "content": "[unused13]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50299": {
+      "content": "[unused14]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50300": {
+      "content": "[unused15]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50301": {
+      "content": "[unused16]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50302": {
+      "content": "[unused17]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50303": {
+      "content": "[unused18]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50304": {
+      "content": "[unused19]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50305": {
+      "content": "[unused20]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50306": {
+      "content": "[unused21]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50307": {
+      "content": "[unused22]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50308": {
+      "content": "[unused23]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50309": {
+      "content": "[unused24]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50310": {
+      "content": "[unused25]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50311": {
+      "content": "[unused26]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50312": {
+      "content": "[unused27]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50313": {
+      "content": "[unused28]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50314": {
+      "content": "[unused29]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50315": {
+      "content": "[unused30]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50316": {
+      "content": "[unused31]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50317": {
+      "content": "[unused32]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50318": {
+      "content": "[unused33]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50319": {
+      "content": "[unused34]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50320": {
+      "content": "[unused35]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50321": {
+      "content": "[unused36]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50322": {
+      "content": "[unused37]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50323": {
+      "content": "[unused38]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50324": {
+      "content": "[unused39]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50325": {
+      "content": "[unused40]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50326": {
+      "content": "[unused41]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50327": {
+      "content": "[unused42]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50328": {
+      "content": "[unused43]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50329": {
+      "content": "[unused44]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50330": {
+      "content": "[unused45]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50331": {
+      "content": "[unused46]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50332": {
+      "content": "[unused47]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50333": {
+      "content": "[unused48]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50334": {
+      "content": "[unused49]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50335": {
+      "content": "[unused50]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50336": {
+      "content": "[unused51]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50337": {
+      "content": "[unused52]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50338": {
+      "content": "[unused53]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50339": {
+      "content": "[unused54]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50340": {
+      "content": "[unused55]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50341": {
+      "content": "[unused56]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50342": {
+      "content": "[unused57]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50343": {
+      "content": "[unused58]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50344": {
+      "content": "[unused59]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50345": {
+      "content": "[unused60]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50346": {
+      "content": "[unused61]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50347": {
+      "content": "[unused62]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50348": {
+      "content": "[unused63]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50349": {
+      "content": "[unused64]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50350": {
+      "content": "[unused65]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50351": {
+      "content": "[unused66]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50352": {
+      "content": "[unused67]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50353": {
+      "content": "[unused68]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50354": {
+      "content": "[unused69]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50355": {
+      "content": "[unused70]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50356": {
+      "content": "[unused71]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50357": {
+      "content": "[unused72]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50358": {
+      "content": "[unused73]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50359": {
+      "content": "[unused74]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50360": {
+      "content": "[unused75]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50361": {
+      "content": "[unused76]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50362": {
+      "content": "[unused77]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50363": {
+      "content": "[unused78]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50364": {
+      "content": "[unused79]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50365": {
+      "content": "[unused80]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50366": {
+      "content": "[unused81]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50367": {
+      "content": "[unused82]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "[UNK]"
+}

checkpoint-11543/trainer_state.json

@@ -0,0 +1,3278 @@
+{
+  "best_metric": 0.4268312156200409,
+  "best_model_checkpoint": "gte-modernbert-philosophy-v1-1-autotr/checkpoint-11543",
+  "epoch": 1.0,
+  "eval_steps": 500,
+  "global_step": 11543,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0021658147795200556,
+      "grad_norm": 9.93041706085205,
+      "learning_rate": 6.493506493506493e-07,
+      "loss": 1.2002,
+      "step": 25
+    },
+    {
+      "epoch": 0.004331629559040111,
+      "grad_norm": 10.093696594238281,
+      "learning_rate": 1.2987012987012986e-06,
+      "loss": 1.1623,
+      "step": 50
+    },
+    {
+      "epoch": 0.006497444338560167,
+      "grad_norm": 9.860174179077148,
+      "learning_rate": 1.948051948051948e-06,
+      "loss": 1.2012,
+      "step": 75
+    },
+    {
+      "epoch": 0.008663259118080222,
+      "grad_norm": 8.943851470947266,
+      "learning_rate": 2.597402597402597e-06,
+      "loss": 1.1853,
+      "step": 100
+    },
+    {
+      "epoch": 0.010829073897600277,
+      "grad_norm": 5.130438327789307,
+      "learning_rate": 3.246753246753247e-06,
+      "loss": 0.9767,
+      "step": 125
+    },
+    {
+      "epoch": 0.012994888677120333,
+      "grad_norm": 8.450475692749023,
+      "learning_rate": 3.896103896103896e-06,
+      "loss": 0.865,
+      "step": 150
+    },
+    {
+      "epoch": 0.015160703456640388,
+      "grad_norm": 3.7100517749786377,
+      "learning_rate": 4.5454545454545455e-06,
+      "loss": 0.7733,
+      "step": 175
+    },
+    {
+      "epoch": 0.017326518236160444,
+      "grad_norm": 15.069323539733887,
+      "learning_rate": 5.194805194805194e-06,
+      "loss": 0.9545,
+      "step": 200
+    },
+    {
+      "epoch": 0.0194923330156805,
+      "grad_norm": 9.745051383972168,
+      "learning_rate": 5.8181818181818185e-06,
+      "loss": 0.8309,
+      "step": 225
+    },
+    {
+      "epoch": 0.021658147795200554,
+      "grad_norm": 7.115631580352783,
+      "learning_rate": 6.467532467532467e-06,
+      "loss": 0.7514,
+      "step": 250
+    },
+    {
+      "epoch": 0.02382396257472061,
+      "grad_norm": 15.047070503234863,
+      "learning_rate": 7.116883116883117e-06,
+      "loss": 0.5555,
+      "step": 275
+    },
+    {
+      "epoch": 0.025989777354240667,
+      "grad_norm": 14.228421211242676,
+      "learning_rate": 7.766233766233767e-06,
+      "loss": 0.563,
+      "step": 300
+    },
+    {
+      "epoch": 0.02815559213376072,
+      "grad_norm": 8.77304458618164,
+      "learning_rate": 8.415584415584416e-06,
+      "loss": 0.618,
+      "step": 325
+    },
+    {
+      "epoch": 0.030321406913280776,
+      "grad_norm": 4.78096866607666,
+      "learning_rate": 9.064935064935066e-06,
+      "loss": 0.6538,
+      "step": 350
+    },
+    {
+      "epoch": 0.032487221692800834,
+      "grad_norm": 9.978753089904785,
+      "learning_rate": 9.714285714285715e-06,
+      "loss": 0.5802,
+      "step": 375
+    },
+    {
+      "epoch": 0.03465303647232089,
+      "grad_norm": 15.071464538574219,
+      "learning_rate": 1.0363636363636364e-05,
+      "loss": 0.6568,
+      "step": 400
+    },
+    {
+      "epoch": 0.036818851251840944,
+      "grad_norm": 12.352958679199219,
+      "learning_rate": 1.1012987012987013e-05,
+      "loss": 0.4934,
+      "step": 425
+    },
+    {
+      "epoch": 0.038984666031361,
+      "grad_norm": 8.754806518554688,
+      "learning_rate": 1.1662337662337662e-05,
+      "loss": 0.597,
+      "step": 450
+    },
+    {
+      "epoch": 0.04115048081088105,
+      "grad_norm": 2.432300090789795,
+      "learning_rate": 1.2311688311688312e-05,
+      "loss": 0.3812,
+      "step": 475
+    },
+    {
+      "epoch": 0.04331629559040111,
+      "grad_norm": 7.439950466156006,
+      "learning_rate": 1.2961038961038961e-05,
+      "loss": 0.482,
+      "step": 500
+    },
+    {
+      "epoch": 0.04548211036992116,
+      "grad_norm": 14.198251724243164,
+      "learning_rate": 1.361038961038961e-05,
+      "loss": 0.5347,
+      "step": 525
+    },
+    {
+      "epoch": 0.04764792514944122,
+      "grad_norm": 5.91489839553833,
+      "learning_rate": 1.425974025974026e-05,
+      "loss": 0.5012,
+      "step": 550
+    },
+    {
+      "epoch": 0.04981373992896128,
+      "grad_norm": 14.394525527954102,
+      "learning_rate": 1.490909090909091e-05,
+      "loss": 0.5765,
+      "step": 575
+    },
+    {
+      "epoch": 0.05197955470848133,
+      "grad_norm": 16.823543548583984,
+      "learning_rate": 1.555844155844156e-05,
+      "loss": 0.4286,
+      "step": 600
+    },
+    {
+      "epoch": 0.05414536948800139,
+      "grad_norm": 4.72399377822876,
+      "learning_rate": 1.6207792207792207e-05,
+      "loss": 0.5167,
+      "step": 625
+    },
+    {
+      "epoch": 0.05631118426752144,
+      "grad_norm": 18.0063419342041,
+      "learning_rate": 1.6857142857142858e-05,
+      "loss": 0.4791,
+      "step": 650
+    },
+    {
+      "epoch": 0.0584769990470415,
+      "grad_norm": 11.925456047058105,
+      "learning_rate": 1.750649350649351e-05,
+      "loss": 0.5022,
+      "step": 675
+    },
+    {
+      "epoch": 0.06064281382656155,
+      "grad_norm": 2.7437996864318848,
+      "learning_rate": 1.8155844155844156e-05,
+      "loss": 0.438,
+      "step": 700
+    },
+    {
+      "epoch": 0.0628086286060816,
+      "grad_norm": 1.8270901441574097,
+      "learning_rate": 1.8805194805194806e-05,
+      "loss": 0.3995,
+      "step": 725
+    },
+    {
+      "epoch": 0.06497444338560167,
+      "grad_norm": 4.187374591827393,
+      "learning_rate": 1.9454545454545453e-05,
+      "loss": 0.2924,
+      "step": 750
+    },
+    {
+      "epoch": 0.06714025816512172,
+      "grad_norm": 12.709814071655273,
+      "learning_rate": 2.0103896103896104e-05,
+      "loss": 0.4391,
+      "step": 775
+    },
+    {
+      "epoch": 0.06930607294464178,
+      "grad_norm": 8.789942741394043,
+      "learning_rate": 2.0753246753246755e-05,
+      "loss": 0.4328,
+      "step": 800
+    },
+    {
+      "epoch": 0.07147188772416183,
+      "grad_norm": 10.182008743286133,
+      "learning_rate": 2.137662337662338e-05,
+      "loss": 0.5658,
+      "step": 825
+    },
+    {
+      "epoch": 0.07363770250368189,
+      "grad_norm": 3.5178301334381104,
+      "learning_rate": 2.2025974025974026e-05,
+      "loss": 0.4541,
+      "step": 850
+    },
+    {
+      "epoch": 0.07580351728320193,
+      "grad_norm": 8.124090194702148,
+      "learning_rate": 2.2675324675324676e-05,
+      "loss": 0.5381,
+      "step": 875
+    },
+    {
+      "epoch": 0.077969332062722,
+      "grad_norm": 11.69704532623291,
+      "learning_rate": 2.3324675324675324e-05,
+      "loss": 0.4523,
+      "step": 900
+    },
+    {
+      "epoch": 0.08013514684224206,
+      "grad_norm": 19.822145462036133,
+      "learning_rate": 2.3974025974025974e-05,
+      "loss": 0.3522,
+      "step": 925
+    },
+    {
+      "epoch": 0.0823009616217621,
+      "grad_norm": 8.31993579864502,
+      "learning_rate": 2.4623376623376625e-05,
+      "loss": 0.4475,
+      "step": 950
+    },
+    {
+      "epoch": 0.08446677640128217,
+      "grad_norm": 5.60876989364624,
+      "learning_rate": 2.5246753246753246e-05,
+      "loss": 0.4448,
+      "step": 975
+    },
+    {
+      "epoch": 0.08663259118080222,
+      "grad_norm": 9.872743606567383,
+      "learning_rate": 2.5896103896103896e-05,
+      "loss": 0.407,
+      "step": 1000
+    },
+    {
+      "epoch": 0.08879840596032228,
+      "grad_norm": 7.193666458129883,
+      "learning_rate": 2.6545454545454547e-05,
+      "loss": 0.4616,
+      "step": 1025
+    },
+    {
+      "epoch": 0.09096422073984232,
+      "grad_norm": 17.595991134643555,
+      "learning_rate": 2.7194805194805194e-05,
+      "loss": 0.4213,
+      "step": 1050
+    },
+    {
+      "epoch": 0.09313003551936239,
+      "grad_norm": 3.281184196472168,
+      "learning_rate": 2.7844155844155844e-05,
+      "loss": 0.465,
+      "step": 1075
+    },
+    {
+      "epoch": 0.09529585029888243,
+      "grad_norm": 7.671459197998047,
+      "learning_rate": 2.849350649350649e-05,
+      "loss": 0.2964,
+      "step": 1100
+    },
+    {
+      "epoch": 0.0974616650784025,
+      "grad_norm": 7.963995933532715,
+      "learning_rate": 2.9142857142857142e-05,
+      "loss": 0.4414,
+      "step": 1125
+    },
+    {
+      "epoch": 0.09962747985792256,
+      "grad_norm": 1.8723474740982056,
+      "learning_rate": 2.9792207792207793e-05,
+      "loss": 0.3508,
+      "step": 1150
+    },
+    {
+      "epoch": 0.1017932946374426,
+      "grad_norm": 5.1907877922058105,
+      "learning_rate": 2.995090489025799e-05,
+      "loss": 0.3362,
+      "step": 1175
+    },
+    {
+      "epoch": 0.10395910941696267,
+      "grad_norm": 5.219175815582275,
+      "learning_rate": 2.9878706199460916e-05,
+      "loss": 0.4953,
+      "step": 1200
+    },
+    {
+      "epoch": 0.10612492419648271,
+      "grad_norm": 15.204286575317383,
+      "learning_rate": 2.9806507508663843e-05,
+      "loss": 0.4041,
+      "step": 1225
+    },
+    {
+      "epoch": 0.10829073897600278,
+      "grad_norm": 5.872297286987305,
+      "learning_rate": 2.973430881786677e-05,
+      "loss": 0.3773,
+      "step": 1250
+    },
+    {
+      "epoch": 0.11045655375552282,
+      "grad_norm": 7.201790809631348,
+      "learning_rate": 2.9662110127069697e-05,
+      "loss": 0.3574,
+      "step": 1275
+    },
+    {
+      "epoch": 0.11262236853504289,
+      "grad_norm": 2.872793674468994,
+      "learning_rate": 2.9589911436272623e-05,
+      "loss": 0.642,
+      "step": 1300
+    },
+    {
+      "epoch": 0.11478818331456293,
+      "grad_norm": 10.854488372802734,
+      "learning_rate": 2.951771274547555e-05,
+      "loss": 0.3783,
+      "step": 1325
+    },
+    {
+      "epoch": 0.116953998094083,
+      "grad_norm": 2.162464141845703,
+      "learning_rate": 2.9445514054678477e-05,
+      "loss": 0.4905,
+      "step": 1350
+    },
+    {
+      "epoch": 0.11911981287360304,
+      "grad_norm": 14.541825294494629,
+      "learning_rate": 2.9373315363881403e-05,
+      "loss": 0.3937,
+      "step": 1375
+    },
+    {
+      "epoch": 0.1212856276531231,
+      "grad_norm": 1.6897481679916382,
+      "learning_rate": 2.9301116673084327e-05,
+      "loss": 0.4245,
+      "step": 1400
+    },
+    {
+      "epoch": 0.12345144243264317,
+      "grad_norm": 9.359882354736328,
+      "learning_rate": 2.9228917982287253e-05,
+      "loss": 0.4139,
+      "step": 1425
+    },
+    {
+      "epoch": 0.1256172572121632,
+      "grad_norm": 39.94605255126953,
+      "learning_rate": 2.915671929149018e-05,
+      "loss": 0.4305,
+      "step": 1450
+    },
+    {
+      "epoch": 0.12778307199168326,
+      "grad_norm": 10.268132209777832,
+      "learning_rate": 2.908452060069311e-05,
+      "loss": 0.675,
+      "step": 1475
+    },
+    {
+      "epoch": 0.12994888677120334,
+      "grad_norm": 1.7209604978561401,
+      "learning_rate": 2.9012321909896037e-05,
+      "loss": 0.55,
+      "step": 1500
+    },
+    {
+      "epoch": 0.13211470155072338,
+      "grad_norm": 8.541482925415039,
+      "learning_rate": 2.894012321909896e-05,
+      "loss": 0.4033,
+      "step": 1525
+    },
+    {
+      "epoch": 0.13428051633024343,
+      "grad_norm": 10.4110107421875,
+      "learning_rate": 2.8867924528301887e-05,
+      "loss": 0.4167,
+      "step": 1550
+    },
+    {
+      "epoch": 0.13644633110976348,
+      "grad_norm": 10.823756217956543,
+      "learning_rate": 2.8795725837504814e-05,
+      "loss": 0.3814,
+      "step": 1575
+    },
+    {
+      "epoch": 0.13861214588928356,
+      "grad_norm": 0.6896539926528931,
+      "learning_rate": 2.872352714670774e-05,
+      "loss": 0.5183,
+      "step": 1600
+    },
+    {
+      "epoch": 0.1407779606688036,
+      "grad_norm": 4.357579231262207,
+      "learning_rate": 2.8651328455910667e-05,
+      "loss": 0.3343,
+      "step": 1625
+    },
+    {
+      "epoch": 0.14294377544832365,
+      "grad_norm": 12.074344635009766,
+      "learning_rate": 2.857912976511359e-05,
+      "loss": 0.4212,
+      "step": 1650
+    },
+    {
+      "epoch": 0.14510959022784373,
+      "grad_norm": 11.660531997680664,
+      "learning_rate": 2.850693107431652e-05,
+      "loss": 0.4737,
+      "step": 1675
+    },
+    {
+      "epoch": 0.14727540500736377,
+      "grad_norm": 15.467144966125488,
+      "learning_rate": 2.8434732383519447e-05,
+      "loss": 0.4563,
+      "step": 1700
+    },
+    {
+      "epoch": 0.14944121978688382,
+      "grad_norm": 9.277994155883789,
+      "learning_rate": 2.8362533692722374e-05,
+      "loss": 0.4251,
+      "step": 1725
+    },
+    {
+      "epoch": 0.15160703456640387,
+      "grad_norm": 3.6043941974639893,
+      "learning_rate": 2.82903350019253e-05,
+      "loss": 0.3497,
+      "step": 1750
+    },
+    {
+      "epoch": 0.15377284934592395,
+      "grad_norm": 3.933353900909424,
+      "learning_rate": 2.8218136311128224e-05,
+      "loss": 0.3753,
+      "step": 1775
+    },
+    {
+      "epoch": 0.155938664125444,
+      "grad_norm": 3.8728222846984863,
+      "learning_rate": 2.814593762033115e-05,
+      "loss": 0.4031,
+      "step": 1800
+    },
+    {
+      "epoch": 0.15810447890496404,
+      "grad_norm": 8.067976951599121,
+      "learning_rate": 2.8073738929534077e-05,
+      "loss": 0.4037,
+      "step": 1825
+    },
+    {
+      "epoch": 0.16027029368448412,
+      "grad_norm": 9.141134262084961,
+      "learning_rate": 2.8001540238737004e-05,
+      "loss": 0.4114,
+      "step": 1850
+    },
+    {
+      "epoch": 0.16243610846400416,
+      "grad_norm": 1.8272747993469238,
+      "learning_rate": 2.7929341547939934e-05,
+      "loss": 0.3848,
+      "step": 1875
+    },
+    {
+      "epoch": 0.1646019232435242,
+      "grad_norm": 0.4890976846218109,
+      "learning_rate": 2.7857142857142858e-05,
+      "loss": 0.5088,
+      "step": 1900
+    },
+    {
+      "epoch": 0.16676773802304426,
+      "grad_norm": 9.043623924255371,
+      "learning_rate": 2.7784944166345784e-05,
+      "loss": 0.4032,
+      "step": 1925
+    },
+    {
+      "epoch": 0.16893355280256434,
+      "grad_norm": 9.092608451843262,
+      "learning_rate": 2.771274547554871e-05,
+      "loss": 0.3354,
+      "step": 1950
+    },
+    {
+      "epoch": 0.17109936758208438,
+      "grad_norm": 6.121222972869873,
+      "learning_rate": 2.7640546784751638e-05,
+      "loss": 0.4163,
+      "step": 1975
+    },
+    {
+      "epoch": 0.17326518236160443,
+      "grad_norm": 1.539663314819336,
+      "learning_rate": 2.7568348093954564e-05,
+      "loss": 0.3715,
+      "step": 2000
+    },
+    {
+      "epoch": 0.17543099714112448,
+      "grad_norm": 16.089406967163086,
+      "learning_rate": 2.7496149403157488e-05,
+      "loss": 0.3424,
+      "step": 2025
+    },
+    {
+      "epoch": 0.17759681192064455,
+      "grad_norm": 12.510934829711914,
+      "learning_rate": 2.7423950712360414e-05,
+      "loss": 0.3311,
+      "step": 2050
+    },
+    {
+      "epoch": 0.1797626267001646,
+      "grad_norm": 2.823338508605957,
+      "learning_rate": 2.7351752021563345e-05,
+      "loss": 0.4362,
+      "step": 2075
+    },
+    {
+      "epoch": 0.18192844147968465,
+      "grad_norm": 6.191600322723389,
+      "learning_rate": 2.727955333076627e-05,
+      "loss": 0.4441,
+      "step": 2100
+    },
+    {
+      "epoch": 0.18409425625920472,
+      "grad_norm": 4.86907434463501,
+      "learning_rate": 2.7207354639969198e-05,
+      "loss": 0.3122,
+      "step": 2125
+    },
+    {
+      "epoch": 0.18626007103872477,
+      "grad_norm": 7.323814868927002,
+      "learning_rate": 2.713515594917212e-05,
+      "loss": 0.3717,
+      "step": 2150
+    },
+    {
+      "epoch": 0.18842588581824482,
+      "grad_norm": 10.09737491607666,
+      "learning_rate": 2.7062957258375048e-05,
+      "loss": 0.3461,
+      "step": 2175
+    },
+    {
+      "epoch": 0.19059170059776487,
+      "grad_norm": 8.536800384521484,
+      "learning_rate": 2.6990758567577975e-05,
+      "loss": 0.4816,
+      "step": 2200
+    },
+    {
+      "epoch": 0.19275751537728494,
+      "grad_norm": 5.237682819366455,
+      "learning_rate": 2.69185598767809e-05,
+      "loss": 0.4784,
+      "step": 2225
+    },
+    {
+      "epoch": 0.194923330156805,
+      "grad_norm": 10.763497352600098,
+      "learning_rate": 2.6846361185983828e-05,
+      "loss": 0.4334,
+      "step": 2250
+    },
+    {
+      "epoch": 0.19708914493632504,
+      "grad_norm": 0.7019050121307373,
+      "learning_rate": 2.6774162495186755e-05,
+      "loss": 0.3437,
+      "step": 2275
+    },
+    {
+      "epoch": 0.19925495971584511,
+      "grad_norm": 8.020634651184082,
+      "learning_rate": 2.670196380438968e-05,
+      "loss": 0.4333,
+      "step": 2300
+    },
+    {
+      "epoch": 0.20142077449536516,
+      "grad_norm": 10.549779891967773,
+      "learning_rate": 2.662976511359261e-05,
+      "loss": 0.3609,
+      "step": 2325
+    },
+    {
+      "epoch": 0.2035865892748852,
+      "grad_norm": 5.6236677169799805,
+      "learning_rate": 2.6557566422795535e-05,
+      "loss": 0.3437,
+      "step": 2350
+    },
+    {
+      "epoch": 0.20575240405440526,
+      "grad_norm": 1.4388600587844849,
+      "learning_rate": 2.648536773199846e-05,
+      "loss": 0.4911,
+      "step": 2375
+    },
+    {
+      "epoch": 0.20791821883392533,
+      "grad_norm": 4.445183277130127,
+      "learning_rate": 2.6413169041201385e-05,
+      "loss": 0.3872,
+      "step": 2400
+    },
+    {
+      "epoch": 0.21008403361344538,
+      "grad_norm": 9.076152801513672,
+      "learning_rate": 2.6340970350404312e-05,
+      "loss": 0.276,
+      "step": 2425
+    },
+    {
+      "epoch": 0.21224984839296543,
+      "grad_norm": 5.573355197906494,
+      "learning_rate": 2.6268771659607242e-05,
+      "loss": 0.3318,
+      "step": 2450
+    },
+    {
+      "epoch": 0.21441566317248548,
+      "grad_norm": 5.015573024749756,
+      "learning_rate": 2.619657296881017e-05,
+      "loss": 0.4833,
+      "step": 2475
+    },
+    {
+      "epoch": 0.21658147795200555,
+      "grad_norm": 3.9038755893707275,
+      "learning_rate": 2.6124374278013092e-05,
+      "loss": 0.4656,
+      "step": 2500
+    },
+    {
+      "epoch": 0.2187472927315256,
+      "grad_norm": 2.66627836227417,
+      "learning_rate": 2.605217558721602e-05,
+      "loss": 0.4232,
+      "step": 2525
+    },
+    {
+      "epoch": 0.22091310751104565,
+      "grad_norm": 8.859906196594238,
+      "learning_rate": 2.5979976896418945e-05,
+      "loss": 0.434,
+      "step": 2550
+    },
+    {
+      "epoch": 0.22307892229056572,
+      "grad_norm": 3.2811522483825684,
+      "learning_rate": 2.5907778205621872e-05,
+      "loss": 0.2479,
+      "step": 2575
+    },
+    {
+      "epoch": 0.22524473707008577,
+      "grad_norm": 8.53447437286377,
+      "learning_rate": 2.58355795148248e-05,
+      "loss": 0.4656,
+      "step": 2600
+    },
+    {
+      "epoch": 0.22741055184960582,
+      "grad_norm": 6.359921455383301,
+      "learning_rate": 2.5763380824027722e-05,
+      "loss": 0.3881,
+      "step": 2625
+    },
+    {
+      "epoch": 0.22957636662912587,
+      "grad_norm": 6.196253776550293,
+      "learning_rate": 2.5691182133230652e-05,
+      "loss": 0.3637,
+      "step": 2650
+    },
+    {
+      "epoch": 0.23174218140864594,
+      "grad_norm": 7.805304050445557,
+      "learning_rate": 2.561898344243358e-05,
+      "loss": 0.3099,
+      "step": 2675
+    },
+    {
+      "epoch": 0.233907996188166,
+      "grad_norm": 4.51755428314209,
+      "learning_rate": 2.5546784751636506e-05,
+      "loss": 0.3933,
+      "step": 2700
+    },
+    {
+      "epoch": 0.23607381096768604,
+      "grad_norm": 5.72914981842041,
+      "learning_rate": 2.5474586060839432e-05,
+      "loss": 0.3789,
+      "step": 2725
+    },
+    {
+      "epoch": 0.23823962574720609,
+      "grad_norm": 2.4809954166412354,
+      "learning_rate": 2.5402387370042356e-05,
+      "loss": 0.4056,
+      "step": 2750
+    },
+    {
+      "epoch": 0.24040544052672616,
+      "grad_norm": 1.940656065940857,
+      "learning_rate": 2.5330188679245282e-05,
+      "loss": 0.4132,
+      "step": 2775
+    },
+    {
+      "epoch": 0.2425712553062462,
+      "grad_norm": 3.452242851257324,
+      "learning_rate": 2.525798998844821e-05,
+      "loss": 0.375,
+      "step": 2800
+    },
+    {
+      "epoch": 0.24473707008576626,
+      "grad_norm": 9.220993041992188,
+      "learning_rate": 2.5185791297651136e-05,
+      "loss": 0.3026,
+      "step": 2825
+    },
+    {
+      "epoch": 0.24690288486528633,
+      "grad_norm": 10.027073860168457,
+      "learning_rate": 2.5113592606854066e-05,
+      "loss": 0.5372,
+      "step": 2850
+    },
+    {
+      "epoch": 0.24906869964480638,
+      "grad_norm": 2.228799819946289,
+      "learning_rate": 2.504139391605699e-05,
+      "loss": 0.4233,
+      "step": 2875
+    },
+    {
+      "epoch": 0.2512345144243264,
+      "grad_norm": 7.281198978424072,
+      "learning_rate": 2.4969195225259916e-05,
+      "loss": 0.2945,
+      "step": 2900
+    },
+    {
+      "epoch": 0.2534003292038465,
+      "grad_norm": 1.4160314798355103,
+      "learning_rate": 2.4896996534462843e-05,
+      "loss": 0.2916,
+      "step": 2925
+    },
+    {
+      "epoch": 0.2555661439833665,
+      "grad_norm": 4.095098972320557,
+      "learning_rate": 2.482479784366577e-05,
+      "loss": 0.3536,
+      "step": 2950
+    },
+    {
+      "epoch": 0.25773195876288657,
+      "grad_norm": 1.413552165031433,
+      "learning_rate": 2.4752599152868696e-05,
+      "loss": 0.3246,
+      "step": 2975
+    },
+    {
+      "epoch": 0.2598977735424067,
+      "grad_norm": 3.3196184635162354,
+      "learning_rate": 2.468040046207162e-05,
+      "loss": 0.4236,
+      "step": 3000
+    },
+    {
+      "epoch": 0.2620635883219267,
+      "grad_norm": 11.855537414550781,
+      "learning_rate": 2.4608201771274546e-05,
+      "loss": 0.4088,
+      "step": 3025
+    },
+    {
+      "epoch": 0.26422940310144677,
+      "grad_norm": 9.322809219360352,
+      "learning_rate": 2.4536003080477476e-05,
+      "loss": 0.4522,
+      "step": 3050
+    },
+    {
+      "epoch": 0.2663952178809668,
+      "grad_norm": 7.581571578979492,
+      "learning_rate": 2.4463804389680403e-05,
+      "loss": 0.3445,
+      "step": 3075
+    },
+    {
+      "epoch": 0.26856103266048686,
+      "grad_norm": 2.6131093502044678,
+      "learning_rate": 2.439160569888333e-05,
+      "loss": 0.3575,
+      "step": 3100
+    },
+    {
+      "epoch": 0.2707268474400069,
+      "grad_norm": 3.68662166595459,
+      "learning_rate": 2.4319407008086253e-05,
+      "loss": 0.3809,
+      "step": 3125
+    },
+    {
+      "epoch": 0.27289266221952696,
+      "grad_norm": 2.3688032627105713,
+      "learning_rate": 2.424720831728918e-05,
+      "loss": 0.3364,
+      "step": 3150
+    },
+    {
+      "epoch": 0.27505847699904706,
+      "grad_norm": 1.155315637588501,
+      "learning_rate": 2.4175009626492106e-05,
+      "loss": 0.4103,
+      "step": 3175
+    },
+    {
+      "epoch": 0.2772242917785671,
+      "grad_norm": 35.7138671875,
+      "learning_rate": 2.4102810935695033e-05,
+      "loss": 0.3502,
+      "step": 3200
+    },
+    {
+      "epoch": 0.27939010655808716,
+      "grad_norm": 6.429433822631836,
+      "learning_rate": 2.403061224489796e-05,
+      "loss": 0.2632,
+      "step": 3225
+    },
+    {
+      "epoch": 0.2815559213376072,
+      "grad_norm": 9.816515922546387,
+      "learning_rate": 2.3958413554100887e-05,
+      "loss": 0.406,
+      "step": 3250
+    },
+    {
+      "epoch": 0.28372173611712725,
+      "grad_norm": 1.9653140306472778,
+      "learning_rate": 2.3886214863303813e-05,
+      "loss": 0.4363,
+      "step": 3275
+    },
+    {
+      "epoch": 0.2858875508966473,
+      "grad_norm": 9.559599876403809,
+      "learning_rate": 2.381401617250674e-05,
+      "loss": 0.2819,
+      "step": 3300
+    },
+    {
+      "epoch": 0.28805336567616735,
+      "grad_norm": 10.623549461364746,
+      "learning_rate": 2.3741817481709667e-05,
+      "loss": 0.3421,
+      "step": 3325
+    },
+    {
+      "epoch": 0.29021918045568745,
+      "grad_norm": 2.4988913536071777,
+      "learning_rate": 2.366961879091259e-05,
+      "loss": 0.269,
+      "step": 3350
+    },
+    {
+      "epoch": 0.2923849952352075,
+      "grad_norm": 4.704137802124023,
+      "learning_rate": 2.3597420100115517e-05,
+      "loss": 0.2902,
+      "step": 3375
+    },
+    {
+      "epoch": 0.29455081001472755,
+      "grad_norm": 9.48901653289795,
+      "learning_rate": 2.3525221409318443e-05,
+      "loss": 0.3548,
+      "step": 3400
+    },
+    {
+      "epoch": 0.2967166247942476,
+      "grad_norm": 0.5201269388198853,
+      "learning_rate": 2.3453022718521374e-05,
+      "loss": 0.4575,
+      "step": 3425
+    },
+    {
+      "epoch": 0.29888243957376764,
+      "grad_norm": 8.074861526489258,
+      "learning_rate": 2.33808240277243e-05,
+      "loss": 0.3942,
+      "step": 3450
+    },
+    {
+      "epoch": 0.3010482543532877,
+      "grad_norm": 8.45334243774414,
+      "learning_rate": 2.3308625336927224e-05,
+      "loss": 0.3537,
+      "step": 3475
+    },
+    {
+      "epoch": 0.30321406913280774,
+      "grad_norm": 2.7069313526153564,
+      "learning_rate": 2.323642664613015e-05,
+      "loss": 0.3672,
+      "step": 3500
+    },
+    {
+      "epoch": 0.30537988391232784,
+      "grad_norm": 13.849508285522461,
+      "learning_rate": 2.3164227955333077e-05,
+      "loss": 0.3502,
+      "step": 3525
+    },
+    {
+      "epoch": 0.3075456986918479,
+      "grad_norm": 4.5892462730407715,
+      "learning_rate": 2.3092029264536004e-05,
+      "loss": 0.2545,
+      "step": 3550
+    },
+    {
+      "epoch": 0.30971151347136794,
+      "grad_norm": 1.035447120666504,
+      "learning_rate": 2.301983057373893e-05,
+      "loss": 0.2544,
+      "step": 3575
+    },
+    {
+      "epoch": 0.311877328250888,
+      "grad_norm": 5.170057773590088,
+      "learning_rate": 2.2947631882941854e-05,
+      "loss": 0.3443,
+      "step": 3600
+    },
+    {
+      "epoch": 0.31404314303040803,
+      "grad_norm": 2.908191204071045,
+      "learning_rate": 2.2875433192144784e-05,
+      "loss": 0.3784,
+      "step": 3625
+    },
+    {
+      "epoch": 0.3162089578099281,
+      "grad_norm": 9.946891784667969,
+      "learning_rate": 2.280323450134771e-05,
+      "loss": 0.3828,
+      "step": 3650
+    },
+    {
+      "epoch": 0.31837477258944813,
+      "grad_norm": 10.337167739868164,
+      "learning_rate": 2.2731035810550637e-05,
+      "loss": 0.4032,
+      "step": 3675
+    },
+    {
+      "epoch": 0.32054058736896823,
+      "grad_norm": 10.093758583068848,
+      "learning_rate": 2.2658837119753564e-05,
+      "loss": 0.2556,
+      "step": 3700
+    },
+    {
+      "epoch": 0.3227064021484883,
+      "grad_norm": 7.309471130371094,
+      "learning_rate": 2.2586638428956487e-05,
+      "loss": 0.3352,
+      "step": 3725
+    },
+    {
+      "epoch": 0.32487221692800833,
+      "grad_norm": 10.050370216369629,
+      "learning_rate": 2.2514439738159414e-05,
+      "loss": 0.4054,
+      "step": 3750
+    },
+    {
+      "epoch": 0.3270380317075284,
+      "grad_norm": 3.858546733856201,
+      "learning_rate": 2.244224104736234e-05,
+      "loss": 0.3049,
+      "step": 3775
+    },
+    {
+      "epoch": 0.3292038464870484,
+      "grad_norm": 5.640537261962891,
+      "learning_rate": 2.2370042356565267e-05,
+      "loss": 0.2223,
+      "step": 3800
+    },
+    {
+      "epoch": 0.33136966126656847,
+      "grad_norm": 5.106541633605957,
+      "learning_rate": 2.2297843665768198e-05,
+      "loss": 0.4878,
+      "step": 3825
+    },
+    {
+      "epoch": 0.3335354760460885,
+      "grad_norm": 7.738224029541016,
+      "learning_rate": 2.222564497497112e-05,
+      "loss": 0.3015,
+      "step": 3850
+    },
+    {
+      "epoch": 0.33570129082560857,
+      "grad_norm": 12.313666343688965,
+      "learning_rate": 2.2153446284174048e-05,
+      "loss": 0.3816,
+      "step": 3875
+    },
+    {
+      "epoch": 0.33786710560512867,
+      "grad_norm": 0.9929437041282654,
+      "learning_rate": 2.2081247593376974e-05,
+      "loss": 0.3334,
+      "step": 3900
+    },
+    {
+      "epoch": 0.3400329203846487,
+      "grad_norm": 5.753032207489014,
+      "learning_rate": 2.20090489025799e-05,
+      "loss": 0.3724,
+      "step": 3925
+    },
+    {
+      "epoch": 0.34219873516416877,
+      "grad_norm": 8.37396240234375,
+      "learning_rate": 2.1936850211782828e-05,
+      "loss": 0.4217,
+      "step": 3950
+    },
+    {
+      "epoch": 0.3443645499436888,
+      "grad_norm": 7.365005016326904,
+      "learning_rate": 2.186465152098575e-05,
+      "loss": 0.4339,
+      "step": 3975
+    },
+    {
+      "epoch": 0.34653036472320886,
+      "grad_norm": 1.91083824634552,
+      "learning_rate": 2.1792452830188678e-05,
+      "loss": 0.3642,
+      "step": 4000
+    },
+    {
+      "epoch": 0.3486961795027289,
+      "grad_norm": 3.0427494049072266,
+      "learning_rate": 2.1720254139391608e-05,
+      "loss": 0.3819,
+      "step": 4025
+    },
+    {
+      "epoch": 0.35086199428224896,
+      "grad_norm": 1.176952838897705,
+      "learning_rate": 2.1648055448594535e-05,
+      "loss": 0.2796,
+      "step": 4050
+    },
+    {
+      "epoch": 0.35302780906176906,
+      "grad_norm": 1.0579583644866943,
+      "learning_rate": 2.157585675779746e-05,
+      "loss": 0.4277,
+      "step": 4075
+    },
+    {
+      "epoch": 0.3551936238412891,
+      "grad_norm": 11.798035621643066,
+      "learning_rate": 2.1503658067000385e-05,
+      "loss": 0.3407,
+      "step": 4100
+    },
+    {
+      "epoch": 0.35735943862080916,
+      "grad_norm": 15.57787036895752,
+      "learning_rate": 2.143145937620331e-05,
+      "loss": 0.2781,
+      "step": 4125
+    },
+    {
+      "epoch": 0.3595252534003292,
+      "grad_norm": 8.533368110656738,
+      "learning_rate": 2.1359260685406238e-05,
+      "loss": 0.4274,
+      "step": 4150
+    },
+    {
+      "epoch": 0.36169106817984925,
+      "grad_norm": 8.470250129699707,
+      "learning_rate": 2.1287061994609165e-05,
+      "loss": 0.3609,
+      "step": 4175
+    },
+    {
+      "epoch": 0.3638568829593693,
+      "grad_norm": 6.417985439300537,
+      "learning_rate": 2.121486330381209e-05,
+      "loss": 0.3476,
+      "step": 4200
+    },
+    {
+      "epoch": 0.36602269773888935,
+      "grad_norm": 8.685192108154297,
+      "learning_rate": 2.1142664613015018e-05,
+      "loss": 0.41,
+      "step": 4225
+    },
+    {
+      "epoch": 0.36818851251840945,
+      "grad_norm": 7.082727432250977,
+      "learning_rate": 2.1070465922217945e-05,
+      "loss": 0.4003,
+      "step": 4250
+    },
+    {
+      "epoch": 0.3703543272979295,
+      "grad_norm": 4.621776103973389,
+      "learning_rate": 2.099826723142087e-05,
+      "loss": 0.306,
+      "step": 4275
+    },
+    {
+      "epoch": 0.37252014207744955,
+      "grad_norm": 3.1071817874908447,
+      "learning_rate": 2.09260685406238e-05,
+      "loss": 0.2335,
+      "step": 4300
+    },
+    {
+      "epoch": 0.3746859568569696,
+      "grad_norm": 7.23638916015625,
+      "learning_rate": 2.085386984982672e-05,
+      "loss": 0.2733,
+      "step": 4325
+    },
+    {
+      "epoch": 0.37685177163648964,
+      "grad_norm": 6.893523693084717,
+      "learning_rate": 2.078167115902965e-05,
+      "loss": 0.3007,
+      "step": 4350
+    },
+    {
+      "epoch": 0.3790175864160097,
+      "grad_norm": 5.9917073249816895,
+      "learning_rate": 2.0709472468232575e-05,
+      "loss": 0.3086,
+      "step": 4375
+    },
+    {
+      "epoch": 0.38118340119552974,
+      "grad_norm": 6.596795558929443,
+      "learning_rate": 2.0637273777435502e-05,
+      "loss": 0.365,
+      "step": 4400
+    },
+    {
+      "epoch": 0.38334921597504984,
+      "grad_norm": 9.045963287353516,
+      "learning_rate": 2.0565075086638432e-05,
+      "loss": 0.3255,
+      "step": 4425
+    },
+    {
+      "epoch": 0.3855150307545699,
+      "grad_norm": 6.755446434020996,
+      "learning_rate": 2.0492876395841355e-05,
+      "loss": 0.3765,
+      "step": 4450
+    },
+    {
+      "epoch": 0.38768084553408994,
+      "grad_norm": 11.626537322998047,
+      "learning_rate": 2.0420677705044282e-05,
+      "loss": 0.2946,
+      "step": 4475
+    },
+    {
+      "epoch": 0.38984666031361,
+      "grad_norm": 4.125662326812744,
+      "learning_rate": 2.034847901424721e-05,
+      "loss": 0.3298,
+      "step": 4500
+    },
+    {
+      "epoch": 0.39201247509313003,
+      "grad_norm": 1.2437127828598022,
+      "learning_rate": 2.0276280323450135e-05,
+      "loss": 0.3645,
+      "step": 4525
+    },
+    {
+      "epoch": 0.3941782898726501,
+      "grad_norm": 10.272943496704102,
+      "learning_rate": 2.0204081632653062e-05,
+      "loss": 0.2403,
+      "step": 4550
+    },
+    {
+      "epoch": 0.3963441046521701,
+      "grad_norm": 2.164606809616089,
+      "learning_rate": 2.0131882941855985e-05,
+      "loss": 0.28,
+      "step": 4575
+    },
+    {
+      "epoch": 0.39850991943169023,
+      "grad_norm": 9.157061576843262,
+      "learning_rate": 2.0059684251058916e-05,
+      "loss": 0.3814,
+      "step": 4600
+    },
+    {
+      "epoch": 0.4006757342112103,
+      "grad_norm": 4.034579277038574,
+      "learning_rate": 1.9987485560261842e-05,
+      "loss": 0.3419,
+      "step": 4625
+    },
+    {
+      "epoch": 0.4028415489907303,
+      "grad_norm": 2.5503344535827637,
+      "learning_rate": 1.991528686946477e-05,
+      "loss": 0.3374,
+      "step": 4650
+    },
+    {
+      "epoch": 0.4050073637702504,
+      "grad_norm": 4.660188674926758,
+      "learning_rate": 1.9843088178667696e-05,
+      "loss": 0.3511,
+      "step": 4675
+    },
+    {
+      "epoch": 0.4071731785497704,
+      "grad_norm": 7.020951747894287,
+      "learning_rate": 1.977088948787062e-05,
+      "loss": 0.4339,
+      "step": 4700
+    },
+    {
+      "epoch": 0.40933899332929047,
+      "grad_norm": 5.4507269859313965,
+      "learning_rate": 1.9698690797073546e-05,
+      "loss": 0.3441,
+      "step": 4725
+    },
+    {
+      "epoch": 0.4115048081088105,
+      "grad_norm": 11.266243934631348,
+      "learning_rate": 1.9626492106276472e-05,
+      "loss": 0.346,
+      "step": 4750
+    },
+    {
+      "epoch": 0.41367062288833056,
+      "grad_norm": 2.1191511154174805,
+      "learning_rate": 1.95542934154794e-05,
+      "loss": 0.3723,
+      "step": 4775
+    },
+    {
+      "epoch": 0.41583643766785067,
+      "grad_norm": 1.9068052768707275,
+      "learning_rate": 1.948209472468233e-05,
+      "loss": 0.2075,
+      "step": 4800
+    },
+    {
+      "epoch": 0.4180022524473707,
+      "grad_norm": 0.36394304037094116,
+      "learning_rate": 1.9409896033885253e-05,
+      "loss": 0.2431,
+      "step": 4825
+    },
+    {
+      "epoch": 0.42016806722689076,
+      "grad_norm": 6.177628993988037,
+      "learning_rate": 1.933769734308818e-05,
+      "loss": 0.2642,
+      "step": 4850
+    },
+    {
+      "epoch": 0.4223338820064108,
+      "grad_norm": 3.6669273376464844,
+      "learning_rate": 1.9265498652291106e-05,
+      "loss": 0.1763,
+      "step": 4875
+    },
+    {
+      "epoch": 0.42449969678593086,
+      "grad_norm": 7.836557865142822,
+      "learning_rate": 1.9193299961494033e-05,
+      "loss": 0.3862,
+      "step": 4900
+    },
+    {
+      "epoch": 0.4266655115654509,
+      "grad_norm": 10.140486717224121,
+      "learning_rate": 1.912110127069696e-05,
+      "loss": 0.3053,
+      "step": 4925
+    },
+    {
+      "epoch": 0.42883132634497095,
+      "grad_norm": 2.8873586654663086,
+      "learning_rate": 1.9048902579899883e-05,
+      "loss": 0.3162,
+      "step": 4950
+    },
+    {
+      "epoch": 0.43099714112449106,
+      "grad_norm": 1.758466362953186,
+      "learning_rate": 1.897670388910281e-05,
+      "loss": 0.3178,
+      "step": 4975
+    },
+    {
+      "epoch": 0.4331629559040111,
+      "grad_norm": 7.523651599884033,
+      "learning_rate": 1.890450519830574e-05,
+      "loss": 0.2789,
+      "step": 5000
+    },
+    {
+      "epoch": 0.43532877068353115,
+      "grad_norm": 5.955496311187744,
+      "learning_rate": 1.8832306507508666e-05,
+      "loss": 0.1777,
+      "step": 5025
+    },
+    {
+      "epoch": 0.4374945854630512,
+      "grad_norm": 9.068547248840332,
+      "learning_rate": 1.8760107816711593e-05,
+      "loss": 0.4155,
+      "step": 5050
+    },
+    {
+      "epoch": 0.43966040024257125,
+      "grad_norm": 4.900373458862305,
+      "learning_rate": 1.8687909125914516e-05,
+      "loss": 0.2983,
+      "step": 5075
+    },
+    {
+      "epoch": 0.4418262150220913,
+      "grad_norm": 3.5501790046691895,
+      "learning_rate": 1.8615710435117443e-05,
+      "loss": 0.3687,
+      "step": 5100
+    },
+    {
+      "epoch": 0.44399202980161134,
+      "grad_norm": 1.0216822624206543,
+      "learning_rate": 1.854351174432037e-05,
+      "loss": 0.2428,
+      "step": 5125
+    },
+    {
+      "epoch": 0.44615784458113145,
+      "grad_norm": 7.637403964996338,
+      "learning_rate": 1.8471313053523296e-05,
+      "loss": 0.3071,
+      "step": 5150
+    },
+    {
+      "epoch": 0.4483236593606515,
+      "grad_norm": 9.478981018066406,
+      "learning_rate": 1.8399114362726223e-05,
+      "loss": 0.2911,
+      "step": 5175
+    },
+    {
+      "epoch": 0.45048947414017154,
+      "grad_norm": 3.875411033630371,
+      "learning_rate": 1.832691567192915e-05,
+      "loss": 0.3152,
+      "step": 5200
+    },
+    {
+      "epoch": 0.4526552889196916,
+      "grad_norm": 1.1700037717819214,
+      "learning_rate": 1.8254716981132077e-05,
+      "loss": 0.2776,
+      "step": 5225
+    },
+    {
+      "epoch": 0.45482110369921164,
+      "grad_norm": 4.037864685058594,
+      "learning_rate": 1.8182518290335003e-05,
+      "loss": 0.2674,
+      "step": 5250
+    },
+    {
+      "epoch": 0.4569869184787317,
+      "grad_norm": 2.6295673847198486,
+      "learning_rate": 1.811031959953793e-05,
+      "loss": 0.3035,
+      "step": 5275
+    },
+    {
+      "epoch": 0.45915273325825173,
+      "grad_norm": 9.654006004333496,
+      "learning_rate": 1.8038120908740853e-05,
+      "loss": 0.3352,
+      "step": 5300
+    },
+    {
+      "epoch": 0.46131854803777184,
+      "grad_norm": 7.339272975921631,
+      "learning_rate": 1.796592221794378e-05,
+      "loss": 0.3879,
+      "step": 5325
+    },
+    {
+      "epoch": 0.4634843628172919,
+      "grad_norm": 5.668703079223633,
+      "learning_rate": 1.7893723527146707e-05,
+      "loss": 0.3828,
+      "step": 5350
+    },
+    {
+      "epoch": 0.46565017759681193,
+      "grad_norm": 11.843222618103027,
+      "learning_rate": 1.7821524836349633e-05,
+      "loss": 0.2797,
+      "step": 5375
+    },
+    {
+      "epoch": 0.467815992376332,
+      "grad_norm": 3.3071844577789307,
+      "learning_rate": 1.7749326145552564e-05,
+      "loss": 0.3492,
+      "step": 5400
+    },
+    {
+      "epoch": 0.469981807155852,
+      "grad_norm": 11.303645133972168,
+      "learning_rate": 1.7677127454755487e-05,
+      "loss": 0.5,
+      "step": 5425
+    },
+    {
+      "epoch": 0.4721476219353721,
+      "grad_norm": 1.1275362968444824,
+      "learning_rate": 1.7604928763958414e-05,
+      "loss": 0.2317,
+      "step": 5450
+    },
+    {
+      "epoch": 0.4743134367148921,
+      "grad_norm": 11.97022533416748,
+      "learning_rate": 1.753273007316134e-05,
+      "loss": 0.2411,
+      "step": 5475
+    },
+    {
+      "epoch": 0.47647925149441217,
+      "grad_norm": 2.9647443294525146,
+      "learning_rate": 1.7460531382364267e-05,
+      "loss": 0.277,
+      "step": 5500
+    },
+    {
+      "epoch": 0.4786450662739323,
+      "grad_norm": 5.046292781829834,
+      "learning_rate": 1.7388332691567194e-05,
+      "loss": 0.4112,
+      "step": 5525
+    },
+    {
+      "epoch": 0.4808108810534523,
+      "grad_norm": 8.11351203918457,
+      "learning_rate": 1.7316134000770117e-05,
+      "loss": 0.5116,
+      "step": 5550
+    },
+    {
+      "epoch": 0.48297669583297237,
+      "grad_norm": 1.0861672163009644,
+      "learning_rate": 1.7243935309973047e-05,
+      "loss": 0.3264,
+      "step": 5575
+    },
+    {
+      "epoch": 0.4851425106124924,
+      "grad_norm": 2.311553955078125,
+      "learning_rate": 1.7171736619175974e-05,
+      "loss": 0.3688,
+      "step": 5600
+    },
+    {
+      "epoch": 0.48730832539201246,
+      "grad_norm": 2.371721029281616,
+      "learning_rate": 1.70995379283789e-05,
+      "loss": 0.3224,
+      "step": 5625
+    },
+    {
+      "epoch": 0.4894741401715325,
+      "grad_norm": 7.7612714767456055,
+      "learning_rate": 1.7027339237581827e-05,
+      "loss": 0.3778,
+      "step": 5650
+    },
+    {
+      "epoch": 0.49163995495105256,
+      "grad_norm": 7.416019916534424,
+      "learning_rate": 1.695514054678475e-05,
+      "loss": 0.3671,
+      "step": 5675
+    },
+    {
+      "epoch": 0.49380576973057266,
+      "grad_norm": 7.0320940017700195,
+      "learning_rate": 1.6882941855987677e-05,
+      "loss": 0.3331,
+      "step": 5700
+    },
+    {
+      "epoch": 0.4959715845100927,
+      "grad_norm": 0.8671308159828186,
+      "learning_rate": 1.6810743165190604e-05,
+      "loss": 0.3426,
+      "step": 5725
+    },
+    {
+      "epoch": 0.49813739928961276,
+      "grad_norm": 6.607793807983398,
+      "learning_rate": 1.673854447439353e-05,
+      "loss": 0.2863,
+      "step": 5750
+    },
+    {
+      "epoch": 0.5003032140691328,
+      "grad_norm": 10.399803161621094,
+      "learning_rate": 1.666634578359646e-05,
+      "loss": 0.5822,
+      "step": 5775
+    },
+    {
+      "epoch": 0.5024690288486529,
+      "grad_norm": 2.4261348247528076,
+      "learning_rate": 1.6594147092799384e-05,
+      "loss": 0.2687,
+      "step": 5800
+    },
+    {
+      "epoch": 0.5046348436281729,
+      "grad_norm": 0.30012401938438416,
+      "learning_rate": 1.652194840200231e-05,
+      "loss": 0.3365,
+      "step": 5825
+    },
+    {
+      "epoch": 0.506800658407693,
+      "grad_norm": 8.255668640136719,
+      "learning_rate": 1.6449749711205238e-05,
+      "loss": 0.4609,
+      "step": 5850
+    },
+    {
+      "epoch": 0.508966473187213,
+      "grad_norm": 6.495670795440674,
+      "learning_rate": 1.6377551020408164e-05,
+      "loss": 0.3127,
+      "step": 5875
+    },
+    {
+      "epoch": 0.511132287966733,
+      "grad_norm": 4.311783790588379,
+      "learning_rate": 1.630535232961109e-05,
+      "loss": 0.2705,
+      "step": 5900
+    },
+    {
+      "epoch": 0.5132981027462531,
+      "grad_norm": 7.5022430419921875,
+      "learning_rate": 1.6233153638814014e-05,
+      "loss": 0.3089,
+      "step": 5925
+    },
+    {
+      "epoch": 0.5154639175257731,
+      "grad_norm": 9.813260078430176,
+      "learning_rate": 1.616095494801694e-05,
+      "loss": 0.3386,
+      "step": 5950
+    },
+    {
+      "epoch": 0.5176297323052933,
+      "grad_norm": 8.11892318725586,
+      "learning_rate": 1.608875625721987e-05,
+      "loss": 0.3796,
+      "step": 5975
+    },
+    {
+      "epoch": 0.5197955470848133,
+      "grad_norm": 8.750290870666504,
+      "learning_rate": 1.6016557566422798e-05,
+      "loss": 0.4231,
+      "step": 6000
+    },
+    {
+      "epoch": 0.5219613618643334,
+      "grad_norm": 8.316088676452637,
+      "learning_rate": 1.5944358875625725e-05,
+      "loss": 0.3922,
+      "step": 6025
+    },
+    {
+      "epoch": 0.5241271766438534,
+      "grad_norm": 4.458547592163086,
+      "learning_rate": 1.5872160184828648e-05,
+      "loss": 0.3138,
+      "step": 6050
+    },
+    {
+      "epoch": 0.5262929914233735,
+      "grad_norm": 4.100847244262695,
+      "learning_rate": 1.5799961494031575e-05,
+      "loss": 0.3106,
+      "step": 6075
+    },
+    {
+      "epoch": 0.5284588062028935,
+      "grad_norm": 3.5927000045776367,
+      "learning_rate": 1.57277628032345e-05,
+      "loss": 0.188,
+      "step": 6100
+    },
+    {
+      "epoch": 0.5306246209824136,
+      "grad_norm": 0.6444216370582581,
+      "learning_rate": 1.5655564112437428e-05,
+      "loss": 0.209,
+      "step": 6125
+    },
+    {
+      "epoch": 0.5327904357619336,
+      "grad_norm": 6.649785041809082,
+      "learning_rate": 1.5583365421640355e-05,
+      "loss": 0.2617,
+      "step": 6150
+    },
+    {
+      "epoch": 0.5349562505414537,
+      "grad_norm": 8.491826057434082,
+      "learning_rate": 1.551116673084328e-05,
+      "loss": 0.3059,
+      "step": 6175
+    },
+    {
+      "epoch": 0.5371220653209737,
+      "grad_norm": 22.71511459350586,
+      "learning_rate": 1.5438968040046208e-05,
+      "loss": 0.2764,
+      "step": 6200
+    },
+    {
+      "epoch": 0.5392878801004938,
+      "grad_norm": 6.877171516418457,
+      "learning_rate": 1.5366769349249135e-05,
+      "loss": 0.2801,
+      "step": 6225
+    },
+    {
+      "epoch": 0.5414536948800138,
+      "grad_norm": 0.46479833126068115,
+      "learning_rate": 1.529457065845206e-05,
+      "loss": 0.3744,
+      "step": 6250
+    },
+    {
+      "epoch": 0.5436195096595339,
+      "grad_norm": 7.200215816497803,
+      "learning_rate": 1.5222371967654987e-05,
+      "loss": 0.3067,
+      "step": 6275
+    },
+    {
+      "epoch": 0.5457853244390539,
+      "grad_norm": 6.230359077453613,
+      "learning_rate": 1.5150173276857913e-05,
+      "loss": 0.3305,
+      "step": 6300
+    },
+    {
+      "epoch": 0.5479511392185741,
+      "grad_norm": 3.2241950035095215,
+      "learning_rate": 1.5077974586060838e-05,
+      "loss": 0.2827,
+      "step": 6325
+    },
+    {
+      "epoch": 0.5501169539980941,
+      "grad_norm": 10.813590049743652,
+      "learning_rate": 1.5005775895263765e-05,
+      "loss": 0.2712,
+      "step": 6350
+    },
+    {
+      "epoch": 0.5522827687776142,
+      "grad_norm": 3.5207877159118652,
+      "learning_rate": 1.4933577204466692e-05,
+      "loss": 0.2677,
+      "step": 6375
+    },
+    {
+      "epoch": 0.5544485835571342,
+      "grad_norm": 6.884098529815674,
+      "learning_rate": 1.4861378513669619e-05,
+      "loss": 0.4269,
+      "step": 6400
+    },
+    {
+      "epoch": 0.5566143983366543,
+      "grad_norm": 12.490416526794434,
+      "learning_rate": 1.4789179822872547e-05,
+      "loss": 0.3834,
+      "step": 6425
+    },
+    {
+      "epoch": 0.5587802131161743,
+      "grad_norm": 6.844019889831543,
+      "learning_rate": 1.4716981132075472e-05,
+      "loss": 0.4177,
+      "step": 6450
+    },
+    {
+      "epoch": 0.5609460278956944,
+      "grad_norm": 2.4574711322784424,
+      "learning_rate": 1.4644782441278399e-05,
+      "loss": 0.2457,
+      "step": 6475
+    },
+    {
+      "epoch": 0.5631118426752144,
+      "grad_norm": 4.939560413360596,
+      "learning_rate": 1.4572583750481324e-05,
+      "loss": 0.348,
+      "step": 6500
+    },
+    {
+      "epoch": 0.5652776574547345,
+      "grad_norm": 11.443745613098145,
+      "learning_rate": 1.4500385059684252e-05,
+      "loss": 0.3035,
+      "step": 6525
+    },
+    {
+      "epoch": 0.5674434722342545,
+      "grad_norm": 5.136826515197754,
+      "learning_rate": 1.4428186368887177e-05,
+      "loss": 0.39,
+      "step": 6550
+    },
+    {
+      "epoch": 0.5696092870137746,
+      "grad_norm": 8.772330284118652,
+      "learning_rate": 1.4355987678090104e-05,
+      "loss": 0.366,
+      "step": 6575
+    },
+    {
+      "epoch": 0.5717751017932946,
+      "grad_norm": 0.46080633997917175,
+      "learning_rate": 1.428378898729303e-05,
+      "loss": 0.2299,
+      "step": 6600
+    },
+    {
+      "epoch": 0.5739409165728147,
+      "grad_norm": 5.478773593902588,
+      "learning_rate": 1.4211590296495957e-05,
+      "loss": 0.1737,
+      "step": 6625
+    },
+    {
+      "epoch": 0.5761067313523347,
+      "grad_norm": 11.235420227050781,
+      "learning_rate": 1.4139391605698884e-05,
+      "loss": 0.3773,
+      "step": 6650
+    },
+    {
+      "epoch": 0.5782725461318549,
+      "grad_norm": 7.810971260070801,
+      "learning_rate": 1.4067192914901809e-05,
+      "loss": 0.3409,
+      "step": 6675
+    },
+    {
+      "epoch": 0.5804383609113749,
+      "grad_norm": 2.817094087600708,
+      "learning_rate": 1.3994994224104737e-05,
+      "loss": 0.1739,
+      "step": 6700
+    },
+    {
+      "epoch": 0.582604175690895,
+      "grad_norm": 0.4941748082637787,
+      "learning_rate": 1.3922795533307664e-05,
+      "loss": 0.3462,
+      "step": 6725
+    },
+    {
+      "epoch": 0.584769990470415,
+      "grad_norm": 1.5013363361358643,
+      "learning_rate": 1.3850596842510589e-05,
+      "loss": 0.2976,
+      "step": 6750
+    },
+    {
+      "epoch": 0.586935805249935,
+      "grad_norm": 4.63820219039917,
+      "learning_rate": 1.3778398151713516e-05,
+      "loss": 0.3246,
+      "step": 6775
+    },
+    {
+      "epoch": 0.5891016200294551,
+      "grad_norm": 0.6134036779403687,
+      "learning_rate": 1.3706199460916443e-05,
+      "loss": 0.3808,
+      "step": 6800
+    },
+    {
+      "epoch": 0.5912674348089751,
+      "grad_norm": 9.693577766418457,
+      "learning_rate": 1.363400077011937e-05,
+      "loss": 0.2926,
+      "step": 6825
+    },
+    {
+      "epoch": 0.5934332495884952,
+      "grad_norm": 8.138602256774902,
+      "learning_rate": 1.3561802079322296e-05,
+      "loss": 0.2709,
+      "step": 6850
+    },
+    {
+      "epoch": 0.5955990643680152,
+      "grad_norm": 5.065515041351318,
+      "learning_rate": 1.3489603388525221e-05,
+      "loss": 0.3777,
+      "step": 6875
+    },
+    {
+      "epoch": 0.5977648791475353,
+      "grad_norm": 6.169302463531494,
+      "learning_rate": 1.341740469772815e-05,
+      "loss": 0.2834,
+      "step": 6900
+    },
+    {
+      "epoch": 0.5999306939270553,
+      "grad_norm": 1.4236884117126465,
+      "learning_rate": 1.3345206006931074e-05,
+      "loss": 0.2965,
+      "step": 6925
+    },
+    {
+      "epoch": 0.6020965087065754,
+      "grad_norm": 4.954479217529297,
+      "learning_rate": 1.3273007316134001e-05,
+      "loss": 0.2399,
+      "step": 6950
+    },
+    {
+      "epoch": 0.6042623234860954,
+      "grad_norm": 1.1738444566726685,
+      "learning_rate": 1.3200808625336928e-05,
+      "loss": 0.2936,
+      "step": 6975
+    },
+    {
+      "epoch": 0.6064281382656155,
+      "grad_norm": 6.822793006896973,
+      "learning_rate": 1.3128609934539855e-05,
+      "loss": 0.2674,
+      "step": 7000
+    },
+    {
+      "epoch": 0.6085939530451355,
+      "grad_norm": 9.408463478088379,
+      "learning_rate": 1.3056411243742781e-05,
+      "loss": 0.265,
+      "step": 7025
+    },
+    {
+      "epoch": 0.6107597678246557,
+      "grad_norm": 24.97877311706543,
+      "learning_rate": 1.2984212552945706e-05,
+      "loss": 0.3257,
+      "step": 7050
+    },
+    {
+      "epoch": 0.6129255826041757,
+      "grad_norm": 2.854039192199707,
+      "learning_rate": 1.2912013862148633e-05,
+      "loss": 0.3504,
+      "step": 7075
+    },
+    {
+      "epoch": 0.6150913973836958,
+      "grad_norm": 0.40900859236717224,
+      "learning_rate": 1.283981517135156e-05,
+      "loss": 0.1485,
+      "step": 7100
+    },
+    {
+      "epoch": 0.6172572121632158,
+      "grad_norm": 5.776600360870361,
+      "learning_rate": 1.2767616480554486e-05,
+      "loss": 0.2598,
+      "step": 7125
+    },
+    {
+      "epoch": 0.6194230269427359,
+      "grad_norm": 1.7507195472717285,
+      "learning_rate": 1.2695417789757413e-05,
+      "loss": 0.2838,
+      "step": 7150
+    },
+    {
+      "epoch": 0.6215888417222559,
+      "grad_norm": 7.723363399505615,
+      "learning_rate": 1.2623219098960338e-05,
+      "loss": 0.3391,
+      "step": 7175
+    },
+    {
+      "epoch": 0.623754656501776,
+      "grad_norm": 6.485815048217773,
+      "learning_rate": 1.2551020408163267e-05,
+      "loss": 0.3568,
+      "step": 7200
+    },
+    {
+      "epoch": 0.625920471281296,
+      "grad_norm": 0.392874151468277,
+      "learning_rate": 1.2481709664998075e-05,
+      "loss": 0.3001,
+      "step": 7225
+    },
+    {
+      "epoch": 0.6280862860608161,
+      "grad_norm": 1.3930811882019043,
+      "learning_rate": 1.2409510974201001e-05,
+      "loss": 0.2613,
+      "step": 7250
+    },
+    {
+      "epoch": 0.6302521008403361,
+      "grad_norm": 0.3461158275604248,
+      "learning_rate": 1.2337312283403928e-05,
+      "loss": 0.3379,
+      "step": 7275
+    },
+    {
+      "epoch": 0.6324179156198562,
+      "grad_norm": 3.489888906478882,
+      "learning_rate": 1.2265113592606855e-05,
+      "loss": 0.3347,
+      "step": 7300
+    },
+    {
+      "epoch": 0.6345837303993762,
+      "grad_norm": 2.3235511779785156,
+      "learning_rate": 1.219291490180978e-05,
+      "loss": 0.242,
+      "step": 7325
+    },
+    {
+      "epoch": 0.6367495451788963,
+      "grad_norm": 10.576093673706055,
+      "learning_rate": 1.2120716211012708e-05,
+      "loss": 0.3076,
+      "step": 7350
+    },
+    {
+      "epoch": 0.6389153599584163,
+      "grad_norm": 4.862971305847168,
+      "learning_rate": 1.2048517520215633e-05,
+      "loss": 0.3055,
+      "step": 7375
+    },
+    {
+      "epoch": 0.6410811747379365,
+      "grad_norm": 4.282524108886719,
+      "learning_rate": 1.197631882941856e-05,
+      "loss": 0.4014,
+      "step": 7400
+    },
+    {
+      "epoch": 0.6432469895174565,
+      "grad_norm": 1.2869305610656738,
+      "learning_rate": 1.1904120138621487e-05,
+      "loss": 0.3723,
+      "step": 7425
+    },
+    {
+      "epoch": 0.6454128042969766,
+      "grad_norm": 8.37488842010498,
+      "learning_rate": 1.1831921447824414e-05,
+      "loss": 0.3421,
+      "step": 7450
+    },
+    {
+      "epoch": 0.6475786190764966,
+      "grad_norm": 8.292667388916016,
+      "learning_rate": 1.175972275702734e-05,
+      "loss": 0.4306,
+      "step": 7475
+    },
+    {
+      "epoch": 0.6497444338560167,
+      "grad_norm": 7.678843975067139,
+      "learning_rate": 1.1687524066230265e-05,
+      "loss": 0.2536,
+      "step": 7500
+    },
+    {
+      "epoch": 0.6519102486355367,
+      "grad_norm": 1.5608030557632446,
+      "learning_rate": 1.1615325375433192e-05,
+      "loss": 0.264,
+      "step": 7525
+    },
+    {
+      "epoch": 0.6540760634150568,
+      "grad_norm": 7.649046897888184,
+      "learning_rate": 1.1543126684636119e-05,
+      "loss": 0.1767,
+      "step": 7550
+    },
+    {
+      "epoch": 0.6562418781945768,
+      "grad_norm": 4.701557636260986,
+      "learning_rate": 1.1470927993839045e-05,
+      "loss": 0.259,
+      "step": 7575
+    },
+    {
+      "epoch": 0.6584076929740968,
+      "grad_norm": 14.77114200592041,
+      "learning_rate": 1.1398729303041972e-05,
+      "loss": 0.2761,
+      "step": 7600
+    },
+    {
+      "epoch": 0.6605735077536169,
+      "grad_norm": 0.08189712464809418,
+      "learning_rate": 1.1326530612244897e-05,
+      "loss": 0.2934,
+      "step": 7625
+    },
+    {
+      "epoch": 0.6627393225331369,
+      "grad_norm": 8.246410369873047,
+      "learning_rate": 1.1254331921447826e-05,
+      "loss": 0.3055,
+      "step": 7650
+    },
+    {
+      "epoch": 0.664905137312657,
+      "grad_norm": 2.8091800212860107,
+      "learning_rate": 1.118213323065075e-05,
+      "loss": 0.2532,
+      "step": 7675
+    },
+    {
+      "epoch": 0.667070952092177,
+      "grad_norm": 8.43855094909668,
+      "learning_rate": 1.1109934539853677e-05,
+      "loss": 0.2942,
+      "step": 7700
+    },
+    {
+      "epoch": 0.6692367668716971,
+      "grad_norm": 2.259917974472046,
+      "learning_rate": 1.1037735849056604e-05,
+      "loss": 0.2048,
+      "step": 7725
+    },
+    {
+      "epoch": 0.6714025816512171,
+      "grad_norm": 13.296177864074707,
+      "learning_rate": 1.096553715825953e-05,
+      "loss": 0.2884,
+      "step": 7750
+    },
+    {
+      "epoch": 0.6735683964307373,
+      "grad_norm": 7.745298862457275,
+      "learning_rate": 1.0893338467462457e-05,
+      "loss": 0.3598,
+      "step": 7775
+    },
+    {
+      "epoch": 0.6757342112102573,
+      "grad_norm": 1.932173490524292,
+      "learning_rate": 1.0821139776665382e-05,
+      "loss": 0.3318,
+      "step": 7800
+    },
+    {
+      "epoch": 0.6779000259897774,
+      "grad_norm": 7.833034515380859,
+      "learning_rate": 1.0748941085868309e-05,
+      "loss": 0.3058,
+      "step": 7825
+    },
+    {
+      "epoch": 0.6800658407692974,
+      "grad_norm": 8.620037078857422,
+      "learning_rate": 1.0676742395071238e-05,
+      "loss": 0.3395,
+      "step": 7850
+    },
+    {
+      "epoch": 0.6822316555488175,
+      "grad_norm": 8.948209762573242,
+      "learning_rate": 1.0604543704274163e-05,
+      "loss": 0.2973,
+      "step": 7875
+    },
+    {
+      "epoch": 0.6843974703283375,
+      "grad_norm": 5.001883506774902,
+      "learning_rate": 1.053234501347709e-05,
+      "loss": 0.2741,
+      "step": 7900
+    },
+    {
+      "epoch": 0.6865632851078576,
+      "grad_norm": 10.376258850097656,
+      "learning_rate": 1.0460146322680016e-05,
+      "loss": 0.2493,
+      "step": 7925
+    },
+    {
+      "epoch": 0.6887290998873776,
+      "grad_norm": 9.021862030029297,
+      "learning_rate": 1.0387947631882943e-05,
+      "loss": 0.2966,
+      "step": 7950
+    },
+    {
+      "epoch": 0.6908949146668977,
+      "grad_norm": 12.025108337402344,
+      "learning_rate": 1.0315748941085868e-05,
+      "loss": 0.3207,
+      "step": 7975
+    },
+    {
+      "epoch": 0.6930607294464177,
+      "grad_norm": 0.8383066058158875,
+      "learning_rate": 1.0243550250288794e-05,
+      "loss": 0.2501,
+      "step": 8000
+    },
+    {
+      "epoch": 0.6952265442259378,
+      "grad_norm": 2.6812140941619873,
+      "learning_rate": 1.0171351559491723e-05,
+      "loss": 0.4028,
+      "step": 8025
+    },
+    {
+      "epoch": 0.6973923590054578,
+      "grad_norm": 11.301798820495605,
+      "learning_rate": 1.0099152868694648e-05,
+      "loss": 0.3549,
+      "step": 8050
+    },
+    {
+      "epoch": 0.6995581737849779,
+      "grad_norm": 8.55245304107666,
+      "learning_rate": 1.0026954177897575e-05,
+      "loss": 0.3805,
+      "step": 8075
+    },
+    {
+      "epoch": 0.7017239885644979,
+      "grad_norm": 1.9036015272140503,
+      "learning_rate": 9.9547554871005e-06,
+      "loss": 0.353,
+      "step": 8100
+    },
+    {
+      "epoch": 0.7038898033440181,
+      "grad_norm": 1.0196151733398438,
+      "learning_rate": 9.882556796303428e-06,
+      "loss": 0.3569,
+      "step": 8125
+    },
+    {
+      "epoch": 0.7060556181235381,
+      "grad_norm": 2.688908338546753,
+      "learning_rate": 9.810358105506355e-06,
+      "loss": 0.2588,
+      "step": 8150
+    },
+    {
+      "epoch": 0.7082214329030582,
+      "grad_norm": 0.6335782408714294,
+      "learning_rate": 9.73815941470928e-06,
+      "loss": 0.2252,
+      "step": 8175
+    },
+    {
+      "epoch": 0.7103872476825782,
+      "grad_norm": 4.539221286773682,
+      "learning_rate": 9.665960723912206e-06,
+      "loss": 0.2747,
+      "step": 8200
+    },
+    {
+      "epoch": 0.7125530624620983,
+      "grad_norm": 8.757186889648438,
+      "learning_rate": 9.593762033115133e-06,
+      "loss": 0.3239,
+      "step": 8225
+    },
+    {
+      "epoch": 0.7147188772416183,
+      "grad_norm": 1.7275235652923584,
+      "learning_rate": 9.52156334231806e-06,
+      "loss": 0.2954,
+      "step": 8250
+    },
+    {
+      "epoch": 0.7168846920211384,
+      "grad_norm": 6.338670253753662,
+      "learning_rate": 9.449364651520987e-06,
+      "loss": 0.3749,
+      "step": 8275
+    },
+    {
+      "epoch": 0.7190505068006584,
+      "grad_norm": 1.565496563911438,
+      "learning_rate": 9.377165960723912e-06,
+      "loss": 0.2757,
+      "step": 8300
+    },
+    {
+      "epoch": 0.7212163215801785,
+      "grad_norm": 0.0664602667093277,
+      "learning_rate": 9.30496726992684e-06,
+      "loss": 0.3012,
+      "step": 8325
+    },
+    {
+      "epoch": 0.7233821363596985,
+      "grad_norm": 10.375814437866211,
+      "learning_rate": 9.232768579129765e-06,
+      "loss": 0.2985,
+      "step": 8350
+    },
+    {
+      "epoch": 0.7255479511392185,
+      "grad_norm": 16.607072830200195,
+      "learning_rate": 9.160569888332692e-06,
+      "loss": 0.2656,
+      "step": 8375
+    },
+    {
+      "epoch": 0.7277137659187386,
+      "grad_norm": 0.6724597811698914,
+      "learning_rate": 9.088371197535618e-06,
+      "loss": 0.2007,
+      "step": 8400
+    },
+    {
+      "epoch": 0.7298795806982586,
+      "grad_norm": 2.3397414684295654,
+      "learning_rate": 9.016172506738545e-06,
+      "loss": 0.2402,
+      "step": 8425
+    },
+    {
+      "epoch": 0.7320453954777787,
+      "grad_norm": 11.172548294067383,
+      "learning_rate": 8.943973815941472e-06,
+      "loss": 0.3434,
+      "step": 8450
+    },
+    {
+      "epoch": 0.7342112102572987,
+      "grad_norm": 12.031539916992188,
+      "learning_rate": 8.871775125144397e-06,
+      "loss": 0.2628,
+      "step": 8475
+    },
+    {
+      "epoch": 0.7363770250368189,
+      "grad_norm": 0.37211769819259644,
+      "learning_rate": 8.799576434347324e-06,
+      "loss": 0.265,
+      "step": 8500
+    },
+    {
+      "epoch": 0.738542839816339,
+      "grad_norm": 6.181528568267822,
+      "learning_rate": 8.72737774355025e-06,
+      "loss": 0.3748,
+      "step": 8525
+    },
+    {
+      "epoch": 0.740708654595859,
+      "grad_norm": 2.7227742671966553,
+      "learning_rate": 8.655179052753177e-06,
+      "loss": 0.249,
+      "step": 8550
+    },
+    {
+      "epoch": 0.742874469375379,
+      "grad_norm": 7.977476596832275,
+      "learning_rate": 8.582980361956104e-06,
+      "loss": 0.3375,
+      "step": 8575
+    },
+    {
+      "epoch": 0.7450402841548991,
+      "grad_norm": 11.404130935668945,
+      "learning_rate": 8.510781671159029e-06,
+      "loss": 0.3336,
+      "step": 8600
+    },
+    {
+      "epoch": 0.7472060989344191,
+      "grad_norm": 0.4421218931674957,
+      "learning_rate": 8.438582980361957e-06,
+      "loss": 0.3702,
+      "step": 8625
+    },
+    {
+      "epoch": 0.7493719137139392,
+      "grad_norm": 4.386607646942139,
+      "learning_rate": 8.366384289564882e-06,
+      "loss": 0.3494,
+      "step": 8650
+    },
+    {
+      "epoch": 0.7515377284934592,
+      "grad_norm": 5.428525924682617,
+      "learning_rate": 8.294185598767809e-06,
+      "loss": 0.2996,
+      "step": 8675
+    },
+    {
+      "epoch": 0.7537035432729793,
+      "grad_norm": 0.3034394085407257,
+      "learning_rate": 8.221986907970736e-06,
+      "loss": 0.2433,
+      "step": 8700
+    },
+    {
+      "epoch": 0.7558693580524993,
+      "grad_norm": 3.75878643989563,
+      "learning_rate": 8.149788217173662e-06,
+      "loss": 0.3027,
+      "step": 8725
+    },
+    {
+      "epoch": 0.7580351728320194,
+      "grad_norm": 9.965909004211426,
+      "learning_rate": 8.077589526376589e-06,
+      "loss": 0.382,
+      "step": 8750
+    },
+    {
+      "epoch": 0.7602009876115394,
+      "grad_norm": 7.314566135406494,
+      "learning_rate": 8.005390835579514e-06,
+      "loss": 0.2874,
+      "step": 8775
+    },
+    {
+      "epoch": 0.7623668023910595,
+      "grad_norm": 8.704547882080078,
+      "learning_rate": 7.93319214478244e-06,
+      "loss": 0.2737,
+      "step": 8800
+    },
+    {
+      "epoch": 0.7645326171705795,
+      "grad_norm": 10.275945663452148,
+      "learning_rate": 7.86099345398537e-06,
+      "loss": 0.3212,
+      "step": 8825
+    },
+    {
+      "epoch": 0.7666984319500997,
+      "grad_norm": 4.1912641525268555,
+      "learning_rate": 7.788794763188294e-06,
+      "loss": 0.3475,
+      "step": 8850
+    },
+    {
+      "epoch": 0.7688642467296197,
+      "grad_norm": 10.281148910522461,
+      "learning_rate": 7.716596072391221e-06,
+      "loss": 0.221,
+      "step": 8875
+    },
+    {
+      "epoch": 0.7710300615091398,
+      "grad_norm": 9.613810539245605,
+      "learning_rate": 7.644397381594146e-06,
+      "loss": 0.2587,
+      "step": 8900
+    },
+    {
+      "epoch": 0.7731958762886598,
+      "grad_norm": 1.2200976610183716,
+      "learning_rate": 7.572198690797074e-06,
+      "loss": 0.2852,
+      "step": 8925
+    },
+    {
+      "epoch": 0.7753616910681799,
+      "grad_norm": 2.445672035217285,
+      "learning_rate": 7.5e-06,
+      "loss": 0.3837,
+      "step": 8950
+    },
+    {
+      "epoch": 0.7775275058476999,
+      "grad_norm": 13.744851112365723,
+      "learning_rate": 7.427801309202927e-06,
+      "loss": 0.2333,
+      "step": 8975
+    },
+    {
+      "epoch": 0.77969332062722,
+      "grad_norm": 4.426064968109131,
+      "learning_rate": 7.355602618405853e-06,
+      "loss": 0.3036,
+      "step": 9000
+    },
+    {
+      "epoch": 0.78185913540674,
+      "grad_norm": 8.329988479614258,
+      "learning_rate": 7.28340392760878e-06,
+      "loss": 0.3287,
+      "step": 9025
+    },
+    {
+      "epoch": 0.7840249501862601,
+      "grad_norm": 4.122848987579346,
+      "learning_rate": 7.211205236811706e-06,
+      "loss": 0.3248,
+      "step": 9050
+    },
+    {
+      "epoch": 0.7861907649657801,
+      "grad_norm": 6.127285480499268,
+      "learning_rate": 7.139006546014633e-06,
+      "loss": 0.2395,
+      "step": 9075
+    },
+    {
+      "epoch": 0.7883565797453002,
+      "grad_norm": 1.6887600421905518,
+      "learning_rate": 7.066807855217559e-06,
+      "loss": 0.2647,
+      "step": 9100
+    },
+    {
+      "epoch": 0.7905223945248202,
+      "grad_norm": 1.4300670623779297,
+      "learning_rate": 6.9946091644204855e-06,
+      "loss": 0.3345,
+      "step": 9125
+    },
+    {
+      "epoch": 0.7926882093043403,
+      "grad_norm": 9.334101676940918,
+      "learning_rate": 6.922410473623411e-06,
+      "loss": 0.3421,
+      "step": 9150
+    },
+    {
+      "epoch": 0.7948540240838603,
+      "grad_norm": 6.996714115142822,
+      "learning_rate": 6.850211782826339e-06,
+      "loss": 0.3496,
+      "step": 9175
+    },
+    {
+      "epoch": 0.7970198388633805,
+      "grad_norm": 8.47280216217041,
+      "learning_rate": 6.778013092029265e-06,
+      "loss": 0.253,
+      "step": 9200
+    },
+    {
+      "epoch": 0.7991856536429005,
+      "grad_norm": 3.239483118057251,
+      "learning_rate": 6.708702348864075e-06,
+      "loss": 0.3462,
+      "step": 9225
+    },
+    {
+      "epoch": 0.8013514684224206,
+      "grad_norm": 1.6153030395507812,
+      "learning_rate": 6.6365036580670006e-06,
+      "loss": 0.2688,
+      "step": 9250
+    },
+    {
+      "epoch": 0.8035172832019406,
+      "grad_norm": 5.316878795623779,
+      "learning_rate": 6.564304967269927e-06,
+      "loss": 0.3301,
+      "step": 9275
+    },
+    {
+      "epoch": 0.8056830979814606,
+      "grad_norm": 8.06822395324707,
+      "learning_rate": 6.492106276472853e-06,
+      "loss": 0.3382,
+      "step": 9300
+    },
+    {
+      "epoch": 0.8078489127609807,
+      "grad_norm": 2.8038644790649414,
+      "learning_rate": 6.41990758567578e-06,
+      "loss": 0.2219,
+      "step": 9325
+    },
+    {
+      "epoch": 0.8100147275405007,
+      "grad_norm": 5.063823223114014,
+      "learning_rate": 6.3477088948787066e-06,
+      "loss": 0.278,
+      "step": 9350
+    },
+    {
+      "epoch": 0.8121805423200208,
+      "grad_norm": 6.974782466888428,
+      "learning_rate": 6.275510204081633e-06,
+      "loss": 0.2338,
+      "step": 9375
+    },
+    {
+      "epoch": 0.8143463570995408,
+      "grad_norm": 2.8085834980010986,
+      "learning_rate": 6.203311513284559e-06,
+      "loss": 0.2732,
+      "step": 9400
+    },
+    {
+      "epoch": 0.8165121718790609,
+      "grad_norm": 12.976601600646973,
+      "learning_rate": 6.131112822487486e-06,
+      "loss": 0.2973,
+      "step": 9425
+    },
+    {
+      "epoch": 0.8186779866585809,
+      "grad_norm": 2.7448630332946777,
+      "learning_rate": 6.058914131690412e-06,
+      "loss": 0.2783,
+      "step": 9450
+    },
+    {
+      "epoch": 0.820843801438101,
+      "grad_norm": 2.347792387008667,
+      "learning_rate": 5.986715440893339e-06,
+      "loss": 0.2418,
+      "step": 9475
+    },
+    {
+      "epoch": 0.823009616217621,
+      "grad_norm": 2.851559638977051,
+      "learning_rate": 5.914516750096265e-06,
+      "loss": 0.2603,
+      "step": 9500
+    },
+    {
+      "epoch": 0.8251754309971411,
+      "grad_norm": 6.941406726837158,
+      "learning_rate": 5.842318059299192e-06,
+      "loss": 0.1888,
+      "step": 9525
+    },
+    {
+      "epoch": 0.8273412457766611,
+      "grad_norm": 4.45375394821167,
+      "learning_rate": 5.770119368502118e-06,
+      "loss": 0.2581,
+      "step": 9550
+    },
+    {
+      "epoch": 0.8295070605561813,
+      "grad_norm": 5.2709641456604,
+      "learning_rate": 5.6979206777050444e-06,
+      "loss": 0.2742,
+      "step": 9575
+    },
+    {
+      "epoch": 0.8316728753357013,
+      "grad_norm": 2.6814463138580322,
+      "learning_rate": 5.62572198690797e-06,
+      "loss": 0.2156,
+      "step": 9600
+    },
+    {
+      "epoch": 0.8338386901152214,
+      "grad_norm": 0.12416364997625351,
+      "learning_rate": 5.553523296110898e-06,
+      "loss": 0.3317,
+      "step": 9625
+    },
+    {
+      "epoch": 0.8360045048947414,
+      "grad_norm": 5.639218807220459,
+      "learning_rate": 5.481324605313824e-06,
+      "loss": 0.1967,
+      "step": 9650
+    },
+    {
+      "epoch": 0.8381703196742615,
+      "grad_norm": 0.8800064921379089,
+      "learning_rate": 5.4091259145167504e-06,
+      "loss": 0.1701,
+      "step": 9675
+    },
+    {
+      "epoch": 0.8403361344537815,
+      "grad_norm": 2.7125442028045654,
+      "learning_rate": 5.336927223719676e-06,
+      "loss": 0.3064,
+      "step": 9700
+    },
+    {
+      "epoch": 0.8425019492333016,
+      "grad_norm": 3.1365272998809814,
+      "learning_rate": 5.264728532922603e-06,
+      "loss": 0.3511,
+      "step": 9725
+    },
+    {
+      "epoch": 0.8446677640128216,
+      "grad_norm": 10.584244728088379,
+      "learning_rate": 5.19252984212553e-06,
+      "loss": 0.2461,
+      "step": 9750
+    },
+    {
+      "epoch": 0.8468335787923417,
+      "grad_norm": 0.7926290035247803,
+      "learning_rate": 5.1203311513284565e-06,
+      "loss": 0.3047,
+      "step": 9775
+    },
+    {
+      "epoch": 0.8489993935718617,
+      "grad_norm": 10.744616508483887,
+      "learning_rate": 5.048132460531382e-06,
+      "loss": 0.3234,
+      "step": 9800
+    },
+    {
+      "epoch": 0.8511652083513818,
+      "grad_norm": 3.9436535835266113,
+      "learning_rate": 4.975933769734309e-06,
+      "loss": 0.2843,
+      "step": 9825
+    },
+    {
+      "epoch": 0.8533310231309018,
+      "grad_norm": 0.2785266637802124,
+      "learning_rate": 4.903735078937235e-06,
+      "loss": 0.3365,
+      "step": 9850
+    },
+    {
+      "epoch": 0.8554968379104219,
+      "grad_norm": 7.446309566497803,
+      "learning_rate": 4.831536388140162e-06,
+      "loss": 0.3802,
+      "step": 9875
+    },
+    {
+      "epoch": 0.8576626526899419,
+      "grad_norm": 9.687524795532227,
+      "learning_rate": 4.759337697343088e-06,
+      "loss": 0.2587,
+      "step": 9900
+    },
+    {
+      "epoch": 0.8598284674694621,
+      "grad_norm": 0.4837453067302704,
+      "learning_rate": 4.687139006546015e-06,
+      "loss": 0.2367,
+      "step": 9925
+    },
+    {
+      "epoch": 0.8619942822489821,
+      "grad_norm": 0.7170611023902893,
+      "learning_rate": 4.614940315748941e-06,
+      "loss": 0.2971,
+      "step": 9950
+    },
+    {
+      "epoch": 0.8641600970285022,
+      "grad_norm": 16.417407989501953,
+      "learning_rate": 4.542741624951868e-06,
+      "loss": 0.2884,
+      "step": 9975
+    },
+    {
+      "epoch": 0.8663259118080222,
+      "grad_norm": 7.771174430847168,
+      "learning_rate": 4.4705429341547935e-06,
+      "loss": 0.2296,
+      "step": 10000
+    },
+    {
+      "epoch": 0.8684917265875423,
+      "grad_norm": 1.540907859802246,
+      "learning_rate": 4.398344243357721e-06,
+      "loss": 0.3145,
+      "step": 10025
+    },
+    {
+      "epoch": 0.8706575413670623,
+      "grad_norm": 1.4157791137695312,
+      "learning_rate": 4.326145552560647e-06,
+      "loss": 0.178,
+      "step": 10050
+    },
+    {
+      "epoch": 0.8728233561465824,
+      "grad_norm": 4.707205295562744,
+      "learning_rate": 4.253946861763574e-06,
+      "loss": 0.2681,
+      "step": 10075
+    },
+    {
+      "epoch": 0.8749891709261024,
+      "grad_norm": 3.7186520099639893,
+      "learning_rate": 4.1817481709664995e-06,
+      "loss": 0.3191,
+      "step": 10100
+    },
+    {
+      "epoch": 0.8771549857056224,
+      "grad_norm": 1.6584956645965576,
+      "learning_rate": 4.109549480169426e-06,
+      "loss": 0.2544,
+      "step": 10125
+    },
+    {
+      "epoch": 0.8793208004851425,
+      "grad_norm": 9.22360610961914,
+      "learning_rate": 4.037350789372352e-06,
+      "loss": 0.2965,
+      "step": 10150
+    },
+    {
+      "epoch": 0.8814866152646625,
+      "grad_norm": 3.5934746265411377,
+      "learning_rate": 3.96515209857528e-06,
+      "loss": 0.317,
+      "step": 10175
+    },
+    {
+      "epoch": 0.8836524300441826,
+      "grad_norm": 1.5978528261184692,
+      "learning_rate": 3.892953407778206e-06,
+      "loss": 0.2149,
+      "step": 10200
+    },
+    {
+      "epoch": 0.8858182448237026,
+      "grad_norm": 4.726417064666748,
+      "learning_rate": 3.820754716981132e-06,
+      "loss": 0.4876,
+      "step": 10225
+    },
+    {
+      "epoch": 0.8879840596032227,
+      "grad_norm": 7.836237907409668,
+      "learning_rate": 3.7485560261840585e-06,
+      "loss": 0.2984,
+      "step": 10250
+    },
+    {
+      "epoch": 0.8901498743827427,
+      "grad_norm": 6.5479912757873535,
+      "learning_rate": 3.676357335386985e-06,
+      "loss": 0.3024,
+      "step": 10275
+    },
+    {
+      "epoch": 0.8923156891622629,
+      "grad_norm": 1.180179476737976,
+      "learning_rate": 3.6041586445899115e-06,
+      "loss": 0.2447,
+      "step": 10300
+    },
+    {
+      "epoch": 0.8944815039417829,
+      "grad_norm": 5.868828773498535,
+      "learning_rate": 3.5319599537928378e-06,
+      "loss": 0.2684,
+      "step": 10325
+    },
+    {
+      "epoch": 0.896647318721303,
+      "grad_norm": 6.2655816078186035,
+      "learning_rate": 3.4597612629957645e-06,
+      "loss": 0.1714,
+      "step": 10350
+    },
+    {
+      "epoch": 0.898813133500823,
+      "grad_norm": 6.3384270668029785,
+      "learning_rate": 3.387562572198691e-06,
+      "loss": 0.2776,
+      "step": 10375
+    },
+    {
+      "epoch": 0.9009789482803431,
+      "grad_norm": 6.097102165222168,
+      "learning_rate": 3.315363881401617e-06,
+      "loss": 0.2745,
+      "step": 10400
+    },
+    {
+      "epoch": 0.9031447630598631,
+      "grad_norm": 7.250086784362793,
+      "learning_rate": 3.243165190604544e-06,
+      "loss": 0.3299,
+      "step": 10425
+    },
+    {
+      "epoch": 0.9053105778393832,
+      "grad_norm": 9.260988235473633,
+      "learning_rate": 3.17096649980747e-06,
+      "loss": 0.2629,
+      "step": 10450
+    },
+    {
+      "epoch": 0.9074763926189032,
+      "grad_norm": 8.009949684143066,
+      "learning_rate": 3.0987678090103964e-06,
+      "loss": 0.3627,
+      "step": 10475
+    },
+    {
+      "epoch": 0.9096422073984233,
+      "grad_norm": 1.247878074645996,
+      "learning_rate": 3.026569118213323e-06,
+      "loss": 0.2236,
+      "step": 10500
+    },
+    {
+      "epoch": 0.9118080221779433,
+      "grad_norm": 6.759634971618652,
+      "learning_rate": 2.9543704274162494e-06,
+      "loss": 0.2819,
+      "step": 10525
+    },
+    {
+      "epoch": 0.9139738369574634,
+      "grad_norm": 0.09837600588798523,
+      "learning_rate": 2.882171736619176e-06,
+      "loss": 0.3129,
+      "step": 10550
+    },
+    {
+      "epoch": 0.9161396517369834,
+      "grad_norm": 6.850848197937012,
+      "learning_rate": 2.8099730458221024e-06,
+      "loss": 0.3051,
+      "step": 10575
+    },
+    {
+      "epoch": 0.9183054665165035,
+      "grad_norm": 8.94210147857666,
+      "learning_rate": 2.7377743550250287e-06,
+      "loss": 0.3955,
+      "step": 10600
+    },
+    {
+      "epoch": 0.9204712812960235,
+      "grad_norm": 8.595787048339844,
+      "learning_rate": 2.6655756642279554e-06,
+      "loss": 0.2493,
+      "step": 10625
+    },
+    {
+      "epoch": 0.9226370960755437,
+      "grad_norm": 7.062394618988037,
+      "learning_rate": 2.5933769734308817e-06,
+      "loss": 0.2543,
+      "step": 10650
+    },
+    {
+      "epoch": 0.9248029108550637,
+      "grad_norm": 3.371393918991089,
+      "learning_rate": 2.521178282633808e-06,
+      "loss": 0.2222,
+      "step": 10675
+    },
+    {
+      "epoch": 0.9269687256345838,
+      "grad_norm": 1.3468866348266602,
+      "learning_rate": 2.4489795918367347e-06,
+      "loss": 0.2823,
+      "step": 10700
+    },
+    {
+      "epoch": 0.9291345404141038,
+      "grad_norm": 15.475239753723145,
+      "learning_rate": 2.376780901039661e-06,
+      "loss": 0.3098,
+      "step": 10725
+    },
+    {
+      "epoch": 0.9313003551936239,
+      "grad_norm": 6.605096340179443,
+      "learning_rate": 2.3045822102425877e-06,
+      "loss": 0.3009,
+      "step": 10750
+    },
+    {
+      "epoch": 0.9334661699731439,
+      "grad_norm": 3.2146847248077393,
+      "learning_rate": 2.2323835194455144e-06,
+      "loss": 0.2623,
+      "step": 10775
+    },
+    {
+      "epoch": 0.935631984752664,
+      "grad_norm": 2.727200508117676,
+      "learning_rate": 2.1601848286484407e-06,
+      "loss": 0.1952,
+      "step": 10800
+    },
+    {
+      "epoch": 0.937797799532184,
+      "grad_norm": 2.7418553829193115,
+      "learning_rate": 2.0879861378513674e-06,
+      "loss": 0.4527,
+      "step": 10825
+    },
+    {
+      "epoch": 0.939963614311704,
+      "grad_norm": 8.577201843261719,
+      "learning_rate": 2.0157874470542937e-06,
+      "loss": 0.2323,
+      "step": 10850
+    },
+    {
+      "epoch": 0.9421294290912241,
+      "grad_norm": 4.514817237854004,
+      "learning_rate": 1.94358875625722e-06,
+      "loss": 0.3109,
+      "step": 10875
+    },
+    {
+      "epoch": 0.9442952438707441,
+      "grad_norm": 10.761394500732422,
+      "learning_rate": 1.8713900654601463e-06,
+      "loss": 0.3335,
+      "step": 10900
+    },
+    {
+      "epoch": 0.9464610586502642,
+      "grad_norm": 8.004775047302246,
+      "learning_rate": 1.7991913746630728e-06,
+      "loss": 0.2862,
+      "step": 10925
+    },
+    {
+      "epoch": 0.9486268734297842,
+      "grad_norm": 7.491416931152344,
+      "learning_rate": 1.7269926838659993e-06,
+      "loss": 0.4005,
+      "step": 10950
+    },
+    {
+      "epoch": 0.9507926882093043,
+      "grad_norm": 6.168478488922119,
+      "learning_rate": 1.6547939930689255e-06,
+      "loss": 0.2815,
+      "step": 10975
+    },
+    {
+      "epoch": 0.9529585029888243,
+      "grad_norm": 7.221772193908691,
+      "learning_rate": 1.582595302271852e-06,
+      "loss": 0.2157,
+      "step": 11000
+    },
+    {
+      "epoch": 0.9551243177683445,
+      "grad_norm": 5.9744086265563965,
+      "learning_rate": 1.5103966114747788e-06,
+      "loss": 0.3733,
+      "step": 11025
+    },
+    {
+      "epoch": 0.9572901325478645,
+      "grad_norm": 5.776475429534912,
+      "learning_rate": 1.4381979206777053e-06,
+      "loss": 0.2843,
+      "step": 11050
+    },
+    {
+      "epoch": 0.9594559473273846,
+      "grad_norm": 1.3870640993118286,
+      "learning_rate": 1.3659992298806316e-06,
+      "loss": 0.1963,
+      "step": 11075
+    },
+    {
+      "epoch": 0.9616217621069046,
+      "grad_norm": 7.3776535987854,
+      "learning_rate": 1.293800539083558e-06,
+      "loss": 0.3081,
+      "step": 11100
+    },
+    {
+      "epoch": 0.9637875768864247,
+      "grad_norm": 11.289216995239258,
+      "learning_rate": 1.2216018482864846e-06,
+      "loss": 0.2317,
+      "step": 11125
+    },
+    {
+      "epoch": 0.9659533916659447,
+      "grad_norm": 11.621864318847656,
+      "learning_rate": 1.1494031574894108e-06,
+      "loss": 0.3027,
+      "step": 11150
+    },
+    {
+      "epoch": 0.9681192064454648,
+      "grad_norm": 11.617834091186523,
+      "learning_rate": 1.0772044666923373e-06,
+      "loss": 0.3581,
+      "step": 11175
+    },
+    {
+      "epoch": 0.9702850212249848,
+      "grad_norm": 5.500637531280518,
+      "learning_rate": 1.0050057758952638e-06,
+      "loss": 0.3,
+      "step": 11200
+    },
+    {
+      "epoch": 0.9724508360045049,
+      "grad_norm": 3.552578926086426,
+      "learning_rate": 9.328070850981902e-07,
+      "loss": 0.2797,
+      "step": 11225
+    },
+    {
+      "epoch": 0.9746166507840249,
+      "grad_norm": 1.074208378791809,
+      "learning_rate": 8.606083943011167e-07,
+      "loss": 0.2918,
+      "step": 11250
+    },
+    {
+      "epoch": 0.976782465563545,
+      "grad_norm": 11.449936866760254,
+      "learning_rate": 7.884097035040431e-07,
+      "loss": 0.2519,
+      "step": 11275
+    },
+    {
+      "epoch": 0.978948280343065,
+      "grad_norm": 2.988003730773926,
+      "learning_rate": 7.162110127069696e-07,
+      "loss": 0.2183,
+      "step": 11300
+    },
+    {
+      "epoch": 0.9811140951225851,
+      "grad_norm": 2.9280929565429688,
+      "learning_rate": 6.44012321909896e-07,
+      "loss": 0.2764,
+      "step": 11325
+    },
+    {
+      "epoch": 0.9832799099021051,
+      "grad_norm": 3.2279105186462402,
+      "learning_rate": 5.718136311128224e-07,
+      "loss": 0.4107,
+      "step": 11350
+    },
+    {
+      "epoch": 0.9854457246816253,
+      "grad_norm": 2.54160737991333,
+      "learning_rate": 4.996149403157489e-07,
+      "loss": 0.3135,
+      "step": 11375
+    },
+    {
+      "epoch": 0.9876115394611453,
+      "grad_norm": 1.3068925142288208,
+      "learning_rate": 4.2741624951867543e-07,
+      "loss": 0.2138,
+      "step": 11400
+    },
+    {
+      "epoch": 0.9897773542406654,
+      "grad_norm": 8.606940269470215,
+      "learning_rate": 3.5521755872160183e-07,
+      "loss": 0.2984,
+      "step": 11425
+    },
+    {
+      "epoch": 0.9919431690201854,
+      "grad_norm": 1.2513303756713867,
+      "learning_rate": 2.830188679245283e-07,
+      "loss": 0.2407,
+      "step": 11450
+    },
+    {
+      "epoch": 0.9941089837997055,
+      "grad_norm": 11.340466499328613,
+      "learning_rate": 2.1082017712745478e-07,
+      "loss": 0.2449,
+      "step": 11475
+    },
+    {
+      "epoch": 0.9962747985792255,
+      "grad_norm": 6.166193008422852,
+      "learning_rate": 1.386214863303812e-07,
+      "loss": 0.2629,
+      "step": 11500
+    },
+    {
+      "epoch": 0.9984406133587456,
+      "grad_norm": 4.004662990570068,
+      "learning_rate": 6.642279553330766e-08,
+      "loss": 0.3488,
+      "step": 11525
+    },
+    {
+      "epoch": 1.0,
+      "eval_cosine_accuracy": 0.9693415637860082,
+      "eval_loss": 0.4268312156200409,
+      "eval_runtime": 50.4023,
+      "eval_samples_per_second": 96.424,
+      "eval_steps_per_second": 6.031,
+      "step": 11543
+    }
+  ],
+  "logging_steps": 25,
+  "max_steps": 11543,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 5,
+        "early_stopping_threshold": 0.01
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 0
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-11543/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4899e7c561beea78ce03642baf2d821bc65ed7c535dfbe836f5eb0f11ceeaa7f
+size 5624

config.json

@@ -0,0 +1,47 @@
+{
+  "_name_or_path": "Alibaba-NLP/gte-modernbert-base",
+  "architectures": [
+    "ModernBertModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 50281,
+  "classifier_activation": "gelu",
+  "classifier_bias": false,
+  "classifier_dropout": 0.0,
+  "classifier_pooling": "mean",
+  "cls_token_id": 50281,
+  "decoder_bias": true,
+  "deterministic_flash_attn": false,
+  "embedding_dropout": 0.0,
+  "eos_token_id": 50282,
+  "global_attn_every_n_layers": 3,
+  "global_rope_theta": 160000.0,
+  "gradient_checkpointing": false,
+  "hidden_activation": "gelu",
+  "hidden_size": 768,
+  "initializer_cutoff_factor": 2.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 1152,
+  "layer_norm_eps": 1e-05,
+  "local_attention": 128,
+  "local_rope_theta": 10000.0,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "mlp_dropout": 0.0,
+  "model_type": "modernbert",
+  "norm_bias": false,
+  "norm_eps": 1e-05,
+  "num_attention_heads": 12,
+  "num_hidden_layers": 22,
+  "pad_token_id": 50283,
+  "position_embedding_type": "absolute",
+  "reference_compile": true,
+  "repad_logits_with_grad": false,
+  "sep_token_id": 50282,
+  "sparse_pred_ignore_index": -100,
+  "sparse_prediction": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.0",
+  "vocab_size": 50368
+}

config_sentence_transformers.json

@@ -0,0 +1,10 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.3.1",
+    "transformers": "4.48.0",
+    "pytorch": "2.4.0"
+  },
+  "prompts": {},
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96616f5c428a86aaed4423cdda82e8a7d4becc0682e33fca47374d01cd7df333
+size 596070136

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  }
+]

runs/Feb20_14-59-58_r-m7n-autotrain-advanced-kyhs0yea-70f9a-nry3q/events.out.tfevents.1740063604.r-m7n-autotrain-advanced-kyhs0yea-70f9a-nry3q.102.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d80079404c5967a7eadf6a64d5b2a4e10e846c5b8d22924a3aef679045b67d9f
-size 98558
+oid sha256:4c7d57ff79811792dffd2074b6bcadc9392b45a58ea6a637272ac33fbde4dd3d
+size 102407

runs/Feb20_14-59-58_r-m7n-autotrain-advanced-kyhs0yea-70f9a-nry3q/events.out.tfevents.1740065597.r-m7n-autotrain-advanced-kyhs0yea-70f9a-nry3q.102.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:437d6a93141b052dcb68cddaaf972261b9bc923c607fb4740d9434c4f44ee294
+size 418

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 8192,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,945 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "|||IP_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "1": {
+      "content": "<|padding|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50254": {
+      "content": "                        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50255": {
+      "content": "                       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50256": {
+      "content": "                      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50257": {
+      "content": "                     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50258": {
+      "content": "                    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50259": {
+      "content": "                   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50260": {
+      "content": "                  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50261": {
+      "content": "                 ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50262": {
+      "content": "                ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50263": {
+      "content": "               ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50264": {
+      "content": "              ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50265": {
+      "content": "             ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50266": {
+      "content": "            ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50267": {
+      "content": "           ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50268": {
+      "content": "          ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50269": {
+      "content": "         ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50270": {
+      "content": "        ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50271": {
+      "content": "       ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50272": {
+      "content": "      ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50273": {
+      "content": "     ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50274": {
+      "content": "    ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50275": {
+      "content": "   ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50276": {
+      "content": "  ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50277": {
+      "content": "|||EMAIL_ADDRESS|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50278": {
+      "content": "|||PHONE_NUMBER|||",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50279": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50280": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50281": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50282": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50283": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50284": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "50285": {
+      "content": "[unused0]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50286": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50287": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50288": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50289": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50290": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50291": {
+      "content": "[unused6]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50292": {
+      "content": "[unused7]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50293": {
+      "content": "[unused8]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50294": {
+      "content": "[unused9]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50295": {
+      "content": "[unused10]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50296": {
+      "content": "[unused11]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50297": {
+      "content": "[unused12]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50298": {
+      "content": "[unused13]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50299": {
+      "content": "[unused14]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50300": {
+      "content": "[unused15]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50301": {
+      "content": "[unused16]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50302": {
+      "content": "[unused17]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50303": {
+      "content": "[unused18]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50304": {
+      "content": "[unused19]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50305": {
+      "content": "[unused20]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50306": {
+      "content": "[unused21]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50307": {
+      "content": "[unused22]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50308": {
+      "content": "[unused23]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50309": {
+      "content": "[unused24]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50310": {
+      "content": "[unused25]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50311": {
+      "content": "[unused26]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50312": {
+      "content": "[unused27]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50313": {
+      "content": "[unused28]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50314": {
+      "content": "[unused29]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50315": {
+      "content": "[unused30]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50316": {
+      "content": "[unused31]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50317": {
+      "content": "[unused32]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50318": {
+      "content": "[unused33]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50319": {
+      "content": "[unused34]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50320": {
+      "content": "[unused35]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50321": {
+      "content": "[unused36]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50322": {
+      "content": "[unused37]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50323": {
+      "content": "[unused38]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50324": {
+      "content": "[unused39]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50325": {
+      "content": "[unused40]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50326": {
+      "content": "[unused41]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50327": {
+      "content": "[unused42]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50328": {
+      "content": "[unused43]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50329": {
+      "content": "[unused44]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50330": {
+      "content": "[unused45]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50331": {
+      "content": "[unused46]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50332": {
+      "content": "[unused47]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50333": {
+      "content": "[unused48]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50334": {
+      "content": "[unused49]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50335": {
+      "content": "[unused50]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50336": {
+      "content": "[unused51]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50337": {
+      "content": "[unused52]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50338": {
+      "content": "[unused53]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50339": {
+      "content": "[unused54]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50340": {
+      "content": "[unused55]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50341": {
+      "content": "[unused56]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50342": {
+      "content": "[unused57]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50343": {
+      "content": "[unused58]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50344": {
+      "content": "[unused59]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50345": {
+      "content": "[unused60]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50346": {
+      "content": "[unused61]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50347": {
+      "content": "[unused62]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50348": {
+      "content": "[unused63]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50349": {
+      "content": "[unused64]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50350": {
+      "content": "[unused65]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50351": {
+      "content": "[unused66]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50352": {
+      "content": "[unused67]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50353": {
+      "content": "[unused68]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50354": {
+      "content": "[unused69]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50355": {
+      "content": "[unused70]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50356": {
+      "content": "[unused71]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50357": {
+      "content": "[unused72]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50358": {
+      "content": "[unused73]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50359": {
+      "content": "[unused74]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50360": {
+      "content": "[unused75]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50361": {
+      "content": "[unused76]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50362": {
+      "content": "[unused77]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50363": {
+      "content": "[unused78]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50364": {
+      "content": "[unused79]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50365": {
+      "content": "[unused80]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50366": {
+      "content": "[unused81]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "50367": {
+      "content": "[unused82]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "tokenizer_class": "PreTrainedTokenizerFast",
+  "unk_token": "[UNK]"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4899e7c561beea78ce03642baf2d821bc65ed7c535dfbe836f5eb0f11ceeaa7f
+size 5624

training_params.json

@@ -0,0 +1,33 @@
+{
+    "data_path": "gte-modernbert-philosophy-v1-1-autotr/autotrain-data",
+    "model": "Alibaba-NLP/gte-modernbert-base",
+    "lr": 3e-05,
+    "epochs": 1,
+    "max_seq_length": 8192,
+    "batch_size": 8,
+    "warmup_ratio": 0.1,
+    "gradient_accumulation": 1,
+    "optimizer": "adamw_torch",
+    "scheduler": "linear",
+    "weight_decay": 0.0,
+    "max_grad_norm": 1.0,
+    "seed": 42,
+    "train_split": "train",
+    "valid_split": "validation",
+    "logging_steps": -1,
+    "project_name": "gte-modernbert-philosophy-v1-1-autotr",
+    "auto_find_batch_size": false,
+    "mixed_precision": "fp16",
+    "save_total_limit": 1,
+    "push_to_hub": true,
+    "eval_strategy": "epoch",
+    "username": "m7n",
+    "log": "tensorboard",
+    "early_stopping_patience": 5,
+    "early_stopping_threshold": 0.01,
+    "trainer": "triplet",
+    "sentence1_column": "autotrain_sentence1",
+    "sentence2_column": "autotrain_sentence2",
+    "sentence3_column": "autotrain_sentence3",
+    "target_column": "autotrain_target"
+}