Training in progress, step 1040, checkpoint

.gitattributes

@@ -59,3 +59,4 @@ checkpoint-880/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-920/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-960/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 checkpoint-1000/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-1040/tokenizer.json filter=lfs diff=lfs merge=lfs -text

checkpoint-1040/README.md

@@ -0,0 +1,202 @@
+---
+base_model: Qwen/Qwen2.5-1.5B
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.14.0

checkpoint-1040/adapter_config.json

@@ -0,0 +1,37 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen2.5-1.5B",
+  "bias": "none",
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 32,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "up_proj",
+    "v_proj",
+    "k_proj",
+    "o_proj",
+    "down_proj",
+    "q_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-1040/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8547b82a39264ae81d40f5b0cafa761f4e678ee43c398da974c38c26fb46fe23
+size 147770496

checkpoint-1040/added_tokens.json

@@ -0,0 +1,24 @@
+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

checkpoint-1040/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:736315c0e84a1a11d9d3edcc1121c29641ca73f7a89e2edec0683e9dd4dd7e48
+size 295771370

checkpoint-1040/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6cb795a5cea0baa625c50007a6c9da09c6bbb5c16b560424070384a479e7d8a6
+size 14512

checkpoint-1040/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f19604377bd828eb366c68946ad997a4ff4d69beaeea93ee58915135768ec63
+size 14512

checkpoint-1040/scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8c57972c7118a2abafd75cb77f76a7bee4476cdf68d921254140d98a3efb2aca
+size 988

checkpoint-1040/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af536ed4866424bd2e64e6d6d427a251180d7b0ada85ad2f203f3bf78738a1d3
+size 1064

checkpoint-1040/special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1040/tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

checkpoint-1040/tokenizer_config.json

@@ -0,0 +1,209 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% endif %}{% if system_message is defined %}{{ 'System: ' + system_message + '<|endoftext|>' + '\n' }}{% endif %}{% for message in loop_messages %}{% set content = message['content'] %}{% if message['role'] == 'user' %}{{ 'Human: ' + content + '<|endoftext|>' + '\nAssistant:' }}{% elif message['role'] == 'assistant' %}{{ content + '<|endoftext|>' + '\n' }}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

checkpoint-1040/trainer_state.json

@@ -0,0 +1,762 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.2220264317180616,
+  "eval_steps": 500,
+  "global_step": 1040,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.011747430249632892,
+      "grad_norm": 1.5699902772903442,
+      "learning_rate": 0.00027,
+      "loss": 3.0983,
+      "step": 10
+    },
+    {
+      "epoch": 0.023494860499265784,
+      "grad_norm": 1.6029695272445679,
+      "learning_rate": 0.00029991523567092526,
+      "loss": 2.062,
+      "step": 20
+    },
+    {
+      "epoch": 0.03524229074889868,
+      "grad_norm": 1.593436360359192,
+      "learning_rate": 0.00029962234616583063,
+      "loss": 1.2074,
+      "step": 30
+    },
+    {
+      "epoch": 0.04698972099853157,
+      "grad_norm": 0.5851414799690247,
+      "learning_rate": 0.00029912069357315393,
+      "loss": 0.888,
+      "step": 40
+    },
+    {
+      "epoch": 0.05873715124816446,
+      "grad_norm": 0.25992292165756226,
+      "learning_rate": 0.0002984109778320875,
+      "loss": 0.7685,
+      "step": 50
+    },
+    {
+      "epoch": 0.07048458149779736,
+      "grad_norm": 0.21082307398319244,
+      "learning_rate": 0.00029749418918542057,
+      "loss": 0.7096,
+      "step": 60
+    },
+    {
+      "epoch": 0.08223201174743025,
+      "grad_norm": 0.16843102872371674,
+      "learning_rate": 0.0002963716067978866,
+      "loss": 0.6901,
+      "step": 70
+    },
+    {
+      "epoch": 0.09397944199706314,
+      "grad_norm": 0.12076722830533981,
+      "learning_rate": 0.000295044796971387,
+      "loss": 0.6702,
+      "step": 80
+    },
+    {
+      "epoch": 0.10572687224669604,
+      "grad_norm": 0.21371866762638092,
+      "learning_rate": 0.000293515610959582,
+      "loss": 0.6353,
+      "step": 90
+    },
+    {
+      "epoch": 0.11747430249632893,
+      "grad_norm": 0.13458965718746185,
+      "learning_rate": 0.0002917861823848985,
+      "loss": 0.6479,
+      "step": 100
+    },
+    {
+      "epoch": 0.12922173274596183,
+      "grad_norm": 0.265765517950058,
+      "learning_rate": 0.0002898589242615568,
+      "loss": 0.6244,
+      "step": 110
+    },
+    {
+      "epoch": 0.14096916299559473,
+      "grad_norm": 0.1473032385110855,
+      "learning_rate": 0.0002877365256287728,
+      "loss": 0.6217,
+      "step": 120
+    },
+    {
+      "epoch": 0.1527165932452276,
+      "grad_norm": 0.1591167151927948,
+      "learning_rate": 0.00028542194779883047,
+      "loss": 0.6022,
+      "step": 130
+    },
+    {
+      "epoch": 0.1644640234948605,
+      "grad_norm": 0.13270772993564606,
+      "learning_rate": 0.00028291842022526133,
+      "loss": 0.6098,
+      "step": 140
+    },
+    {
+      "epoch": 0.1762114537444934,
+      "grad_norm": 0.1444919854402542,
+      "learning_rate": 0.0002802294359968954,
+      "loss": 0.5971,
+      "step": 150
+    },
+    {
+      "epoch": 0.18795888399412627,
+      "grad_norm": 0.1571902334690094,
+      "learning_rate": 0.0002773587469640702,
+      "loss": 0.5937,
+      "step": 160
+    },
+    {
+      "epoch": 0.19970631424375918,
+      "grad_norm": 0.11585285514593124,
+      "learning_rate": 0.0002743103585037989,
+      "loss": 0.6054,
+      "step": 170
+    },
+    {
+      "epoch": 0.21145374449339208,
+      "grad_norm": 0.10303252190351486,
+      "learning_rate": 0.0002710885239312008,
+      "loss": 0.5708,
+      "step": 180
+    },
+    {
+      "epoch": 0.22320117474302498,
+      "grad_norm": 0.09355439245700836,
+      "learning_rate": 0.00026769773856499167,
+      "loss": 0.5806,
+      "step": 190
+    },
+    {
+      "epoch": 0.23494860499265785,
+      "grad_norm": 0.09288550913333893,
+      "learning_rate": 0.0002641427334553158,
+      "loss": 0.5747,
+      "step": 200
+    },
+    {
+      "epoch": 0.24669603524229075,
+      "grad_norm": 0.10875760763883591,
+      "learning_rate": 0.00026042846878266795,
+      "loss": 0.5879,
+      "step": 210
+    },
+    {
+      "epoch": 0.25844346549192365,
+      "grad_norm": 0.09756477177143097,
+      "learning_rate": 0.0002565601269371192,
+      "loss": 0.5852,
+      "step": 220
+    },
+    {
+      "epoch": 0.2701908957415565,
+      "grad_norm": 0.10926368832588196,
+      "learning_rate": 0.0002525431052874997,
+      "loss": 0.5605,
+      "step": 230
+    },
+    {
+      "epoch": 0.28193832599118945,
+      "grad_norm": 0.09802033007144928,
+      "learning_rate": 0.00024838300865062966,
+      "loss": 0.5738,
+      "step": 240
+    },
+    {
+      "epoch": 0.2936857562408223,
+      "grad_norm": 0.10284294933080673,
+      "learning_rate": 0.00024408564147110443,
+      "loss": 0.5847,
+      "step": 250
+    },
+    {
+      "epoch": 0.3054331864904552,
+      "grad_norm": 0.09890703111886978,
+      "learning_rate": 0.00023965699972254602,
+      "loss": 0.5736,
+      "step": 260
+    },
+    {
+      "epoch": 0.31718061674008813,
+      "grad_norm": 0.09091509878635406,
+      "learning_rate": 0.00023510326254162027,
+      "loss": 0.5577,
+      "step": 270
+    },
+    {
+      "epoch": 0.328928046989721,
+      "grad_norm": 0.0930003970861435,
+      "learning_rate": 0.00023043078360649285,
+      "loss": 0.5651,
+      "step": 280
+    },
+    {
+      "epoch": 0.3406754772393539,
+      "grad_norm": 0.0988878533244133,
+      "learning_rate": 0.00022564608227175316,
+      "loss": 0.548,
+      "step": 290
+    },
+    {
+      "epoch": 0.3524229074889868,
+      "grad_norm": 0.11749754101037979,
+      "learning_rate": 0.0002207558344721757,
+      "loss": 0.5587,
+      "step": 300
+    },
+    {
+      "epoch": 0.3641703377386197,
+      "grad_norm": 0.10936658829450607,
+      "learning_rate": 0.00021576686340800985,
+      "loss": 0.5694,
+      "step": 310
+    },
+    {
+      "epoch": 0.37591776798825255,
+      "grad_norm": 0.15082670748233795,
+      "learning_rate": 0.00021068613002479553,
+      "loss": 0.5688,
+      "step": 320
+    },
+    {
+      "epoch": 0.3876651982378855,
+      "grad_norm": 0.10255635529756546,
+      "learning_rate": 0.00020552072330098716,
+      "loss": 0.56,
+      "step": 330
+    },
+    {
+      "epoch": 0.39941262848751835,
+      "grad_norm": 0.10504507273435593,
+      "learning_rate": 0.0002002778503569374,
+      "loss": 0.557,
+      "step": 340
+    },
+    {
+      "epoch": 0.4111600587371512,
+      "grad_norm": 0.1146383211016655,
+      "learning_rate": 0.00019496482639904194,
+      "loss": 0.5497,
+      "step": 350
+    },
+    {
+      "epoch": 0.42290748898678415,
+      "grad_norm": 0.09596443176269531,
+      "learning_rate": 0.00018958906451307489,
+      "loss": 0.556,
+      "step": 360
+    },
+    {
+      "epoch": 0.434654919236417,
+      "grad_norm": 0.10395421087741852,
+      "learning_rate": 0.0001841580653209565,
+      "loss": 0.5634,
+      "step": 370
+    },
+    {
+      "epoch": 0.44640234948604995,
+      "grad_norm": 0.08797866106033325,
+      "learning_rate": 0.00017867940651538483,
+      "loss": 0.5544,
+      "step": 380
+    },
+    {
+      "epoch": 0.4581497797356828,
+      "grad_norm": 0.1006847620010376,
+      "learning_rate": 0.0001731607322869329,
+      "loss": 0.5562,
+      "step": 390
+    },
+    {
+      "epoch": 0.4698972099853157,
+      "grad_norm": 0.09849337488412857,
+      "learning_rate": 0.00016760974265836331,
+      "loss": 0.5477,
+      "step": 400
+    },
+    {
+      "epoch": 0.48164464023494863,
+      "grad_norm": 0.0988384336233139,
+      "learning_rate": 0.00016203418274104278,
+      "loss": 0.5459,
+      "step": 410
+    },
+    {
+      "epoch": 0.4933920704845815,
+      "grad_norm": 0.09625212848186493,
+      "learning_rate": 0.0001564418319284454,
+      "loss": 0.5516,
+      "step": 420
+    },
+    {
+      "epoch": 0.5051395007342144,
+      "grad_norm": 0.09705183655023575,
+      "learning_rate": 0.00015084049304182347,
+      "loss": 0.5375,
+      "step": 430
+    },
+    {
+      "epoch": 0.5168869309838473,
+      "grad_norm": 0.17180472612380981,
+      "learning_rate": 0.00014523798144319027,
+      "loss": 0.539,
+      "step": 440
+    },
+    {
+      "epoch": 0.5286343612334802,
+      "grad_norm": 0.09553302824497223,
+      "learning_rate": 0.00013964211413080522,
+      "loss": 0.5418,
+      "step": 450
+    },
+    {
+      "epoch": 0.540381791483113,
+      "grad_norm": 0.10648112744092941,
+      "learning_rate": 0.0001340606988323758,
+      "loss": 0.5414,
+      "step": 460
+    },
+    {
+      "epoch": 0.5521292217327459,
+      "grad_norm": 0.09907692670822144,
+      "learning_rate": 0.00012850152311119498,
+      "loss": 0.5353,
+      "step": 470
+    },
+    {
+      "epoch": 0.5638766519823789,
+      "grad_norm": 0.11162377148866653,
+      "learning_rate": 0.00012297234350041228,
+      "loss": 0.528,
+      "step": 480
+    },
+    {
+      "epoch": 0.5756240822320118,
+      "grad_norm": 0.10550152510404587,
+      "learning_rate": 0.00011748087468060128,
+      "loss": 0.533,
+      "step": 490
+    },
+    {
+      "epoch": 0.5873715124816447,
+      "grad_norm": 0.09718377143144608,
+      "learning_rate": 0.0001120347787157222,
+      "loss": 0.5409,
+      "step": 500
+    },
+    {
+      "epoch": 0.5991189427312775,
+      "grad_norm": 0.09185861796140671,
+      "learning_rate": 0.0001066416543624984,
+      "loss": 0.5354,
+      "step": 510
+    },
+    {
+      "epoch": 0.6108663729809104,
+      "grad_norm": 0.0927920788526535,
+      "learning_rate": 0.00010130902646812369,
+      "loss": 0.5454,
+      "step": 520
+    },
+    {
+      "epoch": 0.6226138032305433,
+      "grad_norm": 0.087093785405159,
+      "learning_rate": 9.604433547109344e-05,
+      "loss": 0.5295,
+      "step": 530
+    },
+    {
+      "epoch": 0.6343612334801763,
+      "grad_norm": 0.09994326531887054,
+      "learning_rate": 9.085492701980751e-05,
+      "loss": 0.5322,
+      "step": 540
+    },
+    {
+      "epoch": 0.6461086637298091,
+      "grad_norm": 0.09507084637880325,
+      "learning_rate": 8.574804172343134e-05,
+      "loss": 0.5224,
+      "step": 550
+    },
+    {
+      "epoch": 0.657856093979442,
+      "grad_norm": 0.08571015298366547,
+      "learning_rate": 8.07308050493148e-05,
+      "loss": 0.5378,
+      "step": 560
+    },
+    {
+      "epoch": 0.6696035242290749,
+      "grad_norm": 0.08876761794090271,
+      "learning_rate": 7.581021738106408e-05,
+      "loss": 0.5265,
+      "step": 570
+    },
+    {
+      "epoch": 0.6813509544787077,
+      "grad_norm": 0.09467241168022156,
+      "learning_rate": 7.099314425113907e-05,
+      "loss": 0.5392,
+      "step": 580
+    },
+    {
+      "epoch": 0.6930983847283406,
+      "grad_norm": 0.08804601430892944,
+      "learning_rate": 6.628630676160445e-05,
+      "loss": 0.5365,
+      "step": 590
+    },
+    {
+      "epoch": 0.7048458149779736,
+      "grad_norm": 0.08877623081207275,
+      "learning_rate": 6.169627220639871e-05,
+      "loss": 0.5354,
+      "step": 600
+    },
+    {
+      "epoch": 0.7165932452276065,
+      "grad_norm": 0.09122662246227264,
+      "learning_rate": 5.722944490820774e-05,
+      "loss": 0.5356,
+      "step": 610
+    },
+    {
+      "epoch": 0.7283406754772394,
+      "grad_norm": 0.08744510263204575,
+      "learning_rate": 5.289205728272586e-05,
+      "loss": 0.5424,
+      "step": 620
+    },
+    {
+      "epoch": 0.7400881057268722,
+      "grad_norm": 0.08927814662456512,
+      "learning_rate": 4.869016114277345e-05,
+      "loss": 0.5268,
+      "step": 630
+    },
+    {
+      "epoch": 0.7518355359765051,
+      "grad_norm": 0.09256933629512787,
+      "learning_rate": 4.462961925440341e-05,
+      "loss": 0.5414,
+      "step": 640
+    },
+    {
+      "epoch": 0.7635829662261381,
+      "grad_norm": 0.08703339844942093,
+      "learning_rate": 4.071609715677899e-05,
+      "loss": 0.5376,
+      "step": 650
+    },
+    {
+      "epoch": 0.775330396475771,
+      "grad_norm": 0.08876251429319382,
+      "learning_rate": 3.695505525723465e-05,
+      "loss": 0.5307,
+      "step": 660
+    },
+    {
+      "epoch": 0.7870778267254038,
+      "grad_norm": 0.08702490478754044,
+      "learning_rate": 3.3351741212551595e-05,
+      "loss": 0.5307,
+      "step": 670
+    },
+    {
+      "epoch": 0.7988252569750367,
+      "grad_norm": 0.08601511269807816,
+      "learning_rate": 2.9911182607076516e-05,
+      "loss": 0.5372,
+      "step": 680
+    },
+    {
+      "epoch": 0.8105726872246696,
+      "grad_norm": 0.0857272818684578,
+      "learning_rate": 2.663817993790021e-05,
+      "loss": 0.528,
+      "step": 690
+    },
+    {
+      "epoch": 0.8223201174743024,
+      "grad_norm": 0.08725214004516602,
+      "learning_rate": 2.3537299916883512e-05,
+      "loss": 0.5378,
+      "step": 700
+    },
+    {
+      "epoch": 0.8340675477239354,
+      "grad_norm": 0.0845843032002449,
+      "learning_rate": 2.0612869098875988e-05,
+      "loss": 0.5389,
+      "step": 710
+    },
+    {
+      "epoch": 0.8458149779735683,
+      "grad_norm": 0.08480172604322433,
+      "learning_rate": 1.786896784501778e-05,
+      "loss": 0.5244,
+      "step": 720
+    },
+    {
+      "epoch": 0.8575624082232012,
+      "grad_norm": 0.09265288710594177,
+      "learning_rate": 1.5309424629547164e-05,
+      "loss": 0.5403,
+      "step": 730
+    },
+    {
+      "epoch": 0.869309838472834,
+      "grad_norm": 0.08523637801408768,
+      "learning_rate": 1.2937810698057921e-05,
+      "loss": 0.5332,
+      "step": 740
+    },
+    {
+      "epoch": 0.8810572687224669,
+      "grad_norm": 0.08431612700223923,
+      "learning_rate": 1.0757435084658694e-05,
+      "loss": 0.5198,
+      "step": 750
+    },
+    {
+      "epoch": 0.8928046989720999,
+      "grad_norm": 0.08998807519674301,
+      "learning_rate": 8.771339994987953e-06,
+      "loss": 0.5251,
+      "step": 760
+    },
+    {
+      "epoch": 0.9045521292217328,
+      "grad_norm": 0.08884080499410629,
+      "learning_rate": 6.98229656152543e-06,
+      "loss": 0.5449,
+      "step": 770
+    },
+    {
+      "epoch": 0.9162995594713657,
+      "grad_norm": 0.08583056926727295,
+      "learning_rate": 5.392800977123047e-06,
+      "loss": 0.5264,
+      "step": 780
+    },
+    {
+      "epoch": 0.9280469897209985,
+      "grad_norm": 0.08824951946735382,
+      "learning_rate": 4.005071012149952e-06,
+      "loss": 0.5306,
+      "step": 790
+    },
+    {
+      "epoch": 0.9397944199706314,
+      "grad_norm": 0.08726619184017181,
+      "learning_rate": 2.821042920111427e-06,
+      "loss": 0.5205,
+      "step": 800
+    },
+    {
+      "epoch": 0.9515418502202643,
+      "grad_norm": 0.08729498088359833,
+      "learning_rate": 1.8423687360584137e-06,
+      "loss": 0.5217,
+      "step": 810
+    },
+    {
+      "epoch": 0.9632892804698973,
+      "grad_norm": 0.08497074991464615,
+      "learning_rate": 1.070413971558115e-06,
+      "loss": 0.534,
+      "step": 820
+    },
+    {
+      "epoch": 0.9750367107195301,
+      "grad_norm": 0.08497001975774765,
+      "learning_rate": 5.062557094410058e-07,
+      "loss": 0.5241,
+      "step": 830
+    },
+    {
+      "epoch": 0.986784140969163,
+      "grad_norm": 0.08295251429080963,
+      "learning_rate": 1.5068110098296338e-07,
+      "loss": 0.5337,
+      "step": 840
+    },
+    {
+      "epoch": 0.9985315712187959,
+      "grad_norm": 0.10860061645507812,
+      "learning_rate": 0.00022638651575377874,
+      "loss": 0.5227,
+      "step": 850
+    },
+    {
+      "epoch": 1.0105726872246696,
+      "grad_norm": 0.11796294897794724,
+      "learning_rate": 0.00022478592280680777,
+      "loss": 0.588,
+      "step": 860
+    },
+    {
+      "epoch": 1.0223201174743024,
+      "grad_norm": 0.1594405323266983,
+      "learning_rate": 0.0002231739162937319,
+      "loss": 0.5307,
+      "step": 870
+    },
+    {
+      "epoch": 1.0340675477239354,
+      "grad_norm": 0.10787333548069,
+      "learning_rate": 0.0002215507422333499,
+      "loss": 0.5359,
+      "step": 880
+    },
+    {
+      "epoch": 1.0458149779735684,
+      "grad_norm": 0.10763130336999893,
+      "learning_rate": 0.0002199166483488127,
+      "loss": 0.5407,
+      "step": 890
+    },
+    {
+      "epoch": 1.0575624082232011,
+      "grad_norm": 0.13658902049064636,
+      "learning_rate": 0.00021827188402981652,
+      "loss": 0.5255,
+      "step": 900
+    },
+    {
+      "epoch": 1.0693098384728341,
+      "grad_norm": 0.10522827506065369,
+      "learning_rate": 0.00021661670029454207,
+      "loss": 0.5276,
+      "step": 910
+    },
+    {
+      "epoch": 1.0810572687224669,
+      "grad_norm": 0.1422538459300995,
+      "learning_rate": 0.0002149513497513448,
+      "loss": 0.5245,
+      "step": 920
+    },
+    {
+      "epoch": 1.0928046989720999,
+      "grad_norm": 0.10326780378818512,
+      "learning_rate": 0.00021327608656020305,
+      "loss": 0.5294,
+      "step": 930
+    },
+    {
+      "epoch": 1.1045521292217328,
+      "grad_norm": 0.11100132018327713,
+      "learning_rate": 0.00021159116639392868,
+      "loss": 0.52,
+      "step": 940
+    },
+    {
+      "epoch": 1.1162995594713656,
+      "grad_norm": 0.09583411365747452,
+      "learning_rate": 0.00020989684639914738,
+      "loss": 0.5247,
+      "step": 950
+    },
+    {
+      "epoch": 1.1280469897209986,
+      "grad_norm": 0.10812857002019882,
+      "learning_rate": 0.00020819338515705378,
+      "loss": 0.5236,
+      "step": 960
+    },
+    {
+      "epoch": 1.1397944199706314,
+      "grad_norm": 0.12208293378353119,
+      "learning_rate": 0.00020648104264394784,
+      "loss": 0.5217,
+      "step": 970
+    },
+    {
+      "epoch": 1.1515418502202643,
+      "grad_norm": 0.11540035158395767,
+      "learning_rate": 0.00020476008019155794,
+      "loss": 0.5387,
+      "step": 980
+    },
+    {
+      "epoch": 1.1632892804698973,
+      "grad_norm": 0.10755149275064468,
+      "learning_rate": 0.00020303076044715738,
+      "loss": 0.5057,
+      "step": 990
+    },
+    {
+      "epoch": 1.17503671071953,
+      "grad_norm": 0.10145018994808197,
+      "learning_rate": 0.0002012933473334804,
+      "loss": 0.5202,
+      "step": 1000
+    },
+    {
+      "epoch": 1.186784140969163,
+      "grad_norm": 0.11095395684242249,
+      "learning_rate": 0.00019954810600844277,
+      "loss": 0.5314,
+      "step": 1010
+    },
+    {
+      "epoch": 1.1985315712187958,
+      "grad_norm": 0.097834512591362,
+      "learning_rate": 0.00019779530282467456,
+      "loss": 0.5178,
+      "step": 1020
+    },
+    {
+      "epoch": 1.2102790014684288,
+      "grad_norm": 0.09915532171726227,
+      "learning_rate": 0.00019603520528887027,
+      "loss": 0.5205,
+      "step": 1030
+    },
+    {
+      "epoch": 1.2220264317180616,
+      "grad_norm": 0.1107698306441307,
+      "learning_rate": 0.00019426808202096298,
+      "loss": 0.5268,
+      "step": 1040
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2553,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 40,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.2043653294323139e+18,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1040/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8090e8234c33299c70be5a5af617aad982ec3110875f8573e2c8b0b3be950f52
+size 5752