Training in progress, step 150, checkpoint

last-checkpoint/README.md

@@ -0,0 +1,202 @@
+---
+base_model: katuni4ka/tiny-random-dbrx
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.13.2

last-checkpoint/adapter_config.json

@@ -0,0 +1,30 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "katuni4ka/tiny-random-dbrx",
+  "bias": "none",
+  "fan_in_fan_out": null,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "Wqkv",
+    "layer",
+    "out_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

last-checkpoint/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bde0909f59febea048b0ee3650ebd018c523a81cc85ce191f7bcec130271dcd9
+size 5752

last-checkpoint/added_tokens.json

@@ -0,0 +1,4 @@
+{
+  "<|im_end|>": 100279,
+  "<|im_start|>": 100278
+}

last-checkpoint/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7bf496f678bc626ff6d3915e6bf19c87a148a25b82009c6e7dad568f4f694843
+size 15814

last-checkpoint/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f21454591fe9e608de3ae4bb9afc72817b0d018a9a6151bd757d892eea4334f0
+size 14244

last-checkpoint/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b005d0b4552ae80a8f61c62d752f014caa57bb48adb1e675ddab90b214253e10
+size 1064

last-checkpoint/special_tokens_map.json

@@ -0,0 +1,30 @@
+{
+  "bos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|pad|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

last-checkpoint/tokenizer_config.json

@@ -0,0 +1,205 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "100256": {
+      "content": "<||_unused_0_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100257": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100258": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100259": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100260": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100261": {
+      "content": "<||_unused_1_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100262": {
+      "content": "<||_unused_2_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100263": {
+      "content": "<||_unused_3_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100264": {
+      "content": "<||_unused_4_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100265": {
+      "content": "<||_unused_5_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100266": {
+      "content": "<||_unused_6_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100267": {
+      "content": "<||_unused_7_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100268": {
+      "content": "<||_unused_8_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100269": {
+      "content": "<||_unused_9_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100270": {
+      "content": "<||_unused_10_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100271": {
+      "content": "<||_unused_11_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100272": {
+      "content": "<||_unused_12_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100273": {
+      "content": "<||_unused_13_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100274": {
+      "content": "<||_unused_14_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100275": {
+      "content": "<||_unused_15_||>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100276": {
+      "content": "<|endofprompt|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100277": {
+      "content": "<|pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100278": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100279": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|endoftext|>",
+  "chat_template": "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% set loop_messages = messages %}{% for message in loop_messages %}{% set content = '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'+ message['content'] | trim + '<|eot_id|>' %}{% if loop.index0 == 0 %}{% set content = bos_token + content %}{% endif %}{{ content }}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 32768,
+  "pad_token": "<|pad|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>"
+}

last-checkpoint/trainer_state.json

@@ -0,0 +1,1083 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.024155561818108618,
+  "eval_steps": 500,
+  "global_step": 150,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0001610370787873908,
+      "grad_norm": 9.459293323743623e-06,
+      "learning_rate": 2e-05,
+      "loss": 46.0,
+      "step": 1
+    },
+    {
+      "epoch": 0.0003220741575747816,
+      "grad_norm": 1.8449256458552554e-05,
+      "learning_rate": 4e-05,
+      "loss": 46.0,
+      "step": 2
+    },
+    {
+      "epoch": 0.0004831112363621724,
+      "grad_norm": 1.7989095795201138e-05,
+      "learning_rate": 6e-05,
+      "loss": 46.0,
+      "step": 3
+    },
+    {
+      "epoch": 0.0006441483151495632,
+      "grad_norm": 1.1759363587771077e-05,
+      "learning_rate": 8e-05,
+      "loss": 46.0,
+      "step": 4
+    },
+    {
+      "epoch": 0.000805185393936954,
+      "grad_norm": 1.139960022555897e-05,
+      "learning_rate": 0.0001,
+      "loss": 46.0,
+      "step": 5
+    },
+    {
+      "epoch": 0.0009662224727243448,
+      "grad_norm": 1.4804975762672257e-05,
+      "learning_rate": 0.00012,
+      "loss": 46.0,
+      "step": 6
+    },
+    {
+      "epoch": 0.0011272595515117356,
+      "grad_norm": 2.1032992663094774e-05,
+      "learning_rate": 0.00014,
+      "loss": 46.0,
+      "step": 7
+    },
+    {
+      "epoch": 0.0012882966302991264,
+      "grad_norm": 1.5429051927640103e-05,
+      "learning_rate": 0.00016,
+      "loss": 46.0,
+      "step": 8
+    },
+    {
+      "epoch": 0.001449333709086517,
+      "grad_norm": 1.1477978659968358e-05,
+      "learning_rate": 0.00018,
+      "loss": 46.0,
+      "step": 9
+    },
+    {
+      "epoch": 0.001610370787873908,
+      "grad_norm": 1.7123571524280123e-05,
+      "learning_rate": 0.0002,
+      "loss": 46.0,
+      "step": 10
+    },
+    {
+      "epoch": 0.0017714078666612987,
+      "grad_norm": 1.0772208952403162e-05,
+      "learning_rate": 0.00019999999999998722,
+      "loss": 46.0,
+      "step": 11
+    },
+    {
+      "epoch": 0.0019324449454486896,
+      "grad_norm": 9.808137292566244e-06,
+      "learning_rate": 0.0001999999999999488,
+      "loss": 46.0,
+      "step": 12
+    },
+    {
+      "epoch": 0.0020934820242360804,
+      "grad_norm": 1.8400718545308337e-05,
+      "learning_rate": 0.00019999999999988484,
+      "loss": 46.0,
+      "step": 13
+    },
+    {
+      "epoch": 0.0022545191030234712,
+      "grad_norm": 2.247749398520682e-05,
+      "learning_rate": 0.00019999999999979523,
+      "loss": 46.0,
+      "step": 14
+    },
+    {
+      "epoch": 0.002415556181810862,
+      "grad_norm": 1.1134417945868336e-05,
+      "learning_rate": 0.0001999999999996801,
+      "loss": 46.0,
+      "step": 15
+    },
+    {
+      "epoch": 0.002576593260598253,
+      "grad_norm": 1.4492045920633245e-05,
+      "learning_rate": 0.0001999999999995393,
+      "loss": 46.0,
+      "step": 16
+    },
+    {
+      "epoch": 0.0027376303393856437,
+      "grad_norm": 1.4358376574818976e-05,
+      "learning_rate": 0.00019999999999937293,
+      "loss": 46.0,
+      "step": 17
+    },
+    {
+      "epoch": 0.002898667418173034,
+      "grad_norm": 1.1229038136661984e-05,
+      "learning_rate": 0.00019999999999918098,
+      "loss": 46.0,
+      "step": 18
+    },
+    {
+      "epoch": 0.003059704496960425,
+      "grad_norm": 1.850031731009949e-05,
+      "learning_rate": 0.0001999999999989634,
+      "loss": 46.0,
+      "step": 19
+    },
+    {
+      "epoch": 0.003220741575747816,
+      "grad_norm": 1.7780901544028893e-05,
+      "learning_rate": 0.00019999999999872027,
+      "loss": 46.0,
+      "step": 20
+    },
+    {
+      "epoch": 0.0033817786545352066,
+      "grad_norm": 1.23179188449285e-05,
+      "learning_rate": 0.00019999999999845152,
+      "loss": 46.0,
+      "step": 21
+    },
+    {
+      "epoch": 0.0035428157333225975,
+      "grad_norm": 1.1472806363599375e-05,
+      "learning_rate": 0.00019999999999815716,
+      "loss": 46.0,
+      "step": 22
+    },
+    {
+      "epoch": 0.0037038528121099883,
+      "grad_norm": 9.506068636255804e-06,
+      "learning_rate": 0.00019999999999783724,
+      "loss": 46.0,
+      "step": 23
+    },
+    {
+      "epoch": 0.003864889890897379,
+      "grad_norm": 1.6990292351692915e-05,
+      "learning_rate": 0.0001999999999974917,
+      "loss": 46.0,
+      "step": 24
+    },
+    {
+      "epoch": 0.00402592696968477,
+      "grad_norm": 1.920577051350847e-05,
+      "learning_rate": 0.00019999999999712059,
+      "loss": 46.0,
+      "step": 25
+    },
+    {
+      "epoch": 0.004186964048472161,
+      "grad_norm": 1.57214544742601e-05,
+      "learning_rate": 0.00019999999999672388,
+      "loss": 46.0,
+      "step": 26
+    },
+    {
+      "epoch": 0.004348001127259552,
+      "grad_norm": 2.9738939701928757e-05,
+      "learning_rate": 0.00019999999999630155,
+      "loss": 46.0,
+      "step": 27
+    },
+    {
+      "epoch": 0.0045090382060469425,
+      "grad_norm": 1.2327154763625003e-05,
+      "learning_rate": 0.00019999999999585364,
+      "loss": 46.0,
+      "step": 28
+    },
+    {
+      "epoch": 0.004670075284834333,
+      "grad_norm": 1.3533114724850748e-05,
+      "learning_rate": 0.00019999999999538014,
+      "loss": 46.0,
+      "step": 29
+    },
+    {
+      "epoch": 0.004831112363621724,
+      "grad_norm": 1.5398421965073794e-05,
+      "learning_rate": 0.00019999999999488103,
+      "loss": 46.0,
+      "step": 30
+    },
+    {
+      "epoch": 0.004992149442409115,
+      "grad_norm": 1.6816700735944323e-05,
+      "learning_rate": 0.00019999999999435636,
+      "loss": 46.0,
+      "step": 31
+    },
+    {
+      "epoch": 0.005153186521196506,
+      "grad_norm": 1.3670231965079438e-05,
+      "learning_rate": 0.00019999999999380604,
+      "loss": 46.0,
+      "step": 32
+    },
+    {
+      "epoch": 0.005314223599983897,
+      "grad_norm": 1.3132093954482116e-05,
+      "learning_rate": 0.00019999999999323017,
+      "loss": 46.0,
+      "step": 33
+    },
+    {
+      "epoch": 0.0054752606787712875,
+      "grad_norm": 2.7774303816840984e-05,
+      "learning_rate": 0.00019999999999262868,
+      "loss": 46.0,
+      "step": 34
+    },
+    {
+      "epoch": 0.005636297757558678,
+      "grad_norm": 2.077547469525598e-05,
+      "learning_rate": 0.0001999999999920016,
+      "loss": 46.0,
+      "step": 35
+    },
+    {
+      "epoch": 0.005797334836346068,
+      "grad_norm": 1.980315391847398e-05,
+      "learning_rate": 0.00019999999999134892,
+      "loss": 46.0,
+      "step": 36
+    },
+    {
+      "epoch": 0.005958371915133459,
+      "grad_norm": 2.2662210540147498e-05,
+      "learning_rate": 0.00019999999999067067,
+      "loss": 46.0,
+      "step": 37
+    },
+    {
+      "epoch": 0.00611940899392085,
+      "grad_norm": 2.1207339159445837e-05,
+      "learning_rate": 0.00019999999998996683,
+      "loss": 46.0,
+      "step": 38
+    },
+    {
+      "epoch": 0.006280446072708241,
+      "grad_norm": 1.7205984477186576e-05,
+      "learning_rate": 0.00019999999998923738,
+      "loss": 46.0,
+      "step": 39
+    },
+    {
+      "epoch": 0.006441483151495632,
+      "grad_norm": 2.1629894035868347e-05,
+      "learning_rate": 0.00019999999998848234,
+      "loss": 46.0,
+      "step": 40
+    },
+    {
+      "epoch": 0.006602520230283022,
+      "grad_norm": 2.7965374101768248e-05,
+      "learning_rate": 0.00019999999998770166,
+      "loss": 46.0,
+      "step": 41
+    },
+    {
+      "epoch": 0.006763557309070413,
+      "grad_norm": 2.5096620447584428e-05,
+      "learning_rate": 0.00019999999998689542,
+      "loss": 46.0,
+      "step": 42
+    },
+    {
+      "epoch": 0.006924594387857804,
+      "grad_norm": 3.5594304790720344e-05,
+      "learning_rate": 0.0001999999999860636,
+      "loss": 46.0,
+      "step": 43
+    },
+    {
+      "epoch": 0.007085631466645195,
+      "grad_norm": 3.1788349588168785e-05,
+      "learning_rate": 0.00019999999998520618,
+      "loss": 46.0,
+      "step": 44
+    },
+    {
+      "epoch": 0.007246668545432586,
+      "grad_norm": 2.8515927624539472e-05,
+      "learning_rate": 0.00019999999998432315,
+      "loss": 46.0,
+      "step": 45
+    },
+    {
+      "epoch": 0.007407705624219977,
+      "grad_norm": 2.7847892852150835e-05,
+      "learning_rate": 0.0001999999999834145,
+      "loss": 46.0,
+      "step": 46
+    },
+    {
+      "epoch": 0.007568742703007367,
+      "grad_norm": 3.0626124498667195e-05,
+      "learning_rate": 0.00019999999998248033,
+      "loss": 46.0,
+      "step": 47
+    },
+    {
+      "epoch": 0.007729779781794758,
+      "grad_norm": 2.7933489036513492e-05,
+      "learning_rate": 0.00019999999998152052,
+      "loss": 46.0,
+      "step": 48
+    },
+    {
+      "epoch": 0.007890816860582149,
+      "grad_norm": 3.10891009576153e-05,
+      "learning_rate": 0.0001999999999805351,
+      "loss": 46.0,
+      "step": 49
+    },
+    {
+      "epoch": 0.00805185393936954,
+      "grad_norm": 1.9566661649150774e-05,
+      "learning_rate": 0.00019999999997952412,
+      "loss": 46.0,
+      "step": 50
+    },
+    {
+      "epoch": 0.00821289101815693,
+      "grad_norm": 5.5264958064071834e-05,
+      "learning_rate": 0.00019999999997848754,
+      "loss": 46.0,
+      "step": 51
+    },
+    {
+      "epoch": 0.008373928096944322,
+      "grad_norm": 2.4125354684656486e-05,
+      "learning_rate": 0.00019999999997742532,
+      "loss": 46.0,
+      "step": 52
+    },
+    {
+      "epoch": 0.008534965175731712,
+      "grad_norm": 3.317361915833317e-05,
+      "learning_rate": 0.00019999999997633757,
+      "loss": 46.0,
+      "step": 53
+    },
+    {
+      "epoch": 0.008696002254519103,
+      "grad_norm": 3.8641785067738965e-05,
+      "learning_rate": 0.00019999999997522417,
+      "loss": 46.0,
+      "step": 54
+    },
+    {
+      "epoch": 0.008857039333306494,
+      "grad_norm": 1.588473423907999e-05,
+      "learning_rate": 0.0001999999999740852,
+      "loss": 46.0,
+      "step": 55
+    },
+    {
+      "epoch": 0.009018076412093885,
+      "grad_norm": 3.6028392059961334e-05,
+      "learning_rate": 0.00019999999997292065,
+      "loss": 46.0,
+      "step": 56
+    },
+    {
+      "epoch": 0.009179113490881276,
+      "grad_norm": 3.70649722754024e-05,
+      "learning_rate": 0.0001999999999717305,
+      "loss": 46.0,
+      "step": 57
+    },
+    {
+      "epoch": 0.009340150569668667,
+      "grad_norm": 3.086036304011941e-05,
+      "learning_rate": 0.0001999999999705147,
+      "loss": 46.0,
+      "step": 58
+    },
+    {
+      "epoch": 0.009501187648456057,
+      "grad_norm": 3.073871630476788e-05,
+      "learning_rate": 0.00019999999996927337,
+      "loss": 46.0,
+      "step": 59
+    },
+    {
+      "epoch": 0.009662224727243448,
+      "grad_norm": 5.74203186261002e-05,
+      "learning_rate": 0.00019999999996800642,
+      "loss": 46.0,
+      "step": 60
+    },
+    {
+      "epoch": 0.009823261806030839,
+      "grad_norm": 1.878445800684858e-05,
+      "learning_rate": 0.0001999999999667139,
+      "loss": 46.0,
+      "step": 61
+    },
+    {
+      "epoch": 0.00998429888481823,
+      "grad_norm": 3.903669130522758e-05,
+      "learning_rate": 0.00019999999996539575,
+      "loss": 46.0,
+      "step": 62
+    },
+    {
+      "epoch": 0.01014533596360562,
+      "grad_norm": 6.173171277623624e-05,
+      "learning_rate": 0.000199999999964052,
+      "loss": 46.0,
+      "step": 63
+    },
+    {
+      "epoch": 0.010306373042393012,
+      "grad_norm": 3.220455255359411e-05,
+      "learning_rate": 0.0001999999999626827,
+      "loss": 46.0,
+      "step": 64
+    },
+    {
+      "epoch": 0.010467410121180402,
+      "grad_norm": 6.513421976706013e-05,
+      "learning_rate": 0.00019999999996128778,
+      "loss": 46.0,
+      "step": 65
+    },
+    {
+      "epoch": 0.010628447199967793,
+      "grad_norm": 4.503427044255659e-05,
+      "learning_rate": 0.00019999999995986726,
+      "loss": 46.0,
+      "step": 66
+    },
+    {
+      "epoch": 0.010789484278755184,
+      "grad_norm": 5.680751200998202e-05,
+      "learning_rate": 0.00019999999995842113,
+      "loss": 46.0,
+      "step": 67
+    },
+    {
+      "epoch": 0.010950521357542575,
+      "grad_norm": 5.3842857596464455e-05,
+      "learning_rate": 0.00019999999995694943,
+      "loss": 46.0,
+      "step": 68
+    },
+    {
+      "epoch": 0.011111558436329966,
+      "grad_norm": 3.435022517805919e-05,
+      "learning_rate": 0.00019999999995545212,
+      "loss": 46.0,
+      "step": 69
+    },
+    {
+      "epoch": 0.011272595515117357,
+      "grad_norm": 4.8579728172626346e-05,
+      "learning_rate": 0.00019999999995392925,
+      "loss": 46.0,
+      "step": 70
+    },
+    {
+      "epoch": 0.011433632593904747,
+      "grad_norm": 6.448462227126583e-05,
+      "learning_rate": 0.00019999999995238077,
+      "loss": 46.0,
+      "step": 71
+    },
+    {
+      "epoch": 0.011594669672692137,
+      "grad_norm": 3.0377606890397146e-05,
+      "learning_rate": 0.00019999999995080667,
+      "loss": 46.0,
+      "step": 72
+    },
+    {
+      "epoch": 0.011755706751479527,
+      "grad_norm": 4.2671086703194305e-05,
+      "learning_rate": 0.000199999999949207,
+      "loss": 46.0,
+      "step": 73
+    },
+    {
+      "epoch": 0.011916743830266918,
+      "grad_norm": 5.8799378166440874e-05,
+      "learning_rate": 0.0001999999999475817,
+      "loss": 46.0,
+      "step": 74
+    },
+    {
+      "epoch": 0.012077780909054309,
+      "grad_norm": 3.1305702577810735e-05,
+      "learning_rate": 0.00019999999994593085,
+      "loss": 46.0,
+      "step": 75
+    },
+    {
+      "epoch": 0.0122388179878417,
+      "grad_norm": 4.199046816211194e-05,
+      "learning_rate": 0.00019999999994425437,
+      "loss": 46.0,
+      "step": 76
+    },
+    {
+      "epoch": 0.01239985506662909,
+      "grad_norm": 4.404900028021075e-05,
+      "learning_rate": 0.0001999999999425523,
+      "loss": 46.0,
+      "step": 77
+    },
+    {
+      "epoch": 0.012560892145416482,
+      "grad_norm": 4.4940319639863446e-05,
+      "learning_rate": 0.0001999999999408247,
+      "loss": 46.0,
+      "step": 78
+    },
+    {
+      "epoch": 0.012721929224203872,
+      "grad_norm": 5.616362614091486e-05,
+      "learning_rate": 0.00019999999993907143,
+      "loss": 46.0,
+      "step": 79
+    },
+    {
+      "epoch": 0.012882966302991263,
+      "grad_norm": 4.128489672439173e-05,
+      "learning_rate": 0.00019999999993729257,
+      "loss": 46.0,
+      "step": 80
+    },
+    {
+      "epoch": 0.013044003381778654,
+      "grad_norm": 3.650110375019722e-05,
+      "learning_rate": 0.00019999999993548814,
+      "loss": 46.0,
+      "step": 81
+    },
+    {
+      "epoch": 0.013205040460566045,
+      "grad_norm": 3.355491207912564e-05,
+      "learning_rate": 0.0001999999999336581,
+      "loss": 46.0,
+      "step": 82
+    },
+    {
+      "epoch": 0.013366077539353436,
+      "grad_norm": 3.100166577496566e-05,
+      "learning_rate": 0.00019999999993180247,
+      "loss": 46.0,
+      "step": 83
+    },
+    {
+      "epoch": 0.013527114618140827,
+      "grad_norm": 4.128339787712321e-05,
+      "learning_rate": 0.00019999999992992127,
+      "loss": 46.0,
+      "step": 84
+    },
+    {
+      "epoch": 0.013688151696928217,
+      "grad_norm": 5.978300396236591e-05,
+      "learning_rate": 0.00019999999992801443,
+      "loss": 46.0,
+      "step": 85
+    },
+    {
+      "epoch": 0.013849188775715608,
+      "grad_norm": 6.463456520577893e-05,
+      "learning_rate": 0.00019999999992608203,
+      "loss": 46.0,
+      "step": 86
+    },
+    {
+      "epoch": 0.014010225854502999,
+      "grad_norm": 4.458018156583421e-05,
+      "learning_rate": 0.00019999999992412402,
+      "loss": 46.0,
+      "step": 87
+    },
+    {
+      "epoch": 0.01417126293329039,
+      "grad_norm": 3.9894523069960997e-05,
+      "learning_rate": 0.00019999999992214042,
+      "loss": 46.0,
+      "step": 88
+    },
+    {
+      "epoch": 0.01433230001207778,
+      "grad_norm": 6.879306602058932e-05,
+      "learning_rate": 0.00019999999992013123,
+      "loss": 46.0,
+      "step": 89
+    },
+    {
+      "epoch": 0.014493337090865172,
+      "grad_norm": 7.311927038244903e-05,
+      "learning_rate": 0.00019999999991809642,
+      "loss": 46.0,
+      "step": 90
+    },
+    {
+      "epoch": 0.014654374169652562,
+      "grad_norm": 7.103729149093851e-05,
+      "learning_rate": 0.00019999999991603603,
+      "loss": 46.0,
+      "step": 91
+    },
+    {
+      "epoch": 0.014815411248439953,
+      "grad_norm": 6.073687472962774e-05,
+      "learning_rate": 0.00019999999991395005,
+      "loss": 46.0,
+      "step": 92
+    },
+    {
+      "epoch": 0.014976448327227344,
+      "grad_norm": 8.019880624487996e-05,
+      "learning_rate": 0.0001999999999118385,
+      "loss": 46.0,
+      "step": 93
+    },
+    {
+      "epoch": 0.015137485406014735,
+      "grad_norm": 7.244279549922794e-05,
+      "learning_rate": 0.0001999999999097013,
+      "loss": 46.0,
+      "step": 94
+    },
+    {
+      "epoch": 0.015298522484802126,
+      "grad_norm": 6.675118493149057e-05,
+      "learning_rate": 0.00019999999990753854,
+      "loss": 46.0,
+      "step": 95
+    },
+    {
+      "epoch": 0.015459559563589517,
+      "grad_norm": 8.598087151767686e-05,
+      "learning_rate": 0.0001999999999053502,
+      "loss": 46.0,
+      "step": 96
+    },
+    {
+      "epoch": 0.015620596642376907,
+      "grad_norm": 4.9705315177561715e-05,
+      "learning_rate": 0.00019999999990313622,
+      "loss": 46.0,
+      "step": 97
+    },
+    {
+      "epoch": 0.015781633721164298,
+      "grad_norm": 5.5819342378526926e-05,
+      "learning_rate": 0.00019999999990089667,
+      "loss": 46.0,
+      "step": 98
+    },
+    {
+      "epoch": 0.015942670799951687,
+      "grad_norm": 3.597228351281956e-05,
+      "learning_rate": 0.00019999999989863155,
+      "loss": 46.0,
+      "step": 99
+    },
+    {
+      "epoch": 0.01610370787873908,
+      "grad_norm": 5.974910527584143e-05,
+      "learning_rate": 0.0001999999998963408,
+      "loss": 46.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.01626474495752647,
+      "grad_norm": 5.3069412388140336e-05,
+      "learning_rate": 0.00019999999989402448,
+      "loss": 46.0,
+      "step": 101
+    },
+    {
+      "epoch": 0.01642578203631386,
+      "grad_norm": 5.418806176749058e-05,
+      "learning_rate": 0.00019999999989168252,
+      "loss": 46.0,
+      "step": 102
+    },
+    {
+      "epoch": 0.01658681911510125,
+      "grad_norm": 9.603131184121594e-05,
+      "learning_rate": 0.000199999999889315,
+      "loss": 46.0,
+      "step": 103
+    },
+    {
+      "epoch": 0.016747856193888643,
+      "grad_norm": 5.194047116674483e-05,
+      "learning_rate": 0.0001999999998869219,
+      "loss": 46.0,
+      "step": 104
+    },
+    {
+      "epoch": 0.016908893272676032,
+      "grad_norm": 9.761226829141378e-05,
+      "learning_rate": 0.00019999999988450315,
+      "loss": 46.0,
+      "step": 105
+    },
+    {
+      "epoch": 0.017069930351463425,
+      "grad_norm": 6.508623482659459e-05,
+      "learning_rate": 0.00019999999988205884,
+      "loss": 46.0,
+      "step": 106
+    },
+    {
+      "epoch": 0.017230967430250814,
+      "grad_norm": 5.57337116333656e-05,
+      "learning_rate": 0.00019999999987958895,
+      "loss": 46.0,
+      "step": 107
+    },
+    {
+      "epoch": 0.017392004509038207,
+      "grad_norm": 0.00010947446571663022,
+      "learning_rate": 0.00019999999987709344,
+      "loss": 46.0,
+      "step": 108
+    },
+    {
+      "epoch": 0.017553041587825596,
+      "grad_norm": 5.486145164468326e-05,
+      "learning_rate": 0.00019999999987457235,
+      "loss": 46.0,
+      "step": 109
+    },
+    {
+      "epoch": 0.017714078666612988,
+      "grad_norm": 0.00012351380428299308,
+      "learning_rate": 0.00019999999987202566,
+      "loss": 46.0,
+      "step": 110
+    },
+    {
+      "epoch": 0.017875115745400377,
+      "grad_norm": 0.00010108689457410946,
+      "learning_rate": 0.00019999999986945337,
+      "loss": 46.0,
+      "step": 111
+    },
+    {
+      "epoch": 0.01803615282418777,
+      "grad_norm": 8.059964602580294e-05,
+      "learning_rate": 0.00019999999986685548,
+      "loss": 46.0,
+      "step": 112
+    },
+    {
+      "epoch": 0.01819718990297516,
+      "grad_norm": 0.00013831921387463808,
+      "learning_rate": 0.00019999999986423204,
+      "loss": 46.0,
+      "step": 113
+    },
+    {
+      "epoch": 0.01835822698176255,
+      "grad_norm": 7.420378824463114e-05,
+      "learning_rate": 0.00019999999986158295,
+      "loss": 46.0,
+      "step": 114
+    },
+    {
+      "epoch": 0.01851926406054994,
+      "grad_norm": 4.6814933739369735e-05,
+      "learning_rate": 0.00019999999985890828,
+      "loss": 46.0,
+      "step": 115
+    },
+    {
+      "epoch": 0.018680301139337333,
+      "grad_norm": 9.741364192450419e-05,
+      "learning_rate": 0.00019999999985620802,
+      "loss": 46.0,
+      "step": 116
+    },
+    {
+      "epoch": 0.018841338218124722,
+      "grad_norm": 6.949760427232832e-05,
+      "learning_rate": 0.0001999999998534822,
+      "loss": 46.0,
+      "step": 117
+    },
+    {
+      "epoch": 0.019002375296912115,
+      "grad_norm": 6.653879972873256e-05,
+      "learning_rate": 0.00019999999985073074,
+      "loss": 46.0,
+      "step": 118
+    },
+    {
+      "epoch": 0.019163412375699504,
+      "grad_norm": 0.0001317582791671157,
+      "learning_rate": 0.00019999999984795367,
+      "loss": 46.0,
+      "step": 119
+    },
+    {
+      "epoch": 0.019324449454486897,
+      "grad_norm": 8.747701940592378e-05,
+      "learning_rate": 0.00019999999984515106,
+      "loss": 46.0,
+      "step": 120
+    },
+    {
+      "epoch": 0.019485486533274286,
+      "grad_norm": 0.00013079583004582673,
+      "learning_rate": 0.0001999999998423228,
+      "loss": 46.0,
+      "step": 121
+    },
+    {
+      "epoch": 0.019646523612061678,
+      "grad_norm": 8.933498611440882e-05,
+      "learning_rate": 0.000199999999839469,
+      "loss": 46.0,
+      "step": 122
+    },
+    {
+      "epoch": 0.019807560690849067,
+      "grad_norm": 8.507933671353385e-05,
+      "learning_rate": 0.00019999999983658957,
+      "loss": 46.0,
+      "step": 123
+    },
+    {
+      "epoch": 0.01996859776963646,
+      "grad_norm": 0.00011648085637716576,
+      "learning_rate": 0.00019999999983368456,
+      "loss": 46.0,
+      "step": 124
+    },
+    {
+      "epoch": 0.02012963484842385,
+      "grad_norm": 0.00012115373829146847,
+      "learning_rate": 0.00019999999983075394,
+      "loss": 46.0,
+      "step": 125
+    },
+    {
+      "epoch": 0.02029067192721124,
+      "grad_norm": 0.00017015583580359817,
+      "learning_rate": 0.00019999999982779772,
+      "loss": 46.0,
+      "step": 126
+    },
+    {
+      "epoch": 0.02045170900599863,
+      "grad_norm": 6.490958912763745e-05,
+      "learning_rate": 0.00019999999982481592,
+      "loss": 46.0,
+      "step": 127
+    },
+    {
+      "epoch": 0.020612746084786023,
+      "grad_norm": 0.00011056676157750189,
+      "learning_rate": 0.00019999999982180854,
+      "loss": 46.0,
+      "step": 128
+    },
+    {
+      "epoch": 0.020773783163573412,
+      "grad_norm": 8.274184801848605e-05,
+      "learning_rate": 0.00019999999981877556,
+      "loss": 46.0,
+      "step": 129
+    },
+    {
+      "epoch": 0.020934820242360805,
+      "grad_norm": 7.937616464914754e-05,
+      "learning_rate": 0.00019999999981571695,
+      "loss": 46.0,
+      "step": 130
+    },
+    {
+      "epoch": 0.021095857321148194,
+      "grad_norm": 0.0001607205340405926,
+      "learning_rate": 0.00019999999981263277,
+      "loss": 46.0,
+      "step": 131
+    },
+    {
+      "epoch": 0.021256894399935587,
+      "grad_norm": 7.551934686489403e-05,
+      "learning_rate": 0.000199999999809523,
+      "loss": 46.0,
+      "step": 132
+    },
+    {
+      "epoch": 0.021417931478722976,
+      "grad_norm": 0.00015242550580296665,
+      "learning_rate": 0.00019999999980638763,
+      "loss": 46.0,
+      "step": 133
+    },
+    {
+      "epoch": 0.021578968557510368,
+      "grad_norm": 6.205275713000447e-05,
+      "learning_rate": 0.00019999999980322667,
+      "loss": 46.0,
+      "step": 134
+    },
+    {
+      "epoch": 0.021740005636297757,
+      "grad_norm": 0.00013843170017935336,
+      "learning_rate": 0.0001999999998000401,
+      "loss": 46.0,
+      "step": 135
+    },
+    {
+      "epoch": 0.02190104271508515,
+      "grad_norm": 8.66490590851754e-05,
+      "learning_rate": 0.00019999999979682795,
+      "loss": 46.0,
+      "step": 136
+    },
+    {
+      "epoch": 0.02206207979387254,
+      "grad_norm": 0.00012498012802097946,
+      "learning_rate": 0.0001999999997935902,
+      "loss": 46.0,
+      "step": 137
+    },
+    {
+      "epoch": 0.02222311687265993,
+      "grad_norm": 0.00010799634037539363,
+      "learning_rate": 0.00019999999979032683,
+      "loss": 46.0,
+      "step": 138
+    },
+    {
+      "epoch": 0.02238415395144732,
+      "grad_norm": 0.00010536867921473458,
+      "learning_rate": 0.00019999999978703793,
+      "loss": 46.0,
+      "step": 139
+    },
+    {
+      "epoch": 0.022545191030234713,
+      "grad_norm": 0.00013301914441399276,
+      "learning_rate": 0.00019999999978372336,
+      "loss": 46.0,
+      "step": 140
+    },
+    {
+      "epoch": 0.022706228109022102,
+      "grad_norm": 0.00018815284420270473,
+      "learning_rate": 0.00019999999978038323,
+      "loss": 46.0,
+      "step": 141
+    },
+    {
+      "epoch": 0.022867265187809495,
+      "grad_norm": 0.00010110333823831752,
+      "learning_rate": 0.00019999999977701752,
+      "loss": 46.0,
+      "step": 142
+    },
+    {
+      "epoch": 0.023028302266596884,
+      "grad_norm": 0.00014375909813679755,
+      "learning_rate": 0.0001999999997736262,
+      "loss": 46.0,
+      "step": 143
+    },
+    {
+      "epoch": 0.023189339345384273,
+      "grad_norm": 9.653297456679866e-05,
+      "learning_rate": 0.0001999999997702093,
+      "loss": 46.0,
+      "step": 144
+    },
+    {
+      "epoch": 0.023350376424171666,
+      "grad_norm": 0.00013295281678438187,
+      "learning_rate": 0.00019999999976676676,
+      "loss": 46.0,
+      "step": 145
+    },
+    {
+      "epoch": 0.023511413502959055,
+      "grad_norm": 0.00010267561447108164,
+      "learning_rate": 0.00019999999976329867,
+      "loss": 46.0,
+      "step": 146
+    },
+    {
+      "epoch": 0.023672450581746447,
+      "grad_norm": 0.00014794350136071444,
+      "learning_rate": 0.00019999999975980494,
+      "loss": 46.0,
+      "step": 147
+    },
+    {
+      "epoch": 0.023833487660533836,
+      "grad_norm": 0.00012542445620056242,
+      "learning_rate": 0.00019999999975628567,
+      "loss": 46.0,
+      "step": 148
+    },
+    {
+      "epoch": 0.02399452473932123,
+      "grad_norm": 0.0002975241222884506,
+      "learning_rate": 0.00019999999975274077,
+      "loss": 46.0,
+      "step": 149
+    },
+    {
+      "epoch": 0.024155561818108618,
+      "grad_norm": 7.225852459669113e-05,
+      "learning_rate": 0.0001999999997491703,
+      "loss": 46.0,
+      "step": 150
+    }
+  ],
+  "logging_steps": 1,
+  "max_steps": 6209750,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1001,
+  "save_steps": 150,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2998393208832.0,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

last-checkpoint/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99e8a6417085743ddd21ba8a724fde15fb25b12dc36033109d0875a4efc5cc8d
+size 6776