2025-04-02 09:01:09,892 - INFO: Problem Type: text_causal_language_modeling
2025-04-02 09:01:09,893 - INFO: Global random seed: 742339
2025-04-02 09:01:09,893 - INFO: Preparing the data...
2025-04-02 09:01:09,893 - INFO: Setting up automatic validation split...
2025-04-02 09:01:10,002 - INFO: Preparing train and validation data
2025-04-02 09:01:10,003 - INFO: Loading train dataset...
2025-04-02 09:01:11,291 - INFO: Stop token ids: [tensor([  27,   91, 9125,   91,   29]), tensor([   27,    91, 41681,    91,    29]), tensor([  27,   91, 9399,   91,   29])]
2025-04-02 09:01:11,318 - INFO: Loading validation dataset...
2025-04-02 09:01:12,347 - INFO: Stop token ids: [tensor([  27,   91, 9125,   91,   29]), tensor([   27,    91, 41681,    91,    29]), tensor([  27,   91, 9399,   91,   29])]
2025-04-02 09:01:12,364 - INFO: Number of observations in train dataset: 536
2025-04-02 09:01:12,364 - INFO: Number of observations in validation dataset: 11
2025-04-02 09:01:13,822 - INFO: Stop token ids: [tensor([  27,   91, 9125,   91,   29], device='cuda:0'), tensor([   27,    91, 41681,    91,    29], device='cuda:0'), tensor([  27,   91, 9399,   91,   29], device='cuda:0')]
2025-04-02 09:01:13,841 - WARNING: EOS token id not matching between config and tokenizer. Overwriting [128001, 128008, 128009] with tokenizer id 128009.
2025-04-02 09:01:13,841 - WARNING: PAD token id not matching between config and tokenizer. Overwriting None with tokenizer id 128009.
2025-04-02 09:01:13,841 - INFO: Setting pretraining_tp of model config to 1.
2025-04-02 09:01:13,862 - INFO: Using float16 for backbone
2025-04-02 09:01:13,862 - INFO: Loading meta-llama/Llama-3.1-8B-Instruct. This may take a while.
2025-04-02 09:01:18,516 - INFO: Loaded meta-llama/Llama-3.1-8B-Instruct.
2025-04-02 09:01:18,516 - INFO: Attention implementation: sdpa
2025-04-02 09:01:18,518 - WARNING: EOS token id not matching between generation config and tokenizer. Overwriting with tokenizer id.
2025-04-02 09:01:18,518 - WARNING: PAD token id not matching between generation config and tokenizer. Overwriting with tokenizer id.
2025-04-02 09:01:18,518 - INFO: Lora module names: ['q_proj', 'k_proj', 'v_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj']
2025-04-02 09:01:18,844 - INFO: Trainable parameters count: 43319296
2025-04-02 09:01:18,844 - INFO: Total parameters count: 8073580544
2025-04-02 09:01:18,844 - INFO: Trainable %: 0.5366%
2025-04-02 09:01:18,856 - INFO: Enough space available for saving model weights.Required space: 15946.21MB, Available space: 27997.20MB.
2025-04-02 09:01:19,049 - INFO: Training Epoch: 1 / 15
2025-04-02 09:01:19,049 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 09:01:20,700 - INFO: Evaluation step: 67
2025-04-02 09:01:20,892 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 09:01:22,279 - INFO: Stop token ids: [tensor([  27,   91, 9125,   91,   29]), tensor([   27,    91, 41681,    91,    29]), tensor([  27,   91, 9399,   91,   29])]
2025-04-02 09:02:11,763 - INFO: train loss: 0.31:   4%|4         | 3/67 [00:52<18:44, 17.57s/it]
2025-04-02 09:02:57,347 - INFO: train loss: 0.30:   9%|8         | 6/67 [01:38<16:26, 16.17s/it]
2025-04-02 09:03:46,122 - INFO: train loss: 0.30:  13%|#3        | 9/67 [02:27<15:40, 16.21s/it]
2025-04-02 09:04:29,677 - INFO: train loss: 0.28:  18%|#7        | 12/67 [03:10<14:14, 15.54s/it]
2025-04-02 09:05:12,250 - INFO: train loss: 0.30:  22%|##2       | 15/67 [03:53<13:02, 15.06s/it]
2025-04-02 09:05:54,767 - INFO: train loss: 0.28:  27%|##6       | 18/67 [04:35<12:03, 14.76s/it]
2025-04-02 09:06:41,586 - INFO: train loss: 0.26:  31%|###1      | 21/67 [05:22<11:31, 15.03s/it]
2025-04-02 09:07:29,758 - INFO: train loss: 0.25:  36%|###5      | 24/67 [06:10<11:00, 15.36s/it]
2025-04-02 09:08:17,146 - INFO: train loss: 0.22:  40%|####      | 27/67 [06:58<10:19, 15.50s/it]
2025-04-02 09:09:10,485 - INFO: train loss: 0.23:  45%|####4     | 30/67 [07:51<09:59, 16.20s/it]
2025-04-02 09:09:52,434 - INFO: train loss: 0.21:  49%|####9     | 33/67 [08:33<08:47, 15.52s/it]
2025-04-02 09:10:42,487 - INFO: train loss: 0.22:  54%|#####3    | 36/67 [09:23<08:12, 15.88s/it]
2025-04-02 09:11:26,894 - INFO: train loss: 0.21:  58%|#####8    | 39/67 [10:07<07:15, 15.55s/it]
2025-04-02 09:12:08,610 - INFO: train loss: 0.21:  63%|######2   | 42/67 [10:49<06:16, 15.05s/it]
2025-04-02 09:13:00,581 - INFO: train loss: 0.21:  67%|######7   | 45/67 [11:41<05:46, 15.74s/it]
2025-04-02 09:13:37,943 - INFO: train loss: 0.20:  72%|#######1  | 48/67 [12:18<04:40, 14.75s/it]
2025-04-02 09:14:18,528 - INFO: train loss: 0.19:  76%|#######6  | 51/67 [12:59<03:50, 14.38s/it]
2025-04-02 09:15:05,563 - INFO: train loss: 0.19:  81%|########  | 54/67 [13:46<03:12, 14.77s/it]
2025-04-02 09:15:50,468 - INFO: train loss: 0.18:  85%|########5 | 57/67 [14:31<02:28, 14.83s/it]
2025-04-02 09:16:35,819 - INFO: train loss: 0.19:  90%|########9 | 60/67 [15:16<01:44, 14.92s/it]
2025-04-02 09:17:28,652 - INFO: train loss: 0.20:  94%|#########4| 63/67 [16:09<01:02, 15.73s/it]
2025-04-02 09:18:12,052 - INFO: train loss: 0.19:  99%|#########8| 66/67 [16:53<00:15, 15.35s/it]
2025-04-02 09:18:29,033 - INFO: train loss: 0.19: 100%|##########| 67/67 [17:09<00:00, 15.55s/it]
2025-04-02 09:18:29,043 - INFO: Starting validation inference
2025-04-02 09:18:29,044 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 09:24:32,598 - INFO: validation progress:  50%|#####     | 1/2 [06:03<06:03, 363.55s/it]
2025-04-02 09:29:43,607 - INFO: validation progress: 100%|##########| 2/2 [11:14<00:00, 332.65s/it]
2025-04-02 09:29:43,610 - INFO: validation progress: 100%|##########| 2/2 [11:14<00:00, 337.28s/it]
2025-04-02 09:29:44,715 - INFO: Validation BLEU: 52.46163
2025-04-02 09:29:44,739 - INFO: Saving best model checkpoint: val_BLEU -inf -> 52.462 to /h2o-llmstudio/output/user/canberra.1/
2025-04-02 09:30:09,860 - INFO: train loss: 0.19: 100%|##########| 67/67 [28:50<00:00, 25.83s/it]
2025-04-02 09:30:09,870 - INFO: Training Epoch: 2 / 15
2025-04-02 09:30:09,870 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 09:30:14,175 - INFO: Evaluation step: 67
2025-04-02 09:30:14,367 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 09:31:01,837 - INFO: train loss: 0.18:   4%|4         | 3/67 [00:51<18:28, 17.32s/it]
2025-04-02 09:31:48,255 - INFO: train loss: 0.17:   9%|8         | 6/67 [01:38<16:30, 16.23s/it]
2025-04-02 09:32:37,770 - INFO: train loss: 0.18:  13%|#3        | 9/67 [02:27<15:48, 16.36s/it]
2025-04-02 09:33:23,292 - INFO: train loss: 0.18:  18%|#7        | 12/67 [03:13<14:33, 15.89s/it]
2025-04-02 09:34:02,758 - INFO: train loss: 0.17:  22%|##2       | 15/67 [03:52<12:55, 14.90s/it]
2025-04-02 09:34:43,323 - INFO: train loss: 0.17:  27%|##6       | 18/67 [04:33<11:47, 14.43s/it]
2025-04-02 09:35:25,622 - INFO: train loss: 0.17:  31%|###1      | 21/67 [05:15<10:58, 14.32s/it]
2025-04-02 09:36:12,143 - INFO: train loss: 0.18:  36%|###5      | 24/67 [06:02<10:32, 14.70s/it]
2025-04-02 09:36:59,774 - INFO: train loss: 0.18:  40%|####      | 27/67 [06:49<10:02, 15.07s/it]
2025-04-02 09:37:42,697 - INFO: train loss: 0.18:  45%|####4     | 30/67 [07:32<09:08, 14.83s/it]
2025-04-02 09:38:28,789 - INFO: train loss: 0.18:  49%|####9     | 33/67 [08:18<08:29, 15.00s/it]
2025-04-02 09:39:17,649 - INFO: train loss: 0.17:  54%|#####3    | 36/67 [09:07<07:57, 15.39s/it]
2025-04-02 09:40:05,775 - INFO: train loss: 0.19:  58%|#####8    | 39/67 [09:55<07:16, 15.59s/it]
2025-04-02 09:40:44,303 - INFO: train loss: 0.20:  63%|######2   | 42/67 [10:34<06:08, 14.76s/it]
2025-04-02 09:41:29,229 - INFO: train loss: 0.20:  67%|######7   | 45/67 [11:19<05:26, 14.82s/it]
2025-04-02 09:42:17,171 - INFO: train loss: 0.18:  72%|#######1  | 48/67 [12:07<04:48, 15.17s/it]
2025-04-02 09:43:06,742 - INFO: train loss: 0.20:  76%|#######6  | 51/67 [12:56<04:09, 15.58s/it]
2025-04-02 09:43:57,409 - INFO: train loss: 0.20:  81%|########  | 54/67 [13:47<03:27, 15.97s/it]
2025-04-02 09:44:40,852 - INFO: train loss: 0.20:  85%|########5 | 57/67 [14:30<02:35, 15.52s/it]
2025-04-02 09:45:26,464 - INFO: train loss: 0.17:  90%|########9 | 60/67 [15:16<01:47, 15.43s/it]
2025-04-02 09:46:08,537 - INFO: train loss: 0.16:  94%|#########4| 63/67 [15:58<01:00, 15.01s/it]
2025-04-02 09:46:56,984 - INFO: train loss: 0.16:  99%|#########8| 66/67 [16:47<00:15, 15.35s/it]
2025-04-02 09:47:13,056 - INFO: train loss: 0.17: 100%|##########| 67/67 [17:03<00:00, 15.44s/it]
2025-04-02 09:47:13,066 - INFO: Starting validation inference
2025-04-02 09:47:13,067 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 09:53:59,657 - INFO: validation progress:  50%|#####     | 1/2 [06:46<06:46, 406.59s/it]
2025-04-02 09:59:02,660 - INFO: validation progress: 100%|##########| 2/2 [11:49<00:00, 345.66s/it]
2025-04-02 09:59:02,662 - INFO: validation progress: 100%|##########| 2/2 [11:49<00:00, 354.80s/it]
2025-04-02 09:59:03,988 - INFO: Validation BLEU: 54.05531
2025-04-02 09:59:04,004 - INFO: Saving best model checkpoint: val_BLEU 52.462 -> 54.055 to /h2o-llmstudio/output/user/canberra.1/
2025-04-02 09:59:33,781 - INFO: train loss: 0.17: 100%|##########| 67/67 [29:23<00:00, 26.33s/it]
2025-04-02 09:59:33,802 - INFO: Training Epoch: 3 / 15
2025-04-02 09:59:33,802 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 09:59:37,966 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 09:59:37,983 - INFO: Evaluation step: 67
2025-04-02 10:00:19,808 - INFO: train loss: 0.16:   4%|4         | 3/67 [00:46<16:21, 15.34s/it]
2025-04-02 10:01:04,172 - INFO: train loss: 0.17:   9%|8         | 6/67 [01:30<15:15, 15.01s/it]
2025-04-02 10:01:56,822 - INFO: train loss: 0.17:  13%|#3        | 9/67 [02:23<15:37, 16.17s/it]
2025-04-02 10:02:39,322 - INFO: train loss: 0.17:  18%|#7        | 12/67 [03:05<14:05, 15.38s/it]
2025-04-02 10:03:18,724 - INFO: train loss: 0.15:  22%|##2       | 15/67 [03:44<12:37, 14.57s/it]
2025-04-02 10:04:02,573 - INFO: train loss: 0.14:  27%|##6       | 18/67 [04:28<11:54, 14.59s/it]
2025-04-02 10:04:51,788 - INFO: train loss: 0.14:  31%|###1      | 21/67 [05:17<11:38, 15.18s/it]
2025-04-02 10:05:33,737 - INFO: train loss: 0.14:  36%|###5      | 24/67 [05:59<10:36, 14.80s/it]
2025-04-02 10:06:17,488 - INFO: train loss: 0.16:  40%|####      | 27/67 [06:43<09:49, 14.73s/it]
2025-04-02 10:06:59,011 - INFO: train loss: 0.16:  45%|####4     | 30/67 [07:25<08:54, 14.46s/it]
2025-04-02 10:07:46,518 - INFO: train loss: 0.16:  49%|####9     | 33/67 [08:12<08:25, 14.88s/it]
2025-04-02 10:08:29,730 - INFO: train loss: 0.16:  54%|#####3    | 36/67 [08:55<07:36, 14.73s/it]
2025-04-02 10:09:12,688 - INFO: train loss: 0.16:  58%|#####8    | 39/67 [09:38<06:49, 14.61s/it]
2025-04-02 10:09:52,998 - INFO: train loss: 0.16:  63%|######2   | 42/67 [10:19<05:56, 14.25s/it]
2025-04-02 10:10:42,060 - INFO: train loss: 0.16:  67%|######7   | 45/67 [11:08<05:27, 14.89s/it]
2025-04-02 10:11:31,154 - INFO: train loss: 0.16:  72%|#######1  | 48/67 [11:57<04:51, 15.33s/it]
2025-04-02 10:12:15,893 - INFO: train loss: 0.16:  76%|#######6  | 51/67 [12:42<04:03, 15.21s/it]
2025-04-02 10:13:09,151 - INFO: train loss: 0.15:  81%|########  | 54/67 [13:35<03:27, 15.97s/it]
2025-04-02 10:14:01,094 - INFO: train loss: 0.16:  85%|########5 | 57/67 [14:27<02:43, 16.37s/it]
2025-04-02 10:14:51,034 - INFO: train loss: 0.15:  90%|########9 | 60/67 [15:17<01:55, 16.46s/it]
2025-04-02 10:15:35,370 - INFO: train loss: 0.19:  94%|#########4| 63/67 [16:01<01:03, 15.95s/it]
2025-04-02 10:16:25,878 - INFO: train loss: 0.20:  99%|#########8| 66/67 [16:52<00:16, 16.22s/it]
2025-04-02 10:16:40,728 - INFO: train loss: 0.19: 100%|##########| 67/67 [17:06<00:00, 16.05s/it]
2025-04-02 10:16:40,739 - INFO: Starting validation inference
2025-04-02 10:16:40,739 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 10:22:08,194 - INFO: validation progress:  50%|#####     | 1/2 [05:27<05:27, 327.45s/it]
2025-04-02 10:27:19,042 - INFO: validation progress: 100%|##########| 2/2 [10:38<00:00, 317.69s/it]
2025-04-02 10:27:19,045 - INFO: validation progress: 100%|##########| 2/2 [10:38<00:00, 319.15s/it]
2025-04-02 10:27:20,314 - INFO: Validation BLEU: 59.05834
2025-04-02 10:27:20,334 - INFO: Saving best model checkpoint: val_BLEU 54.055 -> 59.058 to /h2o-llmstudio/output/user/canberra.1/
2025-04-02 10:27:47,177 - INFO: train loss: 0.19: 100%|##########| 67/67 [28:13<00:00, 25.27s/it]
2025-04-02 10:27:47,187 - INFO: Training Epoch: 4 / 15
2025-04-02 10:27:47,187 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 10:27:51,187 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 10:27:51,270 - INFO: Evaluation step: 67
2025-04-02 10:28:36,058 - INFO: train loss: 0.14:   4%|4         | 3/67 [00:48<17:22, 16.29s/it]
2025-04-02 10:29:14,682 - INFO: train loss: 0.15:   9%|8         | 6/67 [01:27<14:31, 14.28s/it]
2025-04-02 10:30:00,640 - INFO: train loss: 0.14:  13%|#3        | 9/67 [02:13<14:15, 14.76s/it]
2025-04-02 10:30:43,864 - INFO: train loss: 0.14:  18%|#7        | 12/67 [02:56<13:23, 14.62s/it]
2025-04-02 10:31:24,560 - INFO: train loss: 0.14:  22%|##2       | 15/67 [03:37<12:20, 14.24s/it]
2025-04-02 10:32:11,304 - INFO: train loss: 0.14:  27%|##6       | 18/67 [04:24<12:00, 14.70s/it]
2025-04-02 10:32:56,834 - INFO: train loss: 0.16:  31%|###1      | 21/67 [05:09<11:23, 14.85s/it]
2025-04-02 10:33:46,440 - INFO: train loss: 0.16:  36%|###5      | 24/67 [05:59<11:01, 15.39s/it]
2025-04-02 10:34:29,344 - INFO: train loss: 0.17:  40%|####      | 27/67 [06:42<10:01, 15.05s/it]
2025-04-02 10:35:21,562 - INFO: train loss: 0.16:  45%|####4     | 30/67 [07:34<09:43, 15.78s/it]
2025-04-02 10:36:06,864 - INFO: train loss: 0.16:  49%|####9     | 33/67 [08:19<08:49, 15.57s/it]
2025-04-02 10:36:49,543 - INFO: train loss: 0.15:  54%|#####3    | 36/67 [09:02<07:49, 15.16s/it]
2025-04-02 10:37:37,109 - INFO: train loss: 0.16:  58%|#####8    | 39/67 [09:49<07:10, 15.37s/it]
2025-04-02 10:38:25,094 - INFO: train loss: 0.15:  63%|######2   | 42/67 [10:37<06:28, 15.56s/it]
2025-04-02 10:39:14,020 - INFO: train loss: 0.15:  67%|######7   | 45/67 [11:26<05:47, 15.79s/it]
2025-04-02 10:39:59,059 - INFO: train loss: 0.15:  72%|#######1  | 48/67 [12:11<04:55, 15.55s/it]
2025-04-02 10:40:41,847 - INFO: train loss: 0.14:  76%|#######6  | 51/67 [12:54<04:02, 15.16s/it]
2025-04-02 10:41:31,644 - INFO: train loss: 0.15:  81%|########  | 54/67 [13:44<03:22, 15.60s/it]
2025-04-02 10:42:15,141 - INFO: train loss: 0.14:  85%|########5 | 57/67 [14:27<02:32, 15.27s/it]
2025-04-02 10:42:58,507 - INFO: train loss: 0.14:  90%|########9 | 60/67 [15:11<01:45, 15.02s/it]
2025-04-02 10:43:47,986 - INFO: train loss: 0.13:  94%|#########4| 63/67 [16:00<01:01, 15.46s/it]
2025-04-02 10:44:33,687 - INFO: train loss: 0.14:  99%|#########8| 66/67 [16:46<00:15, 15.40s/it]
2025-04-02 10:44:47,325 - INFO: train loss: 0.13: 100%|##########| 67/67 [17:00<00:00, 15.18s/it]
2025-04-02 10:44:47,338 - INFO: Starting validation inference
2025-04-02 10:44:47,338 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 10:51:10,410 - INFO: validation progress:  50%|#####     | 1/2 [06:23<06:23, 383.07s/it]
2025-04-02 10:56:19,118 - INFO: validation progress: 100%|##########| 2/2 [11:31<00:00, 339.33s/it]
2025-04-02 10:56:19,120 - INFO: validation progress: 100%|##########| 2/2 [11:31<00:00, 345.89s/it]
2025-04-02 10:56:20,324 - INFO: Validation BLEU: 58.74221
2025-04-02 10:56:22,285 - INFO: train loss: 0.13: 100%|##########| 67/67 [28:35<00:00, 25.60s/it]
2025-04-02 10:56:22,300 - INFO: Training Epoch: 5 / 15
2025-04-02 10:56:22,300 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 10:56:26,474 - INFO: Evaluation step: 67
2025-04-02 10:56:26,697 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 10:57:14,867 - INFO: train loss: 0.12:   4%|4         | 3/67 [00:52<18:41, 17.52s/it]
2025-04-02 10:57:59,336 - INFO: train loss: 0.11:   9%|8         | 6/67 [01:37<16:12, 15.93s/it]
2025-04-02 10:58:44,042 - INFO: train loss: 0.12:  13%|#3        | 9/67 [02:21<14:56, 15.46s/it]
2025-04-02 10:59:36,342 - INFO: train loss: 0.12:  18%|#7        | 12/67 [03:14<14:53, 16.24s/it]
2025-04-02 11:00:14,884 - INFO: train loss: 0.13:  22%|##2       | 15/67 [03:52<13:00, 15.02s/it]
2025-04-02 11:01:04,585 - INFO: train loss: 0.14:  27%|##6       | 18/67 [04:42<12:41, 15.54s/it]
2025-04-02 11:01:51,043 - INFO: train loss: 0.14:  31%|###1      | 21/67 [05:28<11:54, 15.53s/it]
2025-04-02 11:02:31,239 - INFO: train loss: 0.15:  36%|###5      | 24/67 [06:08<10:38, 14.85s/it]
2025-04-02 11:03:14,808 - INFO: train loss: 0.15:  40%|####      | 27/67 [06:52<09:49, 14.75s/it]
2025-04-02 11:03:55,926 - INFO: train loss: 0.15:  45%|####4     | 30/67 [07:33<08:53, 14.43s/it]
2025-04-02 11:04:36,930 - INFO: train loss: 0.15:  49%|####9     | 33/67 [08:14<08:02, 14.19s/it]
2025-04-02 11:05:21,777 - INFO: train loss: 0.14:  54%|#####3    | 36/67 [08:59<07:27, 14.42s/it]
2025-04-02 11:06:03,854 - INFO: train loss: 0.14:  58%|#####8    | 39/67 [09:41<06:40, 14.30s/it]
2025-04-02 11:06:51,813 - INFO: train loss: 0.14:  63%|######2   | 42/67 [10:29<06:10, 14.81s/it]
2025-04-02 11:07:36,095 - INFO: train loss: 0.14:  67%|######7   | 45/67 [11:13<05:25, 14.80s/it]
2025-04-02 11:08:34,799 - INFO: train loss: 0.13:  72%|#######1  | 48/67 [12:12<05:08, 16.23s/it]
2025-04-02 11:09:20,933 - INFO: train loss: 0.12:  76%|#######6  | 51/67 [12:58<04:15, 15.98s/it]
2025-04-02 11:10:03,600 - INFO: train loss: 0.13:  81%|########  | 54/67 [13:41<03:20, 15.45s/it]
2025-04-02 11:10:50,199 - INFO: train loss: 0.14:  85%|########5 | 57/67 [14:27<02:34, 15.47s/it]
2025-04-02 11:11:37,218 - INFO: train loss: 0.14:  90%|########9 | 60/67 [15:14<01:48, 15.53s/it]
2025-04-02 11:12:28,857 - INFO: train loss: 0.15:  94%|#########4| 63/67 [16:06<01:04, 16.04s/it]
2025-04-02 11:13:22,201 - INFO: train loss: 0.14:  99%|#########8| 66/67 [16:59<00:16, 16.56s/it]
2025-04-02 11:13:35,187 - INFO: train loss: 0.13: 100%|##########| 67/67 [17:12<00:00, 16.11s/it]
2025-04-02 11:13:35,197 - INFO: Starting validation inference
2025-04-02 11:13:35,198 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 11:20:12,741 - INFO: validation progress:  50%|#####     | 1/2 [06:37<06:37, 397.54s/it]
2025-04-02 11:25:24,281 - INFO: validation progress: 100%|##########| 2/2 [11:49<00:00, 346.95s/it]
2025-04-02 11:25:24,284 - INFO: validation progress: 100%|##########| 2/2 [11:49<00:00, 354.54s/it]
2025-04-02 11:25:25,888 - INFO: Validation BLEU: 59.45291
2025-04-02 11:25:25,920 - INFO: Saving best model checkpoint: val_BLEU 59.058 -> 59.453 to /h2o-llmstudio/output/user/canberra.1/
2025-04-02 11:25:56,804 - INFO: train loss: 0.13: 100%|##########| 67/67 [29:34<00:00, 26.49s/it]
2025-04-02 11:25:56,813 - INFO: Training Epoch: 6 / 15
2025-04-02 11:25:56,813 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 11:26:00,970 - INFO: Evaluation step: 67
2025-04-02 11:26:00,974 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 11:26:53,872 - INFO: train loss: 0.12:   4%|4         | 3/67 [00:57<20:17, 19.02s/it]
2025-04-02 11:27:43,151 - INFO: train loss: 0.13:   9%|8         | 6/67 [01:46<17:47, 17.49s/it]
2025-04-02 11:28:29,402 - INFO: train loss: 0.12:  13%|#3        | 9/67 [02:32<15:59, 16.55s/it]
2025-04-02 11:29:15,037 - INFO: train loss: 0.12:  18%|#7        | 12/67 [03:18<14:41, 16.02s/it]
2025-04-02 11:30:00,480 - INFO: train loss: 0.12:  22%|##2       | 15/67 [04:03<13:36, 15.70s/it]
2025-04-02 11:30:42,454 - INFO: train loss: 0.12:  27%|##6       | 18/67 [04:45<12:20, 15.12s/it]
2025-04-02 11:31:20,686 - INFO: train loss: 0.12:  31%|###1      | 21/67 [05:23<10:59, 14.34s/it]
2025-04-02 11:32:10,572 - INFO: train loss: 0.13:  36%|###5      | 24/67 [06:13<10:48, 15.07s/it]
2025-04-02 11:32:53,635 - INFO: train loss: 0.13:  40%|####      | 27/67 [06:56<09:53, 14.85s/it]
2025-04-02 11:33:36,281 - INFO: train loss: 0.13:  45%|####4     | 30/67 [07:39<09:02, 14.65s/it]
2025-04-02 11:34:29,406 - INFO: train loss: 0.12:  49%|####9     | 33/67 [08:32<08:49, 15.59s/it]
2025-04-02 11:35:22,471 - INFO: train loss: 0.12:  54%|#####3    | 36/67 [09:25<08:23, 16.23s/it]
2025-04-02 11:36:06,853 - INFO: train loss: 0.12:  58%|#####8    | 39/67 [10:10<07:22, 15.79s/it]
2025-04-02 11:36:57,961 - INFO: train loss: 0.13:  63%|######2   | 42/67 [11:01<06:44, 16.17s/it]
2025-04-02 11:37:40,687 - INFO: train loss: 0.13:  67%|######7   | 45/67 [11:43<05:42, 15.59s/it]
2025-04-02 11:38:33,248 - INFO: train loss: 0.13:  72%|#######1  | 48/67 [12:36<05:07, 16.17s/it]
2025-04-02 11:39:20,125 - INFO: train loss: 0.13:  76%|#######6  | 51/67 [13:23<04:16, 16.01s/it]
2025-04-02 11:40:01,241 - INFO: train loss: 0.12:  81%|########  | 54/67 [14:04<03:19, 15.31s/it]
2025-04-02 11:40:40,447 - INFO: train loss: 0.12:  85%|########5 | 57/67 [14:43<02:26, 14.64s/it]
2025-04-02 11:41:23,390 - INFO: train loss: 0.11:  90%|########9 | 60/67 [15:26<01:41, 14.54s/it]
2025-04-02 11:42:08,997 - INFO: train loss: 0.11:  94%|#########4| 63/67 [16:12<00:58, 14.74s/it]
2025-04-02 11:42:51,288 - INFO: train loss: 0.11:  99%|#########8| 66/67 [16:54<00:14, 14.55s/it]
2025-04-02 11:43:05,938 - INFO: train loss: 0.11: 100%|##########| 67/67 [17:09<00:00, 14.56s/it]
2025-04-02 11:43:05,949 - INFO: Starting validation inference
2025-04-02 11:43:05,950 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 11:49:42,094 - INFO: validation progress:  50%|#####     | 1/2 [06:36<06:36, 396.14s/it]
2025-04-02 11:54:52,522 - INFO: validation progress: 100%|##########| 2/2 [11:46<00:00, 345.72s/it]
2025-04-02 11:54:52,524 - INFO: validation progress: 100%|##########| 2/2 [11:46<00:00, 353.29s/it]
2025-04-02 11:54:53,899 - INFO: Validation BLEU: 59.43863
2025-04-02 11:54:56,187 - INFO: train loss: 0.11: 100%|##########| 67/67 [28:59<00:00, 25.96s/it]
2025-04-02 11:54:56,196 - INFO: Training Epoch: 7 / 15
2025-04-02 11:54:56,196 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 11:55:00,189 - INFO: Evaluation step: 67
2025-04-02 11:55:00,373 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 11:55:44,478 - INFO: train loss: 0.12:   4%|4         | 3/67 [00:48<17:09, 16.09s/it]
2025-04-02 11:56:28,346 - INFO: train loss: 0.11:   9%|8         | 6/67 [01:32<15:28, 15.23s/it]
2025-04-02 11:57:17,167 - INFO: train loss: 0.11:  13%|#3        | 9/67 [02:20<15:10, 15.71s/it]
2025-04-02 11:58:00,723 - INFO: train loss: 0.12:  18%|#7        | 12/67 [03:04<13:58, 15.24s/it]
2025-04-02 11:58:52,623 - INFO: train loss: 0.12:  22%|##2       | 15/67 [03:56<13:51, 15.98s/it]
2025-04-02 11:59:39,591 - INFO: train loss: 0.12:  27%|##6       | 18/67 [04:43<12:57, 15.87s/it]
2025-04-02 12:00:31,810 - INFO: train loss: 0.11:  31%|###1      | 21/67 [05:35<12:33, 16.37s/it]
2025-04-02 12:01:09,679 - INFO: train loss: 0.11:  36%|###5      | 24/67 [06:13<10:52, 15.18s/it]
2025-04-02 12:01:52,004 - INFO: train loss: 0.10:  40%|####      | 27/67 [06:55<09:53, 14.84s/it]
2025-04-02 12:02:37,039 - INFO: train loss: 0.10:  45%|####4     | 30/67 [07:40<09:11, 14.90s/it]
2025-04-02 12:03:27,579 - INFO: train loss: 0.10:  49%|####9     | 33/67 [08:31<08:46, 15.49s/it]
2025-04-02 12:04:13,193 - INFO: train loss: 0.10:  54%|#####3    | 36/67 [09:16<07:57, 15.41s/it]
2025-04-02 12:04:59,580 - INFO: train loss: 0.11:  58%|#####8    | 39/67 [10:03<07:11, 15.42s/it]
2025-04-02 12:05:41,997 - INFO: train loss: 0.11:  63%|######2   | 42/67 [10:45<06:15, 15.03s/it]
2025-04-02 12:06:20,402 - INFO: train loss: 0.11:  67%|######7   | 45/67 [11:24<05:15, 14.36s/it]
2025-04-02 12:07:05,870 - INFO: train loss: 0.11:  72%|#######1  | 48/67 [12:09<04:37, 14.60s/it]
2025-04-02 12:07:59,074 - INFO: train loss: 0.10:  76%|#######6  | 51/67 [13:02<04:08, 15.54s/it]
2025-04-02 12:08:42,419 - INFO: train loss: 0.12:  81%|########  | 54/67 [13:46<03:17, 15.21s/it]
2025-04-02 12:09:25,026 - INFO: train loss: 0.12:  85%|########5 | 57/67 [14:28<02:29, 14.91s/it]
2025-04-02 12:10:14,668 - INFO: train loss: 0.12:  90%|########9 | 60/67 [15:18<01:47, 15.40s/it]
2025-04-02 12:11:05,739 - INFO: train loss: 0.12:  94%|#########4| 63/67 [16:09<01:03, 15.89s/it]
2025-04-02 12:11:56,036 - INFO: train loss: 0.13:  99%|#########8| 66/67 [16:59<00:16, 16.15s/it]
2025-04-02 12:12:13,070 - INFO: train loss: 0.14: 100%|##########| 67/67 [17:16<00:00, 16.26s/it]
2025-04-02 12:12:13,080 - INFO: Starting validation inference
2025-04-02 12:12:13,081 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 12:18:51,852 - INFO: validation progress:  50%|#####     | 1/2 [06:38<06:38, 398.77s/it]
2025-04-02 12:24:07,951 - INFO: validation progress: 100%|##########| 2/2 [11:54<00:00, 350.14s/it]
2025-04-02 12:24:07,953 - INFO: validation progress: 100%|##########| 2/2 [11:54<00:00, 357.44s/it]
2025-04-02 12:24:09,421 - INFO: Validation BLEU: 60.23522
2025-04-02 12:24:09,442 - INFO: Saving best model checkpoint: val_BLEU 59.453 -> 60.235 to /h2o-llmstudio/output/user/canberra.1/
2025-04-02 12:24:36,583 - INFO: train loss: 0.14: 100%|##########| 67/67 [29:40<00:00, 26.57s/it]
2025-04-02 12:24:36,595 - INFO: Training Epoch: 8 / 15
2025-04-02 12:24:36,596 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 12:24:41,077 - INFO: Evaluation step: 67
2025-04-02 12:24:41,399 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 12:25:20,529 - INFO: train loss: 0.10:   4%|4         | 3/67 [00:43<15:37, 14.64s/it]
2025-04-02 12:26:05,307 - INFO: train loss: 0.11:   9%|8         | 6/67 [01:28<15:03, 14.81s/it]
2025-04-02 12:26:48,220 - INFO: train loss: 0.10:  13%|#3        | 9/67 [02:11<14:05, 14.58s/it]
2025-04-02 12:27:40,379 - INFO: train loss: 0.11:  18%|#7        | 12/67 [03:03<14:22, 15.69s/it]
2025-04-02 12:28:32,587 - INFO: train loss: 0.11:  22%|##2       | 15/67 [03:55<14:07, 16.31s/it]
2025-04-02 12:29:23,000 - INFO: train loss: 0.12:  27%|##6       | 18/67 [04:46<13:27, 16.48s/it]
2025-04-02 12:30:04,507 - INFO: train loss: 0.12:  31%|###1      | 21/67 [05:27<11:58, 15.61s/it]
2025-04-02 12:30:45,741 - INFO: train loss: 0.11:  36%|###5      | 24/67 [06:09<10:45, 15.02s/it]
2025-04-02 12:31:34,880 - INFO: train loss: 0.11:  40%|####      | 27/67 [06:58<10:17, 15.44s/it]
2025-04-02 12:32:19,097 - INFO: train loss: 0.11:  45%|####4     | 30/67 [07:42<09:23, 15.23s/it]
2025-04-02 12:33:00,024 - INFO: train loss: 0.11:  49%|####9     | 33/67 [08:23<08:21, 14.74s/it]
2025-04-02 12:33:55,105 - INFO: train loss: 0.10:  54%|#####3    | 36/67 [09:18<08:11, 15.84s/it]
2025-04-02 12:34:47,170 - INFO: train loss: 0.10:  58%|#####8    | 39/67 [10:10<07:36, 16.30s/it]
2025-04-02 12:35:30,057 - INFO: train loss: 0.10:  63%|######2   | 42/67 [10:53<06:32, 15.69s/it]
2025-04-02 12:36:16,299 - INFO: train loss: 0.11:  67%|######7   | 45/67 [11:39<05:43, 15.61s/it]
2025-04-02 12:37:01,222 - INFO: train loss: 0.11:  72%|#######1  | 48/67 [12:24<04:52, 15.42s/it]
2025-04-02 12:37:51,335 - INFO: train loss: 0.10:  76%|#######6  | 51/67 [13:14<04:12, 15.81s/it]
2025-04-02 12:38:32,343 - INFO: train loss: 0.10:  81%|########  | 54/67 [13:55<03:17, 15.16s/it]
2025-04-02 12:39:23,937 - INFO: train loss: 0.10:  85%|########5 | 57/67 [14:47<02:37, 15.77s/it]
2025-04-02 12:40:04,810 - INFO: train loss: 0.09:  90%|########9 | 60/67 [15:28<01:45, 15.13s/it]
2025-04-02 12:40:49,173 - INFO: train loss: 0.10:  94%|#########4| 63/67 [16:12<01:00, 15.03s/it]
2025-04-02 12:41:35,315 - INFO: train loss: 0.10:  99%|#########8| 66/67 [16:58<00:15, 15.13s/it]
2025-04-02 12:41:48,136 - INFO: train loss: 0.10: 100%|##########| 67/67 [17:11<00:00, 14.84s/it]
2025-04-02 12:41:48,148 - INFO: Starting validation inference
2025-04-02 12:41:48,148 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 12:47:45,316 - INFO: validation progress:  50%|#####     | 1/2 [05:57<05:57, 357.17s/it]
2025-04-02 12:52:58,270 - INFO: validation progress: 100%|##########| 2/2 [11:10<00:00, 331.16s/it]
2025-04-02 12:52:58,273 - INFO: validation progress: 100%|##########| 2/2 [11:10<00:00, 335.06s/it]
2025-04-02 12:52:59,475 - INFO: Validation BLEU: 57.96725
2025-04-02 12:53:01,255 - INFO: train loss: 0.10: 100%|##########| 67/67 [28:24<00:00, 25.44s/it]
2025-04-02 12:53:01,267 - INFO: Training Epoch: 9 / 15
2025-04-02 12:53:01,267 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 12:53:06,379 - INFO: Evaluation step: 67
2025-04-02 12:53:06,468 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 12:53:45,675 - INFO: train loss: 0.09:   4%|4         | 3/67 [00:44<15:47, 14.80s/it]
2025-04-02 12:54:32,771 - INFO: train loss: 0.10:   9%|8         | 6/67 [01:31<15:35, 15.33s/it]
2025-04-02 12:55:20,749 - INFO: train loss: 0.10:  13%|#3        | 9/67 [02:19<15:06, 15.63s/it]
2025-04-02 12:56:04,861 - INFO: train loss: 0.10:  18%|#7        | 12/67 [03:03<13:59, 15.27s/it]
2025-04-02 12:56:45,607 - INFO: train loss: 0.09:  22%|##2       | 15/67 [03:44<12:42, 14.66s/it]
2025-04-02 12:57:28,278 - INFO: train loss: 0.09:  27%|##6       | 18/67 [04:27<11:51, 14.51s/it]
2025-04-02 12:58:24,713 - INFO: train loss: 0.09:  31%|###1      | 21/67 [05:23<12:12, 15.92s/it]
2025-04-02 12:59:01,921 - INFO: train loss: 0.09:  36%|###5      | 24/67 [06:00<10:36, 14.80s/it]
2025-04-02 12:59:43,348 - INFO: train loss: 0.09:  40%|####      | 27/67 [06:42<09:39, 14.49s/it]
2025-04-02 13:00:30,835 - INFO: train loss: 0.09:  45%|####4     | 30/67 [07:29<09:11, 14.90s/it]
2025-04-02 13:01:17,068 - INFO: train loss: 0.09:  49%|####9     | 33/67 [08:15<08:31, 15.06s/it]
2025-04-02 13:01:56,996 - INFO: train loss: 0.09:  54%|#####3    | 36/67 [08:55<07:30, 14.53s/it]
2025-04-02 13:02:44,903 - INFO: train loss: 0.09:  58%|#####8    | 39/67 [09:43<06:58, 14.96s/it]
2025-04-02 13:03:30,953 - INFO: train loss: 0.09:  63%|######2   | 42/67 [10:29<06:17, 15.08s/it]
2025-04-02 13:04:15,944 - INFO: train loss: 0.10:  67%|######7   | 45/67 [11:14<05:31, 15.05s/it]
2025-04-02 13:05:03,877 - INFO: train loss: 0.10:  72%|#######1  | 48/67 [12:02<04:51, 15.33s/it]
2025-04-02 13:05:52,858 - INFO: train loss: 0.10:  76%|#######6  | 51/67 [12:51<04:10, 15.63s/it]
2025-04-02 13:06:38,113 - INFO: train loss: 0.10:  81%|########  | 54/67 [13:36<03:21, 15.47s/it]
2025-04-02 13:07:28,076 - INFO: train loss: 0.10:  85%|########5 | 57/67 [14:26<02:38, 15.82s/it]
2025-04-02 13:08:17,588 - INFO: train loss: 0.09:  90%|########9 | 60/67 [15:16<01:52, 16.03s/it]
2025-04-02 13:09:02,535 - INFO: train loss: 0.10:  94%|#########4| 63/67 [16:01<01:02, 15.71s/it]
2025-04-02 13:09:52,782 - INFO: train loss: 0.10:  99%|#########8| 66/67 [16:51<00:16, 16.02s/it]
2025-04-02 13:10:08,471 - INFO: train loss: 0.10: 100%|##########| 67/67 [17:07<00:00, 15.98s/it]
2025-04-02 13:10:08,482 - INFO: Starting validation inference
2025-04-02 13:10:08,482 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 13:16:02,551 - INFO: validation progress:  50%|#####     | 1/2 [05:54<05:54, 354.07s/it]
2025-04-02 13:21:12,917 - INFO: validation progress: 100%|##########| 2/2 [11:04<00:00, 328.36s/it]
2025-04-02 13:21:12,919 - INFO: validation progress: 100%|##########| 2/2 [11:04<00:00, 332.22s/it]
2025-04-02 13:21:14,200 - INFO: Validation BLEU: 56.93096
2025-04-02 13:21:15,503 - INFO: train loss: 0.10: 100%|##########| 67/67 [28:14<00:00, 25.29s/it]
2025-04-02 13:21:15,511 - INFO: Training Epoch: 10 / 15
2025-04-02 13:21:15,511 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 13:21:19,766 - INFO: Evaluation step: 67
2025-04-02 13:21:19,778 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 13:22:04,321 - INFO: train loss: 0.09:   4%|4         | 3/67 [00:48<17:21, 16.27s/it]
2025-04-02 13:22:54,898 - INFO: train loss: 0.09:   9%|8         | 6/67 [01:39<16:53, 16.62s/it]
2025-04-02 13:23:43,965 - INFO: train loss: 0.09:  13%|#3        | 9/67 [02:28<15:56, 16.50s/it]
2025-04-02 13:24:28,499 - INFO: train loss: 0.09:  18%|#7        | 12/67 [03:12<14:31, 15.84s/it]
2025-04-02 13:25:10,433 - INFO: train loss: 0.09:  22%|##2       | 15/67 [03:54<13:08, 15.17s/it]
2025-04-02 13:25:54,835 - INFO: train loss: 0.09:  27%|##6       | 18/67 [04:39<12:17, 15.05s/it]
2025-04-02 13:26:37,463 - INFO: train loss: 0.08:  31%|###1      | 21/67 [05:21<11:19, 14.77s/it]
2025-04-02 13:27:29,123 - INFO: train loss: 0.09:  36%|###5      | 24/67 [06:13<11:08, 15.55s/it]
2025-04-02 13:28:10,180 - INFO: train loss: 0.09:  40%|####      | 27/67 [06:54<09:58, 14.97s/it]
2025-04-02 13:28:57,348 - INFO: train loss: 0.09:  45%|####4     | 30/67 [07:41<09:22, 15.20s/it]
2025-04-02 13:29:38,270 - INFO: train loss: 0.09:  49%|####9     | 33/67 [08:22<08:20, 14.72s/it]
2025-04-02 13:30:20,146 - INFO: train loss: 0.09:  54%|#####3    | 36/67 [09:04<07:29, 14.49s/it]
2025-04-02 13:31:02,008 - INFO: train loss: 0.09:  58%|#####8    | 39/67 [09:46<06:41, 14.33s/it]
2025-04-02 13:31:51,169 - INFO: train loss: 0.08:  63%|######2   | 42/67 [10:35<06:13, 14.95s/it]
2025-04-02 13:32:34,614 - INFO: train loss: 0.08:  67%|######7   | 45/67 [11:19<05:25, 14.81s/it]
2025-04-02 13:33:22,972 - INFO: train loss: 0.09:  72%|#######1  | 48/67 [12:07<04:48, 15.20s/it]
2025-04-02 13:34:08,132 - INFO: train loss: 0.09:  76%|#######6  | 51/67 [12:52<04:02, 15.16s/it]
2025-04-02 13:34:55,213 - INFO: train loss: 0.09:  81%|########  | 54/67 [13:39<03:19, 15.32s/it]
2025-04-02 13:35:46,617 - INFO: train loss: 0.08:  85%|########5 | 57/67 [14:31<02:38, 15.86s/it]
2025-04-02 13:36:34,551 - INFO: train loss: 0.09:  90%|########9 | 60/67 [15:19<01:51, 15.90s/it]
2025-04-02 13:37:15,140 - INFO: train loss: 0.09:  94%|#########4| 63/67 [15:59<01:00, 15.19s/it]
2025-04-02 13:38:08,842 - INFO: train loss: 0.10:  99%|#########8| 66/67 [16:53<00:16, 16.00s/it]
2025-04-02 13:38:24,734 - INFO: train loss: 0.10: 100%|##########| 67/67 [17:09<00:00, 15.99s/it]
2025-04-02 13:38:24,744 - INFO: Starting validation inference
2025-04-02 13:38:24,744 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 13:44:21,662 - INFO: validation progress:  50%|#####     | 1/2 [05:56<05:56, 356.92s/it]
2025-04-02 13:49:32,948 - INFO: validation progress: 100%|##########| 2/2 [11:08<00:00, 330.08s/it]
2025-04-02 13:49:32,951 - INFO: validation progress: 100%|##########| 2/2 [11:08<00:00, 334.10s/it]
2025-04-02 13:49:34,629 - INFO: Validation BLEU: 58.60959
2025-04-02 13:49:36,589 - INFO: train loss: 0.10: 100%|##########| 67/67 [28:21<00:00, 25.39s/it]
2025-04-02 13:49:36,598 - INFO: Training Epoch: 11 / 15
2025-04-02 13:49:36,598 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 13:49:41,476 - INFO: Evaluation step: 67
2025-04-02 13:49:41,575 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 13:50:25,622 - INFO: train loss: 0.08:   4%|4         | 3/67 [00:49<17:25, 16.34s/it]
2025-04-02 13:51:09,320 - INFO: train loss: 0.08:   9%|8         | 6/67 [01:32<15:33, 15.30s/it]
2025-04-02 13:51:53,063 - INFO: train loss: 0.08:  13%|#3        | 9/67 [02:16<14:28, 14.97s/it]
2025-04-02 13:52:38,162 - INFO: train loss: 0.08:  18%|#7        | 12/67 [03:01<13:44, 14.99s/it]
2025-04-02 13:53:21,567 - INFO: train loss: 0.08:  22%|##2       | 15/67 [03:44<12:49, 14.80s/it]
2025-04-02 13:54:05,064 - INFO: train loss: 0.08:  27%|##6       | 18/67 [04:28<12:00, 14.70s/it]
2025-04-02 13:54:56,500 - INFO: train loss: 0.07:  31%|###1      | 21/67 [05:19<11:53, 15.50s/it]
2025-04-02 13:55:39,680 - INFO: train loss: 0.08:  36%|###5      | 24/67 [06:03<10:51, 15.15s/it]
2025-04-02 13:56:28,005 - INFO: train loss: 0.08:  40%|####      | 27/67 [06:51<10:17, 15.45s/it]
2025-04-02 13:57:13,166 - INFO: train loss: 0.08:  45%|####4     | 30/67 [07:36<09:27, 15.33s/it]
2025-04-02 13:57:57,191 - INFO: train loss: 0.08:  49%|####9     | 33/67 [08:20<08:34, 15.13s/it]
2025-04-02 13:58:47,980 - INFO: train loss: 0.08:  54%|#####3    | 36/67 [09:11<08:05, 15.68s/it]
2025-04-02 13:59:33,573 - INFO: train loss: 0.08:  58%|#####8    | 39/67 [09:56<07:14, 15.53s/it]
2025-04-02 14:00:11,912 - INFO: train loss: 0.09:  63%|######2   | 42/67 [10:35<06:07, 14.70s/it]
2025-04-02 14:00:53,504 - INFO: train loss: 0.09:  67%|######7   | 45/67 [11:16<05:17, 14.45s/it]
2025-04-02 14:01:49,439 - INFO: train loss: 0.09:  72%|#######1  | 48/67 [12:12<04:58, 15.71s/it]
2025-04-02 14:02:36,474 - INFO: train loss: 0.09:  76%|#######6  | 51/67 [12:59<04:11, 15.70s/it]
2025-04-02 14:03:17,121 - INFO: train loss: 0.09:  81%|########  | 54/67 [13:40<03:15, 15.05s/it]
2025-04-02 14:04:02,227 - INFO: train loss: 0.08:  85%|########5 | 57/67 [14:25<02:30, 15.05s/it]
2025-04-02 14:04:46,304 - INFO: train loss: 0.08:  90%|########9 | 60/67 [15:09<01:44, 14.94s/it]
2025-04-02 14:05:36,398 - INFO: train loss: 0.08:  94%|#########4| 63/67 [15:59<01:01, 15.47s/it]
2025-04-02 14:06:19,358 - INFO: train loss: 0.08:  99%|#########8| 66/67 [16:42<00:15, 15.12s/it]
2025-04-02 14:06:33,962 - INFO: train loss: 0.08: 100%|##########| 67/67 [16:57<00:00, 15.06s/it]
2025-04-02 14:06:33,972 - INFO: Starting validation inference
2025-04-02 14:06:33,973 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 14:11:57,303 - INFO: validation progress:  50%|#####     | 1/2 [05:23<05:23, 323.33s/it]
2025-04-02 14:17:03,984 - INFO: validation progress: 100%|##########| 2/2 [10:30<00:00, 313.54s/it]
2025-04-02 14:17:03,987 - INFO: validation progress: 100%|##########| 2/2 [10:30<00:00, 315.01s/it]
2025-04-02 14:17:05,611 - INFO: Validation BLEU: 59.98899
2025-04-02 14:17:07,497 - INFO: train loss: 0.08: 100%|##########| 67/67 [27:30<00:00, 24.64s/it]
2025-04-02 14:17:07,502 - INFO: Training Epoch: 12 / 15
2025-04-02 14:17:07,502 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 14:17:11,567 - INFO: Evaluation step: 67
2025-04-02 14:17:11,792 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 14:18:01,871 - INFO: train loss: 0.08:   4%|4         | 3/67 [00:54<19:19, 18.12s/it]
2025-04-02 14:18:44,536 - INFO: train loss: 0.07:   9%|8         | 6/67 [01:37<16:05, 15.83s/it]
2025-04-02 14:19:33,946 - INFO: train loss: 0.08:  13%|#3        | 9/67 [02:26<15:35, 16.12s/it]
2025-04-02 14:20:25,645 - INFO: train loss: 0.07:  18%|#7        | 12/67 [03:18<15:10, 16.56s/it]
2025-04-02 14:21:13,855 - INFO: train loss: 0.07:  22%|##2       | 15/67 [04:06<14:11, 16.38s/it]
2025-04-02 14:22:03,689 - INFO: train loss: 0.07:  27%|##6       | 18/67 [04:56<13:26, 16.46s/it]
2025-04-02 14:22:50,685 - INFO: train loss: 0.08:  31%|###1      | 21/67 [05:43<12:25, 16.20s/it]
2025-04-02 14:23:39,063 - INFO: train loss: 0.08:  36%|###5      | 24/67 [06:31<11:35, 16.18s/it]
2025-04-02 14:24:24,119 - INFO: train loss: 0.08:  40%|####      | 27/67 [07:16<10:32, 15.81s/it]
2025-04-02 14:25:11,385 - INFO: train loss: 0.08:  45%|####4     | 30/67 [08:03<09:44, 15.80s/it]
2025-04-02 14:26:06,249 - INFO: train loss: 0.08:  49%|####9     | 33/67 [08:58<09:23, 16.56s/it]
2025-04-02 14:26:56,104 - INFO: train loss: 0.08:  54%|#####3    | 36/67 [09:48<08:33, 16.58s/it]
2025-04-02 14:27:44,336 - INFO: train loss: 0.08:  58%|#####8    | 39/67 [10:36<07:39, 16.43s/it]
2025-04-02 14:28:27,595 - INFO: train loss: 0.07:  63%|######2   | 42/67 [11:20<06:35, 15.82s/it]
2025-04-02 14:29:10,518 - INFO: train loss: 0.07:  67%|######7   | 45/67 [12:03<05:38, 15.36s/it]
2025-04-02 14:29:59,137 - INFO: train loss: 0.08:  72%|#######1  | 48/67 [12:51<04:56, 15.62s/it]
2025-04-02 14:30:38,249 - INFO: train loss: 0.08:  76%|#######6  | 51/67 [13:30<03:57, 14.84s/it]
2025-04-02 14:31:23,428 - INFO: train loss: 0.08:  81%|########  | 54/67 [14:15<03:13, 14.91s/it]
2025-04-02 14:32:06,150 - INFO: train loss: 0.08:  85%|########5 | 57/67 [14:58<02:27, 14.71s/it]
2025-04-02 14:32:50,323 - INFO: train loss: 0.07:  90%|########9 | 60/67 [15:42<01:42, 14.71s/it]
2025-04-02 14:33:34,963 - INFO: train loss: 0.07:  94%|#########4| 63/67 [16:27<00:59, 14.76s/it]
2025-04-02 14:34:16,551 - INFO: train loss: 0.07:  99%|#########8| 66/67 [17:09<00:14, 14.49s/it]
2025-04-02 14:34:29,648 - INFO: train loss: 0.07: 100%|##########| 67/67 [17:22<00:00, 14.32s/it]
2025-04-02 14:34:29,659 - INFO: Starting validation inference
2025-04-02 14:34:29,659 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 14:39:53,001 - INFO: validation progress:  50%|#####     | 1/2 [05:23<05:23, 323.34s/it]
2025-04-02 14:45:00,074 - INFO: validation progress: 100%|##########| 2/2 [10:30<00:00, 313.77s/it]
2025-04-02 14:45:00,077 - INFO: validation progress: 100%|##########| 2/2 [10:30<00:00, 315.21s/it]
2025-04-02 14:45:01,396 - INFO: Validation BLEU: 59.02448
2025-04-02 14:45:03,880 - INFO: train loss: 0.07: 100%|##########| 67/67 [27:56<00:00, 25.02s/it]
2025-04-02 14:45:03,889 - INFO: Training Epoch: 13 / 15
2025-04-02 14:45:03,890 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 14:45:08,382 - INFO: Evaluation step: 67
2025-04-02 14:45:08,485 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 14:45:47,620 - INFO: train loss: 0.06:   4%|4         | 3/67 [00:43<15:32, 14.58s/it]
2025-04-02 14:46:26,818 - INFO: train loss: 0.06:   9%|8         | 6/67 [01:22<13:54, 13.69s/it]
2025-04-02 14:47:11,337 - INFO: train loss: 0.06:  13%|#3        | 9/67 [02:07<13:44, 14.21s/it]
2025-04-02 14:47:54,016 - INFO: train loss: 0.07:  18%|#7        | 12/67 [02:50<13:02, 14.22s/it]
2025-04-02 14:48:40,388 - INFO: train loss: 0.07:  22%|##2       | 15/67 [03:36<12:42, 14.67s/it]
2025-04-02 14:49:29,997 - INFO: train loss: 0.08:  27%|##6       | 18/67 [04:26<12:29, 15.30s/it]
2025-04-02 14:50:21,565 - INFO: train loss: 0.08:  31%|###1      | 21/67 [05:17<12:12, 15.92s/it]
2025-04-02 14:51:06,864 - INFO: train loss: 0.08:  36%|###5      | 24/67 [06:02<11:13, 15.66s/it]
2025-04-02 14:51:59,961 - INFO: train loss: 0.08:  40%|####      | 27/67 [06:56<10:51, 16.30s/it]
2025-04-02 14:52:46,381 - INFO: train loss: 0.08:  45%|####4     | 30/67 [07:42<09:53, 16.04s/it]
2025-04-02 14:53:41,369 - INFO: train loss: 0.08:  49%|####9     | 33/67 [08:37<09:29, 16.74s/it]
2025-04-02 14:54:26,988 - INFO: train loss: 0.07:  54%|#####3    | 36/67 [09:23<08:24, 16.27s/it]
2025-04-02 14:55:08,260 - INFO: train loss: 0.08:  58%|#####8    | 39/67 [10:04<07:14, 15.51s/it]
2025-04-02 14:55:53,855 - INFO: train loss: 0.07:  63%|######2   | 42/67 [10:49<06:25, 15.42s/it]
2025-04-02 14:56:39,728 - INFO: train loss: 0.07:  67%|######7   | 45/67 [11:35<05:38, 15.38s/it]
2025-04-02 14:57:24,966 - INFO: train loss: 0.07:  72%|#######1  | 48/67 [12:21<04:50, 15.29s/it]
2025-04-02 14:58:09,632 - INFO: train loss: 0.07:  76%|#######6  | 51/67 [13:05<04:02, 15.17s/it]
2025-04-02 14:58:56,644 - INFO: train loss: 0.08:  81%|########  | 54/67 [13:52<03:19, 15.32s/it]
2025-04-02 14:59:43,159 - INFO: train loss: 0.08:  85%|########5 | 57/67 [14:39<02:33, 15.38s/it]
2025-04-02 15:00:28,364 - INFO: train loss: 0.07:  90%|########9 | 60/67 [15:24<01:46, 15.28s/it]
2025-04-02 15:01:07,997 - INFO: train loss: 0.06:  94%|#########4| 63/67 [16:04<00:58, 14.66s/it]
2025-04-02 15:01:57,562 - INFO: train loss: 0.06:  99%|#########8| 66/67 [16:53<00:15, 15.22s/it]
2025-04-02 15:02:10,141 - INFO: train loss: 0.07: 100%|##########| 67/67 [17:06<00:00, 14.89s/it]
2025-04-02 15:02:10,152 - INFO: Starting validation inference
2025-04-02 15:02:10,153 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 15:07:03,257 - INFO: validation progress:  50%|#####     | 1/2 [04:53<04:53, 293.10s/it]
2025-04-02 15:12:10,593 - INFO: validation progress: 100%|##########| 2/2 [10:00<00:00, 301.48s/it]
2025-04-02 15:12:10,595 - INFO: validation progress: 100%|##########| 2/2 [10:00<00:00, 300.22s/it]
2025-04-02 15:12:12,214 - INFO: Validation BLEU: 59.10284
2025-04-02 15:12:14,304 - INFO: train loss: 0.07: 100%|##########| 67/67 [27:10<00:00, 24.33s/it]
2025-04-02 15:12:14,316 - INFO: Training Epoch: 14 / 15
2025-04-02 15:12:14,316 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 15:12:19,682 - INFO: Evaluation step: 67
2025-04-02 15:12:20,073 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 15:13:06,295 - INFO: train loss: 0.07:   4%|4         | 3/67 [00:51<18:28, 17.33s/it]
2025-04-02 15:13:48,900 - INFO: train loss: 0.06:   9%|8         | 6/67 [01:34<15:44, 15.49s/it]
2025-04-02 15:14:30,910 - INFO: train loss: 0.06:  13%|#3        | 9/67 [02:16<14:18, 14.81s/it]
2025-04-02 15:15:18,832 - INFO: train loss: 0.06:  18%|#7        | 12/67 [03:04<13:59, 15.27s/it]
2025-04-02 15:16:05,855 - INFO: train loss: 0.06:  22%|##2       | 15/67 [03:51<13:21, 15.42s/it]
2025-04-02 15:16:53,029 - INFO: train loss: 0.07:  27%|##6       | 18/67 [04:38<12:40, 15.52s/it]
2025-04-02 15:17:35,856 - INFO: train loss: 0.06:  31%|###1      | 21/67 [05:21<11:35, 15.11s/it]
2025-04-02 15:18:18,433 - INFO: train loss: 0.07:  36%|###5      | 24/67 [06:04<10:37, 14.82s/it]
2025-04-02 15:19:03,415 - INFO: train loss: 0.07:  40%|####      | 27/67 [06:49<09:54, 14.87s/it]
2025-04-02 15:19:44,837 - INFO: train loss: 0.07:  45%|####4     | 30/67 [07:30<08:58, 14.55s/it]
2025-04-02 15:20:34,725 - INFO: train loss: 0.07:  49%|####9     | 33/67 [08:20<08:36, 15.18s/it]
2025-04-02 15:21:15,869 - INFO: train loss: 0.07:  54%|#####3    | 36/67 [09:01<07:36, 14.74s/it]
2025-04-02 15:21:58,460 - INFO: train loss: 0.07:  58%|#####8    | 39/67 [09:44<06:48, 14.57s/it]
2025-04-02 15:22:45,768 - INFO: train loss: 0.07:  63%|######2   | 42/67 [10:31<06:13, 14.93s/it]
2025-04-02 15:23:28,080 - INFO: train loss: 0.06:  67%|######7   | 45/67 [11:13<05:23, 14.68s/it]
2025-04-02 15:24:12,883 - INFO: train loss: 0.07:  72%|#######1  | 48/67 [11:58<04:40, 14.76s/it]
2025-04-02 15:25:07,488 - INFO: train loss: 0.08:  76%|#######6  | 51/67 [12:53<04:12, 15.79s/it]
2025-04-02 15:26:01,442 - INFO: train loss: 0.08:  81%|########  | 54/67 [13:47<03:33, 16.45s/it]
2025-04-02 15:26:44,867 - INFO: train loss: 0.09:  85%|########5 | 57/67 [14:30<02:38, 15.86s/it]
2025-04-02 15:27:34,320 - INFO: train loss: 0.08:  90%|########9 | 60/67 [15:20<01:52, 16.05s/it]
2025-04-02 15:28:20,264 - INFO: train loss: 0.07:  94%|#########4| 63/67 [16:05<01:03, 15.83s/it]
2025-04-02 15:29:04,838 - INFO: train loss: 0.07:  99%|#########8| 66/67 [16:50<00:15, 15.54s/it]
2025-04-02 15:29:18,025 - INFO: train loss: 0.07: 100%|##########| 67/67 [17:03<00:00, 15.24s/it]
2025-04-02 15:29:18,036 - INFO: Starting validation inference
2025-04-02 15:29:18,036 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 15:34:20,788 - INFO: validation progress:  50%|#####     | 1/2 [05:02<05:02, 302.75s/it]
2025-04-02 15:39:30,404 - INFO: validation progress: 100%|##########| 2/2 [10:12<00:00, 306.79s/it]
2025-04-02 15:39:30,407 - INFO: validation progress: 100%|##########| 2/2 [10:12<00:00, 306.19s/it]
2025-04-02 15:39:31,674 - INFO: Validation BLEU: 59.37098
2025-04-02 15:39:33,780 - INFO: train loss: 0.07: 100%|##########| 67/67 [27:19<00:00, 24.47s/it]
2025-04-02 15:39:33,789 - INFO: Training Epoch: 15 / 15
2025-04-02 15:39:33,789 - INFO: train loss:   0%|          | 0/67 [00:00<?, ?it/s]
2025-04-02 15:39:38,567 - INFO: Evaluation step: 67
2025-04-02 15:39:38,578 - INFO: Input exceeds max_length of 4096, truncating sample.
2025-04-02 15:40:19,542 - INFO: train loss: 0.08:   4%|4         | 3/67 [00:45<16:16, 15.25s/it]
2025-04-02 15:41:14,561 - INFO: train loss: 0.08:   9%|8         | 6/67 [01:40<17:21, 17.07s/it]
2025-04-02 15:41:59,253 - INFO: train loss: 0.07:  13%|#3        | 9/67 [02:25<15:32, 16.08s/it]
2025-04-02 15:42:41,873 - INFO: train loss: 0.07:  18%|#7        | 12/67 [03:08<14:03, 15.34s/it]
2025-04-02 15:43:36,449 - INFO: train loss: 0.07:  22%|##2       | 15/67 [04:02<14:11, 16.37s/it]
2025-04-02 15:44:26,814 - INFO: train loss: 0.07:  27%|##6       | 18/67 [04:53<13:29, 16.51s/it]
2025-04-02 15:45:13,103 - INFO: train loss: 0.07:  31%|###1      | 21/67 [05:39<12:23, 16.16s/it]
2025-04-02 15:45:57,835 - INFO: train loss: 0.07:  36%|###5      | 24/67 [06:24<11:17, 15.76s/it]
2025-04-02 15:46:41,308 - INFO: train loss: 0.07:  40%|####      | 27/67 [07:07<10:14, 15.36s/it]
2025-04-02 15:47:25,340 - INFO: train loss: 0.07:  45%|####4     | 30/67 [07:51<09:20, 15.15s/it]
2025-04-02 15:48:09,250 - INFO: train loss: 0.07:  49%|####9     | 33/67 [08:35<08:29, 14.99s/it]
2025-04-02 15:48:54,938 - INFO: train loss: 0.07:  54%|#####3    | 36/67 [09:21<07:47, 15.07s/it]
2025-04-02 15:49:37,421 - INFO: train loss: 0.07:  58%|#####8    | 39/67 [10:03<06:54, 14.79s/it]
2025-04-02 15:50:19,957 - INFO: train loss: 0.07:  63%|######2   | 42/67 [10:46<06:05, 14.61s/it]
2025-04-02 15:51:00,260 - INFO: train loss: 0.06:  67%|######7   | 45/67 [11:26<05:13, 14.25s/it]
2025-04-02 15:51:55,602 - INFO: train loss: 0.07:  72%|#######1  | 48/67 [12:21<04:54, 15.52s/it]
2025-04-02 15:52:38,083 - INFO: train loss: 0.07:  76%|#######6  | 51/67 [13:04<04:01, 15.11s/it]
2025-04-02 15:53:27,276 - INFO: train loss: 0.07:  81%|########  | 54/67 [13:53<03:21, 15.50s/it]
2025-04-02 15:54:14,275 - INFO: train loss: 0.07:  85%|########5 | 57/67 [14:40<02:35, 15.55s/it]
2025-04-02 15:54:59,571 - INFO: train loss: 0.07:  90%|########9 | 60/67 [15:25<01:47, 15.41s/it]
2025-04-02 15:55:50,037 - INFO: train loss: 0.07:  94%|#########4| 63/67 [16:16<01:03, 15.84s/it]
2025-04-02 15:56:32,141 - INFO: train loss: 0.07:  99%|#########8| 66/67 [16:58<00:15, 15.29s/it]
2025-04-02 15:56:44,454 - INFO: train loss: 0.07: 100%|##########| 67/67 [17:10<00:00, 14.92s/it]
2025-04-02 15:56:44,464 - INFO: Starting validation inference
2025-04-02 15:56:44,465 - INFO: validation progress:   0%|          | 0/2 [00:00<?, ?it/s]
2025-04-02 16:02:17,737 - INFO: validation progress:  50%|#####     | 1/2 [05:33<05:33, 333.27s/it]
2025-04-02 16:07:35,446 - INFO: validation progress: 100%|##########| 2/2 [10:50<00:00, 324.12s/it]
2025-04-02 16:07:35,448 - INFO: validation progress: 100%|##########| 2/2 [10:50<00:00, 325.49s/it]
2025-04-02 16:07:36,580 - INFO: Validation BLEU: 59.35042
2025-04-02 16:07:38,169 - INFO: train loss: 0.07: 100%|##########| 67/67 [28:04<00:00, 25.14s/it]