541349.err

[NbConvertApp] Converting notebook HCP_downstream_raw_flatmaps.ipynb to python
[NbConvertApp] Writing 36673 bytes to HCP_downstream_raw_flatmaps.py
wandb: Using wandb-core as the SDK backend. Please refer to https://wandb.me/wandb-core for more information.
wandb: Currently logged in as: ckadirt. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.18.3
wandb: Run data is saved locally in /weka/proj-fmri/ckadirt/fMRI-foundation-model/src/wandb/run-20241127_020911-HCPflat_raw_beta_trial_type_81853367-3038-4b91-805f-5066c048cef4
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run HCPflat_raw_beta_trial_type
wandb: ⭐️ View project at https://stability.wandb.io/ckadirt/fMRI-foundation-model
wandb: 🚀 View run at https://stability.wandb.io/ckadirt/fMRI-foundation-model/runs/HCPflat_raw_beta_trial_type_81853367-3038-4b91-805f-5066c048cef4

Epoch 1/50 - Training:   0%|          | 0/435 [00:00<?, ?it/s]
Epoch 1/50 - Training:   0%|          | 1/435 [00:26<3:13:32, 26.76s/it]
Epoch 1/50 - Training:   0%|          | 2/435 [00:26<1:20:02, 11.09s/it]
Epoch 1/50 - Training:   1%|          | 3/435 [00:27<43:48,  6.08s/it]  
Epoch 1/50 - Training:   1%|          | 4/435 [00:27<26:48,  3.73s/it]
Epoch 1/50 - Training:   1%|          | 5/435 [00:27<17:25,  2.43s/it]
Epoch 1/50 - Training:   1%|▏         | 6/435 [00:27<11:50,  1.66s/it]
Epoch 1/50 - Training:   2%|▏         | 7/435 [00:27<08:16,  1.16s/it]
Epoch 1/50 - Training:   2%|▏         | 8/435 [00:27<05:57,  1.19it/s]
Epoch 1/50 - Training:   2%|▏         | 9/435 [00:27<04:23,  1.62it/s]
Epoch 1/50 - Training:   2%|▏         | 10/435 [00:27<03:20,  2.12it/s]
Epoch 1/50 - Training:   3%|▎         | 11/435 [00:44<39:04,  5.53s/it]
Epoch 1/50 - Training:   3%|▎         | 13/435 [00:45<21:00,  2.99s/it]
Epoch 1/50 - Training:   3%|▎         | 15/435 [00:45<12:48,  1.83s/it]
Epoch 1/50 - Training:   4%|▎         | 16/435 [00:45<10:07,  1.45s/it]
Epoch 1/50 - Training:   4%|▍         | 17/435 [00:45<07:53,  1.13s/it]
Epoch 1/50 - Training:   4%|▍         | 18/435 [00:45<06:07,  1.14it/s]
Epoch 1/50 - Training:   4%|▍         | 19/435 [00:45<04:43,  1.47it/s]
Epoch 1/50 - Training:   5%|▍         | 20/435 [00:45<03:41,  1.88it/s]
Epoch 1/50 - Training:   5%|▍         | 21/435 [00:55<20:17,  2.94s/it]
Epoch 1/50 - Training:   5%|▌         | 23/435 [00:55<11:23,  1.66s/it]
Epoch 1/50 - Training:   6%|▌         | 25/435 [00:55<07:11,  1.05s/it]
Epoch 1/50 - Training:   6%|▌         | 27/435 [00:55<04:52,  1.40it/s]
Epoch 1/50 - Training:   6%|▋         | 28/435 [00:55<04:03,  1.67it/s]
Epoch 1/50 - Training:   7%|▋         | 29/435 [00:55<03:19,  2.04it/s]
Epoch 1/50 - Training:   7%|▋         | 30/435 [00:56<02:45,  2.45it/s]
Epoch 1/50 - Training:   7%|▋         | 31/435 [01:01<10:54,  1.62s/it]
Epoch 1/50 - Training:   7%|▋         | 32/435 [01:01<08:08,  1.21s/it]
Epoch 1/50 - Training:   8%|▊         | 34/435 [01:01<04:51,  1.37it/s]
Epoch 1/50 - Training:   8%|▊         | 35/435 [01:01<03:52,  1.72it/s]
Epoch 1/50 - Training:   9%|▊         | 37/435 [01:01<02:33,  2.58it/s]
Epoch 1/50 - Training:   9%|▉         | 39/435 [01:01<01:50,  3.59it/s]
Epoch 1/50 - Training:   9%|▉         | 40/435 [01:02<01:36,  4.11it/s]
Epoch 1/50 - Training:   9%|▉         | 41/435 [01:07<09:20,  1.42s/it]
Epoch 1/50 - Training:  10%|▉         | 42/435 [01:07<07:13,  1.10s/it]
Epoch 1/50 - Training:  10%|▉         | 43/435 [01:07<05:30,  1.19it/s]
Epoch 1/50 - Training:  10%|█         | 45/435 [01:07<03:23,  1.92it/s]
Epoch 1/50 - Training:  11%|█         | 47/435 [01:08<02:18,  2.81it/s]
Epoch 1/50 - Training:  11%|█▏        | 49/435 [01:08<01:40,  3.84it/s]
Epoch 1/50 - Training:  12%|█▏        | 51/435 [01:13<06:57,  1.09s/it]
Epoch 1/50 - Training:  12%|█▏        | 52/435 [01:13<05:44,  1.11it/s]
Epoch 1/50 - Training:  12%|█▏        | 54/435 [01:13<03:52,  1.64it/s]
Epoch 1/50 - Training:  13%|█▎        | 56/435 [01:14<02:43,  2.32it/s]
Epoch 1/50 - Training:  13%|█▎        | 58/435 [01:14<01:59,  3.15it/s]
Epoch 1/50 - Training:  14%|█▍        | 60/435 [01:14<01:30,  4.13it/s]
Epoch 1/50 - Training:  14%|█▍        | 62/435 [01:19<06:19,  1.02s/it]
Epoch 1/50 - Training:  14%|█▍        | 63/435 [01:19<05:18,  1.17it/s]
Epoch 1/50 - Training:  15%|█▍        | 64/435 [01:20<04:20,  1.42it/s]
Epoch 1/50 - Training:  15%|█▍        | 65/435 [01:20<03:30,  1.75it/s]
Epoch 1/50 - Training:  15%|█▌        | 66/435 [01:20<02:49,  2.18it/s]
Epoch 1/50 - Training:  15%|█▌        | 67/435 [01:20<02:15,  2.71it/s]
Epoch 1/50 - Training:  16%|█▌        | 68/435 [01:20<01:49,  3.34it/s]
Epoch 1/50 - Training:  16%|█▌        | 69/435 [01:20<01:30,  4.05it/s]
Epoch 1/50 - Training:  16%|█▌        | 70/435 [01:20<01:15,  4.81it/s]
Epoch 1/50 - Training:  16%|█▋        | 71/435 [01:26<11:29,  1.89s/it]
Epoch 1/50 - Training:  17%|█▋        | 72/435 [01:26<08:18,  1.37s/it]
Epoch 1/50 - Training:  17%|█▋        | 73/435 [01:27<06:03,  1.00s/it]
Epoch 1/50 - Training:  17%|█▋        | 74/435 [01:27<04:27,  1.35it/s]
Epoch 1/50 - Training:  17%|█▋        | 75/435 [01:27<03:20,  1.79it/s]
Epoch 1/50 - Training:  17%|█▋        | 76/435 [01:27<02:33,  2.34it/s]
Epoch 1/50 - Training:  18%|█▊        | 77/435 [01:27<02:00,  2.98it/s]
Epoch 1/50 - Training:  18%|█▊        | 78/435 [01:27<01:35,  3.74it/s]
Epoch 1/50 - Training:  18%|█▊        | 79/435 [01:27<01:19,  4.49it/s]
Epoch 1/50 - Training:  18%|█▊        | 80/435 [01:27<01:08,  5.18it/s]
Epoch 1/50 - Training:  19%|█▊        | 81/435 [01:33<11:16,  1.91s/it]
Epoch 1/50 - Training:  19%|█▉        | 82/435 [01:33<08:04,  1.37s/it]
Epoch 1/50 - Training:  19%|█▉        | 83/435 [01:34<05:49,  1.01it/s]
Epoch 1/50 - Training:  19%|█▉        | 84/435 [01:34<04:17,  1.37it/s]
Epoch 1/50 - Training:  20%|█▉        | 85/435 [01:34<03:13,  1.81it/s]
Epoch 1/50 - Training:  20%|█▉        | 86/435 [01:34<02:27,  2.36it/s]
Epoch 1/50 - Training:  20%|██        | 87/435 [01:34<01:55,  3.02it/s]
Epoch 1/50 - Training:  20%|██        | 89/435 [01:34<01:17,  4.46it/s]
Epoch 1/50 - Training:  21%|██        | 90/435 [01:34<01:08,  5.06it/s]
Epoch 1/50 - Training:  21%|██        | 91/435 [01:41<10:09,  1.77s/it]
Epoch 1/50 - Training:  21%|██        | 92/435 [01:41<07:32,  1.32s/it]
Epoch 1/50 - Training:  21%|██▏       | 93/435 [01:41<05:35,  1.02it/s]
Epoch 1/50 - Training:  22%|██▏       | 94/435 [01:41<04:08,  1.37it/s]
Epoch 1/50 - Training:  22%|██▏       | 95/435 [01:41<03:06,  1.82it/s]
Epoch 1/50 - Training:  22%|██▏       | 96/435 [01:41<02:22,  2.37it/s]
Epoch 1/50 - Training:  22%|██▏       | 97/435 [01:41<01:51,  3.04it/s]
Epoch 1/50 - Training:  23%|██▎       | 99/435 [01:41<01:14,  4.49it/s]
Epoch 1/50 - Training:  23%|██▎       | 100/435 [01:42<01:07,  5.00it/s]
Epoch 1/50 - Training:  23%|██▎       | 101/435 [01:48<10:03,  1.81s/it]
Epoch 1/50 - Training:  23%|██▎       | 102/435 [01:48<07:29,  1.35s/it]
Epoch 1/50 - Training:  24%|██▎       | 103/435 [01:48<05:32,  1.00s/it]
Epoch 1/50 - Training:  24%|██▍       | 104/435 [01:48<04:06,  1.34it/s]
Epoch 1/50 - Training:  24%|██▍       | 105/435 [01:49<03:56,  1.40it/s]
Epoch 1/50 - Training:  24%|██▍       | 106/435 [01:49<02:59,  1.83it/s]
Epoch 1/50 - Training:  25%|██▍       | 107/435 [01:49<02:18,  2.36it/s]
Epoch 1/50 - Training:  25%|██▍       | 108/435 [01:49<01:47,  3.03it/s]
Epoch 1/50 - Training:  25%|██▌       | 109/435 [01:49<01:26,  3.78it/s]
Epoch 1/50 - Training:  25%|██▌       | 110/435 [01:49<01:13,  4.43it/s]
Epoch 1/50 - Training:  26%|██▌       | 111/435 [01:55<10:07,  1.88s/it]
Epoch 1/50 - Training:  26%|██▌       | 112/435 [01:55<07:14,  1.35s/it]
Epoch 1/50 - Training:  26%|██▌       | 113/435 [01:55<05:13,  1.03it/s]
Epoch 1/50 - Training:  26%|██▌       | 114/435 [01:56<04:47,  1.12it/s]
Epoch 1/50 - Training:  26%|██▋       | 115/435 [01:57<05:00,  1.06it/s]
Epoch 1/50 - Training:  27%|██▋       | 116/435 [01:57<03:40,  1.45it/s]
Epoch 1/50 - Training:  27%|██▋       | 117/435 [01:57<02:47,  1.89it/s]
Epoch 1/50 - Training:  27%|██▋       | 118/435 [01:58<02:07,  2.48it/s]
Epoch 1/50 - Training:  27%|██▋       | 119/435 [01:58<01:42,  3.10it/s]
Epoch 1/50 - Training:  28%|██▊       | 120/435 [01:58<01:24,  3.74it/s]
Epoch 1/50 - Training:  28%|██▊       | 121/435 [02:03<08:30,  1.63s/it]
Epoch 1/50 - Training:  28%|██▊       | 122/435 [02:03<06:06,  1.17s/it]
Epoch 1/50 - Training:  28%|██▊       | 123/435 [02:03<04:30,  1.16it/s]
Epoch 1/50 - Training:  29%|██▊       | 124/435 [02:05<05:46,  1.11s/it]
Epoch 1/50 - Training:  29%|██▊       | 125/435 [02:06<05:40,  1.10s/it]
Epoch 1/50 - Training:  29%|██▉       | 126/435 [02:06<04:08,  1.24it/s]
Epoch 1/50 - Training:  29%|██▉       | 127/435 [02:06<03:15,  1.58it/s]
Epoch 1/50 - Training:  29%|██▉       | 128/435 [02:06<02:26,  2.10it/s]
Epoch 1/50 - Training:  30%|██▉       | 129/435 [02:06<01:52,  2.73it/s]
Epoch 1/50 - Training:  30%|██▉       | 130/435 [02:06<01:29,  3.41it/s]
Epoch 1/50 - Training:  30%|███       | 131/435 [02:10<06:38,  1.31s/it]
Epoch 1/50 - Training:  30%|███       | 132/435 [02:10<04:49,  1.05it/s]
Epoch 1/50 - Training:  31%|███       | 133/435 [02:11<04:56,  1.02it/s]
Epoch 1/50 - Training:  31%|███       | 134/435 [02:13<06:13,  1.24s/it]
Epoch 1/50 - Training:  31%|███       | 135/435 [02:14<05:49,  1.16s/it]
Epoch 1/50 - Training:  31%|███▏      | 136/435 [02:14<04:15,  1.17it/s]
Epoch 1/50 - Training:  31%|███▏      | 137/435 [02:15<03:38,  1.37it/s]
Epoch 1/50 - Training:  32%|███▏      | 138/435 [02:15<02:42,  1.83it/s]
Epoch 1/50 - Training:  32%|███▏      | 139/435 [02:15<02:03,  2.41it/s]
Epoch 1/50 - Training:  32%|███▏      | 140/435 [02:15<01:37,  3.02it/s]
Epoch 1/50 - Training:  32%|███▏      | 141/435 [02:17<04:45,  1.03it/s]
Epoch 1/50 - Training:  33%|███▎      | 142/435 [02:18<04:11,  1.17it/s]
Epoch 1/50 - Training:  33%|███▎      | 143/435 [02:19<04:58,  1.02s/it]
Epoch 1/50 - Training:  33%|███▎      | 144/435 [02:21<06:25,  1.33s/it]
Epoch 1/50 - Training:  33%|███▎      | 145/435 [02:22<05:49,  1.20s/it]
Epoch 1/50 - Training:  34%|███▎      | 146/435 [02:23<04:16,  1.13it/s]
Epoch 1/50 - Training:  34%|███▍      | 147/435 [02:23<03:51,  1.24it/s]
Epoch 1/50 - Training:  34%|███▍      | 148/435 [02:23<02:51,  1.68it/s]
Epoch 1/50 - Training:  34%|███▍      | 149/435 [02:23<02:09,  2.21it/s]
Epoch 1/50 - Training:  34%|███▍      | 150/435 [02:24<01:40,  2.82it/s]
Epoch 1/50 - Training:  35%|███▍      | 151/435 [02:25<03:01,  1.57it/s]
Epoch 1/50 - Training:  35%|███▍      | 152/435 [02:26<04:04,  1.16it/s]
Epoch 1/50 - Training:  35%|███▌      | 153/435 [02:28<04:57,  1.05s/it]
Epoch 1/50 - Training:  35%|███▌      | 154/435 [02:30<06:35,  1.41s/it]
Epoch 1/50 - Training:  36%|███▌      | 155/435 [02:31<05:48,  1.24s/it]
Epoch 1/50 - Training:  36%|███▌      | 156/435 [02:31<04:13,  1.10it/s]
Epoch 1/50 - Training:  36%|███▌      | 157/435 [02:32<04:03,  1.14it/s]
Epoch 1/50 - Training:  37%|███▋      | 159/435 [02:32<02:22,  1.93it/s]
Epoch 1/50 - Training:  37%|███▋      | 160/435 [02:32<01:55,  2.38it/s]
Epoch 1/50 - Training:  37%|███▋      | 161/435 [02:32<01:37,  2.82it/s]
Epoch 1/50 - Training:  37%|███▋      | 162/435 [02:34<03:50,  1.19it/s]
Epoch 1/50 - Training:  37%|███▋      | 163/435 [02:36<04:46,  1.05s/it]
Epoch 1/50 - Training:  38%|███▊      | 164/435 [02:38<06:32,  1.45s/it]
Epoch 1/50 - Training:  38%|███▊      | 165/435 [02:39<05:40,  1.26s/it]
Epoch 1/50 - Training:  38%|███▊      | 166/435 [02:39<04:10,  1.07it/s]
Epoch 1/50 - Training:  38%|███▊      | 167/435 [02:40<03:50,  1.16it/s]
Epoch 1/50 - Training:  39%|███▊      | 168/435 [02:40<02:50,  1.57it/s]
Epoch 1/50 - Training:  39%|███▉      | 169/435 [02:40<02:07,  2.09it/s]
Epoch 1/50 - Training:  39%|███▉      | 170/435 [02:40<01:38,  2.69it/s]
Epoch 1/50 - Training:  40%|███▉      | 172/435 [02:42<02:52,  1.52it/s]
Epoch 1/50 - Training:  40%|███▉      | 173/435 [02:44<03:58,  1.10it/s]
Epoch 1/50 - Training:  40%|████      | 174/435 [02:47<05:49,  1.34s/it]
Epoch 1/50 - Training:  40%|████      | 175/435 [02:47<05:05,  1.18s/it]
Epoch 1/50 - Training:  40%|████      | 176/435 [02:47<03:49,  1.13it/s]
Epoch 1/50 - Training:  41%|████      | 177/435 [02:48<03:59,  1.08it/s]
Epoch 1/50 - Training:  41%|████      | 179/435 [02:49<02:21,  1.80it/s]
Epoch 1/50 - Training:  41%|████▏     | 180/435 [02:49<01:54,  2.23it/s]
Epoch 1/50 - Training:  42%|████▏     | 182/435 [02:50<02:26,  1.73it/s]
Epoch 1/50 - Training:  42%|████▏     | 183/435 [02:52<03:38,  1.16it/s]
Epoch 1/50 - Training:  42%|████▏     | 184/435 [02:55<05:21,  1.28s/it]
Epoch 1/50 - Training:  43%|████▎     | 185/435 [02:55<04:45,  1.14s/it]
Epoch 1/50 - Training:  43%|████▎     | 186/435 [02:56<03:36,  1.15it/s]
Epoch 1/50 - Training:  43%|████▎     | 187/435 [02:57<04:00,  1.03it/s]
Epoch 1/50 - Training:  43%|████▎     | 189/435 [02:57<02:22,  1.72it/s]
Epoch 1/50 - Training:  44%|████▎     | 190/435 [02:57<01:54,  2.15it/s]
Epoch 1/50 - Training:  44%|████▍     | 192/435 [02:58<02:06,  1.92it/s]
Epoch 1/50 - Training:  44%|████▍     | 193/435 [03:00<03:22,  1.19it/s]
Epoch 1/50 - Training:  45%|████▍     | 194/435 [03:03<05:02,  1.26s/it]
Epoch 1/50 - Training:  45%|████▍     | 195/435 [03:03<04:21,  1.09s/it]
Epoch 1/50 - Training:  45%|████▌     | 196/435 [03:04<03:18,  1.21it/s]
Epoch 1/50 - Training:  45%|████▌     | 197/435 [03:05<04:01,  1.01s/it]
Epoch 1/50 - Training:  46%|████▌     | 199/435 [03:05<02:23,  1.65it/s]
Epoch 1/50 - Training:  46%|████▌     | 200/435 [03:05<01:54,  2.06it/s]
Epoch 1/50 - Training:  46%|████▋     | 202/435 [03:06<01:56,  1.99it/s]
Epoch 1/50 - Training:  47%|████▋     | 203/435 [03:08<02:31,  1.53it/s]
Epoch 1/50 - Training:  47%|████▋     | 204/435 [03:11<05:08,  1.34s/it]
Epoch 1/50 - Training:  47%|████▋     | 205/435 [03:12<04:49,  1.26s/it]
Epoch 1/50 - Training:  48%|████▊     | 207/435 [03:14<03:57,  1.04s/it]
Epoch 1/50 - Training:  48%|████▊     | 209/435 [03:14<02:34,  1.46it/s]
Epoch 1/50 - Training:  48%|████▊     | 210/435 [03:14<02:06,  1.78it/s]
Epoch 1/50 - Training:  49%|████▊     | 212/435 [03:15<02:06,  1.76it/s]
Epoch 1/50 - Training:  49%|████▉     | 213/435 [03:16<02:16,  1.62it/s]
Epoch 1/50 - Training:  49%|████▉     | 214/435 [03:18<04:00,  1.09s/it]
Epoch 1/50 - Training:  49%|████▉     | 215/435 [03:20<04:38,  1.27s/it]
Epoch 1/50 - Training:  50%|████▉     | 216/435 [03:20<03:31,  1.04it/s]
Epoch 1/50 - Training:  50%|████▉     | 217/435 [03:21<03:08,  1.16it/s]
Epoch 1/50 - Training:  50%|█████     | 219/435 [03:22<02:27,  1.46it/s]
Epoch 1/50 - Training:  51%|█████     | 220/435 [03:22<01:59,  1.81it/s]
Epoch 1/50 - Training:  51%|█████     | 222/435 [03:23<01:56,  1.83it/s]
Epoch 1/50 - Training:  51%|█████▏    | 223/435 [03:24<02:01,  1.75it/s]
Epoch 1/50 - Training:  51%|█████▏    | 224/435 [03:26<03:40,  1.04s/it]
Epoch 1/50 - Training:  52%|█████▏    | 225/435 [03:30<05:42,  1.63s/it]
Epoch 1/50 - Training:  52%|█████▏    | 226/435 [03:30<04:16,  1.23s/it]
Epoch 1/50 - Training:  52%|█████▏    | 227/435 [03:30<03:11,  1.09it/s]
Epoch 1/50 - Training:  53%|█████▎    | 229/435 [03:32<03:04,  1.12it/s]
Epoch 1/50 - Training:  53%|█████▎    | 230/435 [03:32<02:25,  1.41it/s]
Epoch 1/50 - Training:  53%|█████▎    | 231/435 [03:32<01:53,  1.79it/s]
Epoch 1/50 - Training:  53%|█████▎    | 232/435 [03:33<02:08,  1.58it/s]
Epoch 1/50 - Training:  54%|█████▎    | 233/435 [03:33<02:03,  1.63it/s]
Epoch 1/50 - Training:  54%|█████▍    | 234/435 [03:34<02:38,  1.27it/s]
Epoch 1/50 - Training:  54%|█████▍    | 235/435 [03:39<06:23,  1.92s/it]
Epoch 1/50 - Training:  54%|█████▍    | 237/435 [03:39<03:37,  1.10s/it]
Epoch 1/50 - Training:  55%|█████▍    | 239/435 [03:41<03:05,  1.05it/s]
Epoch 1/50 - Training:  55%|█████▌    | 241/435 [03:41<02:03,  1.57it/s]
Epoch 1/50 - Training:  56%|█████▌    | 242/435 [03:42<02:21,  1.36it/s]
Epoch 1/50 - Training:  56%|█████▌    | 244/435 [03:43<02:03,  1.55it/s]
Epoch 1/50 - Training:  56%|█████▋    | 245/435 [03:48<04:53,  1.54s/it]
Epoch 1/50 - Training:  57%|█████▋    | 247/435 [03:48<03:08,  1.00s/it]
Epoch 1/50 - Training:  57%|█████▋    | 249/435 [03:49<02:26,  1.27it/s]
Epoch 1/50 - Training:  58%|█████▊    | 251/435 [03:49<01:42,  1.79it/s]
Epoch 1/50 - Training:  58%|█████▊    | 252/435 [04:00<07:33,  2.48s/it]
Epoch 1/50 - Training:  58%|█████▊    | 254/435 [04:00<04:55,  1.63s/it]
Epoch 1/50 - Training:  59%|█████▉    | 256/435 [04:00<03:20,  1.12s/it]
Epoch 1/50 - Training:  59%|█████▉    | 258/435 [04:00<02:19,  1.27it/s]
Epoch 1/50 - Training:  60%|█████▉    | 260/435 [04:00<01:39,  1.77it/s]
Epoch 1/50 - Training:  60%|██████    | 261/435 [04:01<01:23,  2.08it/s]
Epoch 1/50 - Training:  60%|██████    | 262/435 [04:07<04:40,  1.62s/it]
Epoch 1/50 - Training:  61%|██████    | 264/435 [04:07<02:59,  1.05s/it]
Epoch 1/50 - Training:  61%|██████    | 266/435 [04:07<02:01,  1.39it/s]
Epoch 1/50 - Training:  62%|██████▏   | 268/435 [04:07<01:25,  1.96it/s]
Epoch 1/50 - Training:  62%|██████▏   | 269/435 [04:07<01:11,  2.31it/s]
Epoch 1/50 - Training:  62%|██████▏   | 271/435 [04:07<00:51,  3.18it/s]
Epoch 1/50 - Training:  63%|██████▎   | 272/435 [04:16<05:09,  1.90s/it]
Epoch 1/50 - Training:  63%|██████▎   | 274/435 [04:16<03:19,  1.24s/it]
Epoch 1/50 - Training:  63%|██████▎   | 275/435 [04:16<02:40,  1.01s/it]
Epoch 1/50 - Training:  64%|██████▎   | 277/435 [04:16<01:45,  1.50it/s]
Epoch 1/50 - Training:  64%|██████▍   | 278/435 [04:16<01:26,  1.82it/s]
Epoch 1/50 - Training:  64%|██████▍   | 279/435 [04:16<01:09,  2.24it/s]
Epoch 1/50 - Training:  65%|██████▍   | 281/435 [04:16<00:47,  3.24it/s]
Epoch 1/50 - Training:  65%|██████▍   | 282/435 [04:24<05:04,  1.99s/it]
Epoch 1/50 - Training:  65%|██████▌   | 284/435 [04:25<03:09,  1.26s/it]
Epoch 1/50 - Training:  66%|██████▌   | 286/435 [04:25<02:05,  1.18it/s]
Epoch 1/50 - Training:  66%|██████▌   | 287/435 [04:25<01:43,  1.43it/s]
Epoch 1/50 - Training:  66%|██████▌   | 288/435 [04:25<01:24,  1.75it/s]
Epoch 1/50 - Training:  66%|██████▋   | 289/435 [04:25<01:07,  2.17it/s]
Epoch 1/50 - Training:  67%|██████▋   | 291/435 [04:25<00:45,  3.16it/s]
Epoch 1/50 - Training:  67%|██████▋   | 292/435 [04:33<04:38,  1.95s/it]
Epoch 1/50 - Training:  68%|██████▊   | 294/435 [04:33<02:52,  1.22s/it]
Epoch 1/50 - Training:  68%|██████▊   | 296/435 [04:33<01:53,  1.23it/s]
Epoch 1/50 - Training:  69%|██████▊   | 298/435 [04:33<01:18,  1.75it/s]
Epoch 1/50 - Training:  69%|██████▉   | 300/435 [04:34<00:56,  2.39it/s]
Epoch 1/50 - Training:  69%|██████▉   | 301/435 [04:34<00:48,  2.78it/s]
Epoch 1/50 - Training:  69%|██████▉   | 302/435 [04:42<04:24,  1.99s/it]
Epoch 1/50 - Training:  70%|██████▉   | 304/435 [04:42<02:46,  1.27s/it]
Epoch 1/50 - Training:  70%|███████   | 305/435 [04:42<02:13,  1.03s/it]
Epoch 1/50 - Training:  71%|███████   | 307/435 [04:42<01:26,  1.49it/s]
Epoch 1/50 - Training:  71%|███████   | 308/435 [04:42<01:09,  1.81it/s]
Epoch 1/50 - Training:  71%|███████   | 309/435 [04:42<00:56,  2.24it/s]
Epoch 1/50 - Training:  71%|███████▏  | 311/435 [04:43<00:37,  3.27it/s]
Epoch 1/50 - Training:  72%|███████▏  | 312/435 [04:51<04:13,  2.06s/it]
Epoch 1/50 - Training:  72%|███████▏  | 314/435 [04:51<02:37,  1.30s/it]
Epoch 1/50 - Training:  72%|███████▏  | 315/435 [04:51<02:05,  1.05s/it]
Epoch 1/50 - Training:  73%|███████▎  | 317/435 [04:51<01:20,  1.46it/s]
Epoch 1/50 - Training:  73%|███████▎  | 318/435 [04:52<01:06,  1.76it/s]
Epoch 1/50 - Training:  73%|███████▎  | 319/435 [04:52<00:53,  2.17it/s]
Epoch 1/50 - Training:  74%|███████▎  | 320/435 [04:52<00:42,  2.69it/s]
Epoch 1/50 - Training:  74%|███████▍  | 321/435 [04:52<00:34,  3.29it/s]
Epoch 1/50 - Training:  74%|███████▍  | 322/435 [04:59<03:59,  2.12s/it]
Epoch 1/50 - Training:  74%|███████▍  | 324/435 [04:59<02:15,  1.22s/it]
Epoch 1/50 - Training:  75%|███████▍  | 325/435 [04:59<01:45,  1.04it/s]
Epoch 1/50 - Training:  75%|███████▌  | 327/435 [04:59<01:05,  1.65it/s]
Epoch 1/50 - Training:  76%|███████▌  | 329/435 [04:59<00:44,  2.39it/s]
Epoch 1/50 - Training:  76%|███████▌  | 330/435 [05:00<00:37,  2.82it/s]
Epoch 1/50 - Training:  76%|███████▋  | 332/435 [05:07<02:43,  1.59s/it]
Epoch 1/50 - Training:  77%|███████▋  | 334/435 [05:07<01:48,  1.07s/it]
Epoch 1/50 - Training:  77%|███████▋  | 335/435 [05:07<01:28,  1.13it/s]
Epoch 1/50 - Training:  77%|███████▋  | 336/435 [05:07<01:10,  1.40it/s]
Epoch 1/50 - Training:  77%|███████▋  | 337/435 [05:07<00:55,  1.76it/s]
Epoch 1/50 - Training:  78%|███████▊  | 338/435 [05:08<00:43,  2.23it/s]
Epoch 1/50 - Training:  78%|███████▊  | 339/435 [05:08<00:34,  2.82it/s]
Epoch 1/50 - Training:  78%|███████▊  | 341/435 [05:08<00:23,  4.08it/s]
Epoch 1/50 - Training:  79%|███████▊  | 342/435 [05:16<03:14,  2.09s/it]
Epoch 1/50 - Training:  79%|███████▉  | 344/435 [05:16<01:56,  1.28s/it]
Epoch 1/50 - Training:  79%|███████▉  | 345/435 [05:17<01:46,  1.19s/it]
Epoch 1/50 - Training:  80%|███████▉  | 347/435 [05:17<01:07,  1.30it/s]
Epoch 1/50 - Training:  80%|████████  | 348/435 [05:17<00:54,  1.60it/s]
Epoch 1/50 - Training:  80%|████████  | 350/435 [05:17<00:35,  2.37it/s]
Epoch 1/50 - Training:  81%|████████  | 352/435 [05:24<01:59,  1.44s/it]
Epoch 1/50 - Training:  81%|████████▏ | 354/435 [05:24<01:19,  1.02it/s]
Epoch 1/50 - Training:  82%|████████▏ | 355/435 [05:26<01:37,  1.22s/it]
Epoch 1/50 - Training:  82%|████████▏ | 357/435 [05:27<01:04,  1.22it/s]
Epoch 1/50 - Training:  83%|████████▎ | 359/435 [05:27<00:43,  1.73it/s]
Epoch 1/50 - Training:  83%|████████▎ | 360/435 [05:27<00:36,  2.06it/s]
Epoch 1/50 - Training:  83%|████████▎ | 361/435 [05:27<00:29,  2.48it/s]
Epoch 1/50 - Training:  83%|████████▎ | 362/435 [05:32<01:54,  1.57s/it]
Epoch 1/50 - Training:  83%|████████▎ | 363/435 [05:32<01:26,  1.20s/it]
Epoch 1/50 - Training:  84%|████████▎ | 364/435 [05:33<01:04,  1.10it/s]
Epoch 1/50 - Training:  84%|████████▍ | 365/435 [05:36<01:50,  1.58s/it]
Epoch 1/50 - Training:  84%|████████▍ | 366/435 [05:36<01:20,  1.16s/it]
Epoch 1/50 - Training:  85%|████████▍ | 368/435 [05:36<00:46,  1.45it/s]
Epoch 1/50 - Training:  85%|████████▍ | 369/435 [05:36<00:36,  1.82it/s]
Epoch 1/50 - Training:  85%|████████▌ | 370/435 [05:36<00:28,  2.28it/s]
Epoch 1/50 - Training:  86%|████████▌ | 372/435 [05:40<01:05,  1.04s/it]
Epoch 1/50 - Training:  86%|████████▌ | 374/435 [05:40<00:41,  1.46it/s]
Epoch 1/50 - Training:  86%|████████▌ | 375/435 [05:45<01:28,  1.48s/it]
Epoch 1/50 - Training:  87%|████████▋ | 377/435 [05:45<00:55,  1.05it/s]
Epoch 1/50 - Training:  87%|████████▋ | 379/435 [05:45<00:36,  1.53it/s]
Epoch 1/50 - Training:  87%|████████▋ | 380/435 [05:45<00:30,  1.83it/s]
Epoch 1/50 - Training:  88%|████████▊ | 382/435 [05:48<00:44,  1.19it/s]
Epoch 1/50 - Training:  88%|████████▊ | 383/435 [05:48<00:35,  1.45it/s]
Epoch 1/50 - Training:  88%|████████▊ | 384/435 [05:48<00:28,  1.79it/s]
Epoch 1/50 - Training:  89%|████████▊ | 385/435 [05:53<01:22,  1.64s/it]
Epoch 1/50 - Training:  89%|████████▉ | 387/435 [05:53<00:48,  1.01s/it]
Epoch 1/50 - Training:  89%|████████▉ | 388/435 [05:53<00:37,  1.25it/s]
Epoch 1/50 - Training:  89%|████████▉ | 389/435 [05:54<00:29,  1.58it/s]
Epoch 1/50 - Training:  90%|████████▉ | 390/435 [05:54<00:22,  2.01it/s]
Epoch 1/50 - Training:  90%|████████▉ | 391/435 [05:54<00:17,  2.55it/s]
Epoch 1/50 - Training:  90%|█████████ | 392/435 [05:55<00:26,  1.64it/s]
Epoch 1/50 - Training:  90%|█████████ | 393/435 [05:55<00:19,  2.14it/s]
Epoch 1/50 - Training:  91%|█████████ | 394/435 [05:55<00:15,  2.71it/s]
Epoch 1/50 - Training:  91%|█████████ | 395/435 [06:02<01:31,  2.28s/it]
Epoch 1/50 - Training:  91%|█████████▏| 397/435 [06:02<00:48,  1.28s/it]
Epoch 1/50 - Training:  91%|█████████▏| 398/435 [06:02<00:37,  1.00s/it]
Epoch 1/50 - Training:  92%|█████████▏| 399/435 [06:03<00:27,  1.30it/s]
Epoch 1/50 - Training:  92%|█████████▏| 400/435 [06:03<00:20,  1.68it/s]
Epoch 1/50 - Training:  92%|█████████▏| 401/435 [06:03<00:15,  2.17it/s]
Epoch 1/50 - Training:  92%|█████████▏| 402/435 [06:03<00:11,  2.77it/s]
Epoch 1/50 - Training:  93%|█████████▎| 403/435 [06:03<00:09,  3.42it/s]
Epoch 1/50 - Training:  93%|█████████▎| 404/435 [06:03<00:07,  4.10it/s]
Epoch 1/50 - Training:  93%|█████████▎| 405/435 [06:12<01:24,  2.81s/it]
Epoch 1/50 - Training:  94%|█████████▎| 407/435 [06:12<00:43,  1.57s/it]
Epoch 1/50 - Training:  94%|█████████▍| 408/435 [06:12<00:32,  1.21s/it]
Epoch 1/50 - Training:  94%|█████████▍| 409/435 [06:13<00:24,  1.07it/s]
Epoch 1/50 - Training:  94%|█████████▍| 410/435 [06:13<00:17,  1.39it/s]
Epoch 1/50 - Training:  94%|█████████▍| 411/435 [06:13<00:13,  1.82it/s]
Epoch 1/50 - Training:  95%|█████████▍| 412/435 [06:13<00:09,  2.35it/s]
Epoch 1/50 - Training:  95%|█████████▍| 413/435 [06:13<00:07,  2.99it/s]
Epoch 1/50 - Training:  95%|█████████▌| 414/435 [06:13<00:05,  3.70it/s]
Epoch 1/50 - Training:  95%|█████████▌| 415/435 [06:21<00:52,  2.61s/it]
Epoch 1/50 - Training:  96%|█████████▌| 416/435 [06:21<00:35,  1.87s/it]
Epoch 1/50 - Training:  96%|█████████▌| 418/435 [06:22<00:17,  1.05s/it]
Epoch 1/50 - Training:  97%|█████████▋| 420/435 [06:22<00:10,  1.49it/s]
Epoch 1/50 - Training:  97%|█████████▋| 422/435 [06:22<00:05,  2.19it/s]
Epoch 1/50 - Training:  97%|█████████▋| 424/435 [06:22<00:03,  3.00it/s]
Epoch 1/50 - Training:  98%|█████████▊| 426/435 [06:26<00:08,  1.08it/s]
Epoch 1/50 - Training:  98%|█████████▊| 428/435 [06:27<00:04,  1.52it/s]
Epoch 1/50 - Training:  99%|█████████▉| 430/435 [06:27<00:02,  2.09it/s]
Epoch 1/50 - Training:  99%|█████████▉| 432/435 [06:27<00:01,  2.79it/s]
Epoch 1/50 - Training: 100%|█████████▉| 434/435 [06:27<00:00,  3.61it/s]
Epoch 1/50 - Training: 100%|██████████| 435/435 [06:30<00:00,  1.11it/s]

Epoch 1/50 - Validation:   0%|          | 0/48 [00:00<?, ?it/s]
Epoch 1/50 - Validation:   2%|▏         | 1/48 [00:03<03:00,  3.83s/it]
Epoch 1/50 - Validation:   4%|▍         | 2/48 [00:07<02:43,  3.56s/it]
Epoch 1/50 - Validation:   6%|▋         | 3/48 [00:10<02:41,  3.60s/it]
Epoch 1/50 - Validation:   8%|▊         | 4/48 [00:14<02:34,  3.52s/it]
Epoch 1/50 - Validation:  10%|█         | 5/48 [00:17<02:27,  3.44s/it]
Epoch 1/50 - Validation:  12%|█▎        | 6/48 [00:20<02:23,  3.41s/it]
Epoch 1/50 - Validation:  15%|█▍        | 7/48 [00:24<02:17,  3.34s/it]
Epoch 1/50 - Validation:  17%|█▋        | 8/48 [00:27<02:15,  3.38s/it]
Epoch 1/50 - Validation:  19%|█▉        | 9/48 [00:30<02:06,  3.25s/it]
Epoch 1/50 - Validation:  21%|██        | 10/48 [00:33<02:02,  3.24s/it]
Epoch 1/50 - Validation:  23%|██▎       | 11/48 [00:37<02:01,  3.27s/it]
Epoch 1/50 - Validation:  25%|██▌       | 12/48 [00:40<01:54,  3.17s/it]
Epoch 1/50 - Validation:  27%|██▋       | 13/48 [00:43<01:49,  3.13s/it]
Epoch 1/50 - Validation:  29%|██▉       | 14/48 [00:45<01:44,  3.07s/it]
Epoch 1/50 - Validation:  31%|███▏      | 15/48 [00:48<01:38,  2.99s/it]
Epoch 1/50 - Validation:  33%|███▎      | 16/48 [00:51<01:35,  2.98s/it]
Epoch 1/50 - Validation:  35%|███▌      | 17/48 [00:54<01:30,  2.91s/it]
Epoch 1/50 - Validation:  38%|███▊      | 18/48 [00:57<01:25,  2.86s/it]
Epoch 1/50 - Validation:  40%|███▉      | 19/48 [01:00<01:26,  2.98s/it]
Epoch 1/50 - Validation:  42%|████▏     | 20/48 [01:03<01:23,  2.99s/it]
Epoch 1/50 - Validation:  44%|████▍     | 21/48 [01:06<01:19,  2.96s/it]
Epoch 1/50 - Validation:  46%|████▌     | 22/48 [01:09<01:14,  2.87s/it]
Epoch 1/50 - Validation:  48%|████▊     | 23/48 [01:11<01:09,  2.76s/it]
Epoch 1/50 - Validation:  50%|█████     | 24/48 [01:14<01:07,  2.80s/it]
Epoch 1/50 - Validation:  52%|█████▏    | 25/48 [01:17<01:03,  2.74s/it]
Epoch 1/50 - Validation:  54%|█████▍    | 26/48 [01:19<00:59,  2.70s/it]
Epoch 1/50 - Validation:  56%|█████▋    | 27/48 [01:22<00:55,  2.66s/it]
Epoch 1/50 - Validation:  58%|█████▊    | 28/48 [01:24<00:52,  2.61s/it]
Epoch 1/50 - Validation:  60%|██████    | 29/48 [01:27<00:49,  2.59s/it]
Epoch 1/50 - Validation:  62%|██████▎   | 30/48 [01:29<00:45,  2.54s/it]
Epoch 1/50 - Validation:  65%|██████▍   | 31/48 [01:32<00:43,  2.58s/it]
Epoch 1/50 - Validation:  67%|██████▋   | 32/48 [01:34<00:41,  2.57s/it]
Epoch 1/50 - Validation:  69%|██████▉   | 33/48 [01:37<00:38,  2.54s/it]
Epoch 1/50 - Validation:  71%|███████   | 34/48 [01:39<00:35,  2.53s/it]
Epoch 1/50 - Validation:  73%|███████▎  | 35/48 [01:42<00:33,  2.57s/it]
Epoch 1/50 - Validation:  75%|███████▌  | 36/48 [01:45<00:30,  2.54s/it]
Epoch 1/50 - Validation:  77%|███████▋  | 37/48 [01:47<00:27,  2.51s/it]
Epoch 1/50 - Validation:  79%|███████▉  | 38/48 [01:50<00:26,  2.66s/it]
Epoch 1/50 - Validation:  81%|████████▏ | 39/48 [01:52<00:23,  2.61s/it]
Epoch 1/50 - Validation:  83%|████████▎ | 40/48 [01:55<00:20,  2.56s/it]
Epoch 1/50 - Validation:  85%|████████▌ | 41/48 [01:57<00:17,  2.57s/it]
Epoch 1/50 - Validation:  88%|████████▊ | 42/48 [02:00<00:15,  2.54s/it]
Epoch 1/50 - Validation:  90%|████████▉ | 43/48 [02:02<00:12,  2.51s/it]
Epoch 1/50 - Validation:  92%|█████████▏| 44/48 [02:05<00:10,  2.52s/it]
Epoch 1/50 - Validation:  94%|█████████▍| 45/48 [02:07<00:07,  2.51s/it]
Epoch 1/50 - Validation:  96%|█████████▌| 46/48 [02:10<00:04,  2.50s/it]
Epoch 1/50 - Validation:  98%|█████████▊| 47/48 [02:12<00:02,  2.46s/it]
Epoch 1/50 - Validation: 100%|██████████| 48/48 [02:13<00:00,  1.87s/it]
Epoch 1/50 - Validation: 100%|██████████| 48/48 [02:13<00:00,  2.78s/it]

Epoch 2/50 - Training:   0%|          | 0/435 [00:00<?, ?it/s]
Epoch 2/50 - Training:   0%|          | 1/435 [00:12<1:29:40, 12.40s/it]
Epoch 2/50 - Training:   0%|          | 2/435 [00:12<37:17,  5.17s/it]  
Epoch 2/50 - Training:   1%|          | 3/435 [00:12<20:35,  2.86s/it]
Epoch 2/50 - Training:   1%|          | 4/435 [00:12<12:43,  1.77s/it]
Epoch 2/50 - Training:   1%|          | 5/435 [00:12<08:23,  1.17s/it]
Epoch 2/50 - Training:   1%|▏         | 6/435 [00:12<05:47,  1.23it/s]
Epoch 2/50 - Training:   2%|▏         | 7/435 [00:13<04:08,  1.72it/s]
Epoch 2/50 - Training:   2%|▏         | 8/435 [00:13<03:03,  2.33it/s]
Epoch 2/50 - Training:   2%|▏         | 9/435 [00:13<02:20,  3.03it/s]
Epoch 2/50 - Training:   2%|▏         | 10/435 [00:14<05:12,  1.36it/s]
Epoch 2/50 - Training:   3%|▎         | 11/435 [00:25<26:29,  3.75s/it]
Epoch 2/50 - Training:   3%|▎         | 13/435 [00:25<14:22,  2.04s/it]
Epoch 2/50 - Training:   3%|▎         | 15/435 [00:25<08:50,  1.26s/it]
Epoch 2/50 - Training:   4%|▍         | 17/435 [00:26<05:49,  1.19it/s]
Epoch 2/50 - Training:   4%|▍         | 18/435 [00:26<04:45,  1.46it/s]
Epoch 2/50 - Training:   5%|▍         | 20/435 [00:29<07:45,  1.12s/it]
Epoch 2/50 - Training:   5%|▍         | 21/435 [00:36<16:00,  2.32s/it]
Epoch 2/50 - Training:   5%|▌         | 22/435 [00:36<12:28,  1.81s/it]
Epoch 2/50 - Training:   6%|▌         | 24/435 [00:37<07:45,  1.13s/it]
Epoch 2/50 - Training:   6%|▌         | 26/435 [00:37<05:09,  1.32it/s]
Epoch 2/50 - Training:   6%|▋         | 28/435 [00:37<03:35,  1.89it/s]
Epoch 2/50 - Training:   7%|▋         | 29/435 [00:37<03:00,  2.24it/s]
Epoch 2/50 - Training:   7%|▋         | 30/435 [00:41<08:16,  1.23s/it]
Epoch 2/50 - Training:   7%|▋         | 31/435 [00:47<16:12,  2.41s/it]
Epoch 2/50 - Training:   7%|▋         | 32/435 [00:47<12:16,  1.83s/it]
Epoch 2/50 - Training:   8%|▊         | 33/435 [00:47<09:14,  1.38s/it]
Epoch 2/50 - Training:   8%|▊         | 34/435 [00:48<06:56,  1.04s/it]
Epoch 2/50 - Training:   8%|▊         | 35/435 [00:48<05:16,  1.27it/s]
Epoch 2/50 - Training:   8%|▊         | 36/435 [00:48<03:59,  1.66it/s]
Epoch 2/50 - Training:   9%|▊         | 37/435 [00:48<03:06,  2.14it/s]
Epoch 2/50 - Training:   9%|▊         | 38/435 [00:48<02:27,  2.69it/s]
Epoch 2/50 - Training:   9%|▉         | 39/435 [00:48<02:00,  3.28it/s]
Epoch 2/50 - Training:   9%|▉         | 40/435 [00:52<09:15,  1.41s/it]
Epoch 2/50 - Training:   9%|▉         | 41/435 [00:58<17:37,  2.68s/it]
Epoch 2/50 - Training:  10%|▉         | 42/435 [00:58<12:34,  1.92s/it]
Epoch 2/50 - Training:  10%|▉         | 43/435 [00:58<09:04,  1.39s/it]
Epoch 2/50 - Training:  10%|█         | 44/435 [00:58<06:36,  1.01s/it]
Epoch 2/50 - Training:  10%|█         | 45/435 [00:59<04:51,  1.34it/s]
Epoch 2/50 - Training:  11%|█         | 46/435 [00:59<03:37,  1.79it/s]
Epoch 2/50 - Training:  11%|█         | 47/435 [00:59<02:47,  2.32it/s]
Epoch 2/50 - Training:  11%|█         | 48/435 [00:59<02:12,  2.91it/s]
Epoch 2/50 - Training:  11%|█▏        | 49/435 [00:59<01:48,  3.55it/s]
Epoch 2/50 - Training:  11%|█▏        | 50/435 [01:04<10:57,  1.71s/it]
Epoch 2/50 - Training:  12%|█▏        | 51/435 [01:09<17:06,  2.67s/it]
Epoch 2/50 - Training:  12%|█▏        | 52/435 [01:09<12:08,  1.90s/it]
Epoch 2/50 - Training:  12%|█▏        | 53/435 [01:09<08:46,  1.38s/it]
Epoch 2/50 - Training:  12%|█▏        | 54/435 [01:09<06:24,  1.01s/it]
Epoch 2/50 - Training:  13%|█▎        | 55/435 [01:10<04:46,  1.33it/s]
Epoch 2/50 - Training:  13%|█▎        | 56/435 [01:10<03:36,  1.75it/s]
Epoch 2/50 - Training:  13%|█▎        | 57/435 [01:10<02:49,  2.23it/s]
Epoch 2/50 - Training:  13%|█▎        | 58/435 [01:10<02:15,  2.79it/s]
Epoch 2/50 - Training:  14%|█▎        | 59/435 [01:10<01:50,  3.42it/s]
Epoch 2/50 - Training:  14%|█▍        | 60/435 [01:16<12:00,  1.92s/it]
Epoch 2/50 - Training:  14%|█▍        | 61/435 [01:20<15:03,  2.42s/it]
Epoch 2/50 - Training:  14%|█▍        | 62/435 [01:20<10:44,  1.73s/it]
Epoch 2/50 - Training:  15%|█▍        | 64/435 [01:20<06:03,  1.02it/s]
Epoch 2/50 - Training:  15%|█▍        | 65/435 [01:20<04:43,  1.30it/s]
Epoch 2/50 - Training:  15%|█▌        | 66/435 [01:20<03:39,  1.68it/s]
Epoch 2/50 - Training:  15%|█▌        | 67/435 [01:20<02:50,  2.16it/s]
Epoch 2/50 - Training:  16%|█▌        | 68/435 [01:20<02:13,  2.74it/s]
Epoch 2/50 - Training:  16%|█▌        | 69/435 [01:20<01:46,  3.44it/s]
Epoch 2/50 - Training:  16%|█▌        | 70/435 [01:28<14:04,  2.31s/it]
Epoch 2/50 - Training:  16%|█▋        | 71/435 [01:29<12:33,  2.07s/it]
Epoch 2/50 - Training:  17%|█▋        | 72/435 [01:29<09:03,  1.50s/it]
Epoch 2/50 - Training:  17%|█▋        | 74/435 [01:29<05:08,  1.17it/s]
Epoch 2/50 - Training:  17%|█▋        | 76/435 [01:30<03:19,  1.80it/s]
Epoch 2/50 - Training:  18%|█▊        | 77/435 [01:30<02:44,  2.18it/s]
Epoch 2/50 - Training:  18%|█▊        | 79/435 [01:30<01:53,  3.13it/s]
Epoch 2/50 - Training:  18%|█▊        | 80/435 [01:40<14:12,  2.40s/it]
Epoch 2/50 - Training:  19%|█▊        | 81/435 [01:40<10:58,  1.86s/it]
Epoch 2/50 - Training:  19%|█▉        | 82/435 [01:40<08:21,  1.42s/it]
Epoch 2/50 - Training:  19%|█▉        | 83/435 [01:40<06:18,  1.08s/it]
Epoch 2/50 - Training:  19%|█▉        | 84/435 [01:40<04:45,  1.23it/s]
Epoch 2/50 - Training:  20%|█▉        | 86/435 [01:41<02:54,  2.00it/s]
Epoch 2/50 - Training:  20%|██        | 88/435 [01:41<02:00,  2.89it/s]
Epoch 2/50 - Training:  20%|██        | 89/435 [01:41<01:41,  3.41it/s]
Epoch 2/50 - Training:  21%|██        | 90/435 [01:50<13:56,  2.42s/it]
Epoch 2/50 - Training:  21%|██        | 91/435 [01:50<10:34,  1.84s/it]
Epoch 2/50 - Training:  21%|██▏       | 93/435 [01:51<06:18,  1.11s/it]
Epoch 2/50 - Training:  22%|██▏       | 95/435 [01:51<04:05,  1.38it/s]
Epoch 2/50 - Training:  22%|██▏       | 97/435 [01:51<02:48,  2.00it/s]
Epoch 2/50 - Training:  23%|██▎       | 99/435 [01:51<02:01,  2.77it/s]
Epoch 2/50 - Training:  23%|██▎       | 101/435 [01:59<08:52,  1.59s/it]
Epoch 2/50 - Training:  23%|██▎       | 102/435 [01:59<07:18,  1.32s/it]
Epoch 2/50 - Training:  24%|██▎       | 103/435 [02:00<05:52,  1.06s/it]
Epoch 2/50 - Training:  24%|██▍       | 104/435 [02:00<04:38,  1.19it/s]
Epoch 2/50 - Training:  24%|██▍       | 106/435 [02:00<02:59,  1.83it/s]
Epoch 2/50 - Training:  25%|██▍       | 108/435 [02:00<02:05,  2.61it/s]
Epoch 2/50 - Training:  25%|██▌       | 109/435 [02:00<01:47,  3.03it/s]
Epoch 2/50 - Training:  25%|██▌       | 110/435 [02:08<11:13,  2.07s/it]
Epoch 2/50 - Training:  26%|██▌       | 111/435 [02:08<08:35,  1.59s/it]
Epoch 2/50 - Training:  26%|██▌       | 112/435 [02:08<06:31,  1.21s/it]
Epoch 2/50 - Training:  26%|██▌       | 113/435 [02:09<04:54,  1.09it/s]
Epoch 2/50 - Training:  26%|██▌       | 114/435 [02:09<03:42,  1.44it/s]
Epoch 2/50 - Training:  26%|██▋       | 115/435 [02:09<02:48,  1.90it/s]
Epoch 2/50 - Training:  27%|██▋       | 116/435 [02:09<02:09,  2.46it/s]
Epoch 2/50 - Training:  27%|██▋       | 117/435 [02:09<01:41,  3.12it/s]
Epoch 2/50 - Training:  27%|██▋       | 118/435 [02:09<01:21,  3.89it/s]
Epoch 2/50 - Training:  27%|██▋       | 119/435 [02:09<01:08,  4.62it/s]
Epoch 2/50 - Training:  28%|██▊       | 120/435 [02:16<12:01,  2.29s/it]
Epoch 2/50 - Training:  28%|██▊       | 121/435 [02:17<08:35,  1.64s/it]
Epoch 2/50 - Training:  28%|██▊       | 122/435 [02:17<06:12,  1.19s/it]
Epoch 2/50 - Training:  28%|██▊       | 123/435 [02:17<04:31,  1.15it/s]
Epoch 2/50 - Training:  29%|██▊       | 124/435 [02:17<03:20,  1.55it/s]
Epoch 2/50 - Training:  29%|██▊       | 125/435 [02:17<02:30,  2.07it/s]
Epoch 2/50 - Training:  29%|██▉       | 126/435 [02:17<01:54,  2.70it/s]
Epoch 2/50 - Training:  29%|██▉       | 127/435 [02:17<01:29,  3.45it/s]
Epoch 2/50 - Training:  29%|██▉       | 128/435 [02:17<01:13,  4.20it/s]
Epoch 2/50 - Training:  30%|██▉       | 129/435 [02:17<01:01,  5.00it/s]
Epoch 2/50 - Training:  30%|██▉       | 130/435 [02:25<12:10,  2.40s/it]
Epoch 2/50 - Training:  30%|███       | 132/435 [02:25<06:46,  1.34s/it]
Epoch 2/50 - Training:  31%|███       | 133/435 [02:25<05:18,  1.05s/it]
Epoch 2/50 - Training:  31%|███       | 134/435 [02:26<04:06,  1.22it/s]
Epoch 2/50 - Training:  31%|███       | 135/435 [02:26<03:10,  1.58it/s]
Epoch 2/50 - Training:  31%|███▏      | 136/435 [02:26<02:28,  2.02it/s]
Epoch 2/50 - Training:  31%|███▏      | 137/435 [02:26<01:56,  2.57it/s]
Epoch 2/50 - Training:  32%|███▏      | 138/435 [02:26<01:32,  3.22it/s]
Epoch 2/50 - Training:  32%|███▏      | 139/435 [02:26<01:14,  3.95it/s]
Epoch 2/50 - Training:  32%|███▏      | 140/435 [02:34<11:46,  2.39s/it]
Epoch 2/50 - Training:  32%|███▏      | 141/435 [02:34<08:24,  1.72s/it]
Epoch 2/50 - Training:  33%|███▎      | 142/435 [02:34<06:03,  1.24s/it]
Epoch 2/50 - Training:  33%|███▎      | 143/435 [02:34<04:24,  1.10it/s]
Epoch 2/50 - Training:  33%|███▎      | 144/435 [02:34<03:16,  1.48it/s]
Epoch 2/50 - Training:  33%|███▎      | 145/435 [02:34<02:28,  1.95it/s]
Epoch 2/50 - Training:  34%|███▎      | 146/435 [02:34<01:55,  2.51it/s]
Epoch 2/50 - Training:  34%|███▍      | 147/435 [02:35<01:29,  3.20it/s]
Epoch 2/50 - Training:  34%|███▍      | 148/435 [02:35<01:13,  3.92it/s]
Epoch 2/50 - Training:  34%|███▍      | 149/435 [02:35<01:02,  4.59it/s]
Epoch 2/50 - Training:  34%|███▍      | 150/435 [02:43<12:35,  2.65s/it]
Epoch 2/50 - Training:  35%|███▍      | 151/435 [02:43<08:59,  1.90s/it]
Epoch 2/50 - Training:  35%|███▍      | 152/435 [02:43<06:26,  1.36s/it]
Epoch 2/50 - Training:  35%|███▌      | 153/435 [02:44<04:38,  1.01it/s]
Epoch 2/50 - Training:  36%|███▌      | 155/435 [02:44<02:40,  1.74it/s]
Epoch 2/50 - Training:  36%|███▌      | 156/435 [02:44<02:09,  2.16it/s]
Epoch 2/50 - Training:  36%|███▌      | 157/435 [02:44<01:45,  2.64it/s]
Epoch 2/50 - Training:  36%|███▋      | 158/435 [02:44<01:26,  3.21it/s]
Epoch 2/50 - Training:  37%|███▋      | 159/435 [02:44<01:11,  3.85it/s]
Epoch 2/50 - Training:  37%|███▋      | 160/435 [02:52<10:33,  2.30s/it]
Epoch 2/50 - Training:  37%|███▋      | 161/435 [02:52<07:36,  1.66s/it]
Epoch 2/50 - Training:  37%|███▋      | 162/435 [02:52<05:30,  1.21s/it]
Epoch 2/50 - Training:  37%|███▋      | 163/435 [02:52<04:01,  1.13it/s]
Epoch 2/50 - Training:  38%|███▊      | 164/435 [02:52<02:58,  1.51it/s]
Epoch 2/50 - Training:  38%|███▊      | 165/435 [02:52<02:15,  2.00it/s]
Epoch 2/50 - Training:  38%|███▊      | 166/435 [02:52<01:45,  2.56it/s]
Epoch 2/50 - Training:  38%|███▊      | 167/435 [02:52<01:23,  3.19it/s]
Epoch 2/50 - Training:  39%|███▊      | 168/435 [02:53<01:08,  3.92it/s]
Epoch 2/50 - Training:  39%|███▉      | 169/435 [02:53<00:56,  4.68it/s]
Epoch 2/50 - Training:  39%|███▉      | 170/435 [03:01<11:24,  2.58s/it]
Epoch 2/50 - Training:  39%|███▉      | 171/435 [03:01<08:05,  1.84s/it]
Epoch 2/50 - Training:  40%|███▉      | 172/435 [03:01<05:48,  1.32s/it]
Epoch 2/50 - Training:  40%|███▉      | 173/435 [03:01<04:13,  1.03it/s]
Epoch 2/50 - Training:  40%|████      | 174/435 [03:01<03:07,  1.39it/s]
Epoch 2/50 - Training:  40%|████      | 175/435 [03:01<02:21,  1.84it/s]
Epoch 2/50 - Training:  40%|████      | 176/435 [03:02<01:50,  2.35it/s]
Epoch 2/50 - Training:  41%|████      | 177/435 [03:02<01:28,  2.93it/s]
Epoch 2/50 - Training:  41%|████      | 178/435 [03:02<01:11,  3.59it/s]
Epoch 2/50 - Training:  41%|████      | 179/435 [03:02<01:00,  4.22it/s]
Epoch 2/50 - Training:  41%|████▏     | 180/435 [03:10<10:16,  2.42s/it]
Epoch 2/50 - Training:  42%|████▏     | 182/435 [03:10<05:41,  1.35s/it]
Epoch 2/50 - Training:  42%|████▏     | 183/435 [03:10<04:23,  1.04s/it]
Epoch 2/50 - Training:  42%|████▏     | 184/435 [03:10<03:22,  1.24it/s]
Epoch 2/50 - Training:  43%|████▎     | 185/435 [03:10<02:35,  1.61it/s]
Epoch 2/50 - Training:  43%|████▎     | 186/435 [03:10<01:59,  2.08it/s]
Epoch 2/50 - Training:  43%|████▎     | 187/435 [03:10<01:33,  2.65it/s]
Epoch 2/50 - Training:  43%|████▎     | 188/435 [03:10<01:14,  3.31it/s]
Epoch 2/50 - Training:  43%|████▎     | 189/435 [03:11<01:01,  3.97it/s]
Epoch 2/50 - Training:  44%|████▎     | 190/435 [03:18<10:02,  2.46s/it]
Epoch 2/50 - Training:  44%|████▍     | 192/435 [03:19<05:36,  1.39s/it]
Epoch 2/50 - Training:  44%|████▍     | 193/435 [03:19<04:21,  1.08s/it]
Epoch 2/50 - Training:  45%|████▍     | 194/435 [03:19<03:21,  1.20it/s]
Epoch 2/50 - Training:  45%|████▍     | 195/435 [03:19<02:35,  1.54it/s]
Epoch 2/50 - Training:  45%|████▌     | 196/435 [03:19<02:01,  1.96it/s]
Epoch 2/50 - Training:  45%|████▌     | 197/435 [03:19<01:35,  2.49it/s]
Epoch 2/50 - Training:  46%|████▌     | 198/435 [03:19<01:17,  3.06it/s]
Epoch 2/50 - Training:  46%|████▌     | 199/435 [03:20<01:03,  3.69it/s]
Epoch 2/50 - Training:  46%|████▌     | 200/435 [03:28<10:03,  2.57s/it]
Epoch 2/50 - Training:  46%|████▋     | 202/435 [03:28<05:35,  1.44s/it]
Epoch 2/50 - Training:  47%|████▋     | 203/435 [03:28<04:18,  1.11s/it]
Epoch 2/50 - Training:  47%|████▋     | 204/435 [03:28<03:17,  1.17it/s]
Epoch 2/50 - Training:  47%|████▋     | 205/435 [03:28<02:30,  1.53it/s]
Epoch 2/50 - Training:  47%|████▋     | 206/435 [03:28<01:55,  1.98it/s]
Epoch 2/50 - Training:  48%|████▊     | 207/435 [03:28<01:29,  2.54it/s]
Epoch 2/50 - Training:  48%|████▊     | 208/435 [03:29<01:11,  3.18it/s]
Epoch 2/50 - Training:  48%|████▊     | 209/435 [03:29<00:57,  3.90it/s]
Epoch 2/50 - Training:  48%|████▊     | 210/435 [03:36<09:05,  2.42s/it]
Epoch 2/50 - Training:  49%|████▊     | 212/435 [03:36<05:03,  1.36s/it]
Epoch 2/50 - Training:  49%|████▉     | 213/435 [03:37<03:55,  1.06s/it]
Epoch 2/50 - Training:  49%|████▉     | 214/435 [03:37<03:00,  1.22it/s]
Epoch 2/50 - Training:  49%|████▉     | 215/435 [03:37<02:19,  1.58it/s]
Epoch 2/50 - Training:  50%|████▉     | 216/435 [03:37<01:48,  2.02it/s]
Epoch 2/50 - Training:  50%|████▉     | 217/435 [03:37<01:24,  2.57it/s]
Epoch 2/50 - Training:  50%|█████     | 218/435 [03:37<01:07,  3.20it/s]
Epoch 2/50 - Training:  50%|█████     | 219/435 [03:37<00:57,  3.76it/s]
Epoch 2/50 - Training:  51%|█████     | 220/435 [03:45<08:55,  2.49s/it]
Epoch 2/50 - Training:  51%|█████     | 222/435 [03:45<04:57,  1.40s/it]
Epoch 2/50 - Training:  51%|█████▏    | 223/435 [03:46<03:50,  1.09s/it]
Epoch 2/50 - Training:  51%|█████▏    | 224/435 [03:46<02:57,  1.19it/s]
Epoch 2/50 - Training:  52%|█████▏    | 225/435 [03:46<02:16,  1.54it/s]
Epoch 2/50 - Training:  52%|█████▏    | 226/435 [03:46<01:45,  1.99it/s]
Epoch 2/50 - Training:  52%|█████▏    | 227/435 [03:46<01:22,  2.53it/s]
Epoch 2/50 - Training:  52%|█████▏    | 228/435 [03:46<01:05,  3.14it/s]
Epoch 2/50 - Training:  53%|█████▎    | 229/435 [03:46<00:54,  3.79it/s]
Epoch 2/50 - Training:  53%|█████▎    | 230/435 [03:54<07:59,  2.34s/it]
Epoch 2/50 - Training:  53%|█████▎    | 232/435 [03:54<04:26,  1.31s/it]
Epoch 2/50 - Training:  54%|█████▎    | 233/435 [03:54<03:25,  1.02s/it]
Epoch 2/50 - Training:  54%|█████▍    | 235/435 [03:54<02:07,  1.57it/s]
Epoch 2/50 - Training:  54%|█████▍    | 237/435 [03:54<01:26,  2.28it/s]
Epoch 2/50 - Training:  55%|█████▍    | 238/435 [03:54<01:13,  2.67it/s]
Epoch 2/50 - Training:  55%|█████▍    | 239/435 [03:55<01:02,  3.16it/s]
Epoch 2/50 - Training:  55%|█████▌    | 240/435 [04:03<07:11,  2.21s/it]
Epoch 2/50 - Training:  56%|█████▌    | 242/435 [04:03<04:17,  1.33s/it]
Epoch 2/50 - Training:  56%|█████▌    | 243/435 [04:03<03:22,  1.05s/it]
Epoch 2/50 - Training:  56%|█████▌    | 244/435 [04:03<02:37,  1.21it/s]
Epoch 2/50 - Training:  56%|█████▋    | 245/435 [04:03<02:02,  1.55it/s]
Epoch 2/50 - Training:  57%|█████▋    | 246/435 [04:03<01:34,  1.99it/s]
Epoch 2/50 - Training:  57%|█████▋    | 247/435 [04:03<01:14,  2.52it/s]
Epoch 2/50 - Training:  57%|█████▋    | 248/435 [04:03<00:59,  3.15it/s]
Epoch 2/50 - Training:  57%|█████▋    | 249/435 [04:04<00:49,  3.79it/s]
Epoch 2/50 - Training:  57%|█████▋    | 250/435 [04:11<07:03,  2.29s/it]
Epoch 2/50 - Training:  58%|█████▊    | 252/435 [04:11<03:56,  1.29s/it]
Epoch 2/50 - Training:  58%|█████▊    | 253/435 [04:11<03:04,  1.01s/it]
Epoch 2/50 - Training:  58%|█████▊    | 254/435 [04:11<02:21,  1.28it/s]
Epoch 2/50 - Training:  59%|█████▊    | 255/435 [04:11<01:49,  1.65it/s]
Epoch 2/50 - Training:  59%|█████▉    | 256/435 [04:12<01:25,  2.11it/s]
Epoch 2/50 - Training:  59%|█████▉    | 257/435 [04:12<01:07,  2.64it/s]
Epoch 2/50 - Training:  59%|█████▉    | 258/435 [04:12<00:54,  3.27it/s]
Epoch 2/50 - Training:  60%|█████▉    | 259/435 [04:12<00:44,  3.94it/s]
Epoch 2/50 - Training:  60%|█████▉    | 260/435 [04:19<06:28,  2.22s/it]
Epoch 2/50 - Training:  60%|██████    | 262/435 [04:19<03:35,  1.25s/it]
Epoch 2/50 - Training:  60%|██████    | 263/435 [04:19<02:46,  1.03it/s]
Epoch 2/50 - Training:  61%|██████    | 264/435 [04:19<02:07,  1.34it/s]
Epoch 2/50 - Training:  61%|██████    | 265/435 [04:19<01:38,  1.73it/s]
Epoch 2/50 - Training:  61%|██████    | 266/435 [04:19<01:16,  2.22it/s]
Epoch 2/50 - Training:  61%|██████▏   | 267/435 [04:20<00:59,  2.81it/s]
Epoch 2/50 - Training:  62%|██████▏   | 268/435 [04:20<00:48,  3.47it/s]
Epoch 2/50 - Training:  62%|██████▏   | 269/435 [04:20<00:39,  4.18it/s]
Epoch 2/50 - Training:  62%|██████▏   | 270/435 [04:27<06:29,  2.36s/it]
Epoch 2/50 - Training:  63%|██████▎   | 272/435 [04:27<03:35,  1.32s/it]
Epoch 2/50 - Training:  63%|██████▎   | 273/435 [04:28<02:46,  1.03s/it]
Epoch 2/50 - Training:  63%|██████▎   | 274/435 [04:28<02:08,  1.25it/s]
Epoch 2/50 - Training:  63%|██████▎   | 275/435 [04:28<01:39,  1.61it/s]
Epoch 2/50 - Training:  63%|██████▎   | 276/435 [04:28<01:17,  2.04it/s]
Epoch 2/50 - Training:  64%|██████▎   | 277/435 [04:28<01:02,  2.52it/s]
Epoch 2/50 - Training:  64%|██████▍   | 278/435 [04:28<00:50,  3.10it/s]
Epoch 2/50 - Training:  64%|██████▍   | 279/435 [04:28<00:41,  3.72it/s]
Epoch 2/50 - Training:  64%|██████▍   | 280/435 [04:36<05:53,  2.28s/it]
Epoch 2/50 - Training:  65%|██████▍   | 282/435 [04:36<03:16,  1.28s/it]
Epoch 2/50 - Training:  65%|██████▌   | 283/435 [04:36<02:31,  1.00it/s]
Epoch 2/50 - Training:  65%|██████▌   | 284/435 [04:36<01:55,  1.31it/s]
Epoch 2/50 - Training:  66%|██████▌   | 285/435 [04:36<01:28,  1.70it/s]
Epoch 2/50 - Training:  66%|██████▌   | 286/435 [04:36<01:08,  2.18it/s]
Epoch 2/50 - Training:  66%|██████▌   | 287/435 [04:36<00:54,  2.73it/s]
Epoch 2/50 - Training:  66%|██████▌   | 288/435 [04:36<00:43,  3.37it/s]
Epoch 2/50 - Training:  66%|██████▋   | 289/435 [04:37<00:35,  4.14it/s]
Epoch 2/50 - Training:  67%|██████▋   | 290/435 [04:43<05:08,  2.12s/it]
Epoch 2/50 - Training:  67%|██████▋   | 292/435 [04:43<02:50,  1.19s/it]
Epoch 2/50 - Training:  67%|██████▋   | 293/435 [04:44<02:11,  1.08it/s]
Epoch 2/50 - Training:  68%|██████▊   | 294/435 [04:44<01:41,  1.39it/s]
Epoch 2/50 - Training:  68%|██████▊   | 295/435 [04:44<01:18,  1.78it/s]
Epoch 2/50 - Training:  68%|██████▊   | 296/435 [04:44<01:01,  2.28it/s]
Epoch 2/50 - Training:  68%|██████▊   | 297/435 [04:44<00:48,  2.85it/s]
Epoch 2/50 - Training:  69%|██████▊   | 298/435 [04:44<00:40,  3.41it/s]
Epoch 2/50 - Training:  69%|██████▊   | 299/435 [04:44<00:33,  4.06it/s]
Epoch 2/50 - Training:  69%|██████▉   | 300/435 [04:51<04:57,  2.20s/it]
Epoch 2/50 - Training:  69%|██████▉   | 302/435 [04:51<02:44,  1.23s/it]
Epoch 2/50 - Training:  70%|██████▉   | 303/435 [04:52<02:07,  1.03it/s]
Epoch 2/50 - Training:  70%|██████▉   | 304/435 [04:52<01:38,  1.33it/s]
Epoch 2/50 - Training:  70%|███████   | 305/435 [04:52<01:15,  1.72it/s]
Epoch 2/50 - Training:  70%|███████   | 306/435 [04:52<00:58,  2.20it/s]
Epoch 2/50 - Training:  71%|███████   | 307/435 [04:52<00:46,  2.76it/s]
Epoch 2/50 - Training:  71%|███████   | 308/435 [04:52<00:37,  3.41it/s]
Epoch 2/50 - Training:  71%|███████   | 309/435 [04:52<00:30,  4.08it/s]
Epoch 2/50 - Training:  71%|███████▏  | 310/435 [04:59<04:23,  2.11s/it]
Epoch 2/50 - Training:  72%|███████▏  | 312/435 [04:59<02:25,  1.19s/it]
Epoch 2/50 - Training:  72%|███████▏  | 313/435 [04:59<01:53,  1.08it/s]
Epoch 2/50 - Training:  72%|███████▏  | 314/435 [04:59<01:27,  1.39it/s]
Epoch 2/50 - Training:  72%|███████▏  | 315/435 [04:59<01:06,  1.79it/s]
Epoch 2/50 - Training:  73%|███████▎  | 316/435 [05:00<00:51,  2.30it/s]
Epoch 2/50 - Training:  73%|███████▎  | 317/435 [05:00<00:40,  2.91it/s]
Epoch 2/50 - Training:  73%|███████▎  | 318/435 [05:00<00:32,  3.60it/s]
Epoch 2/50 - Training:  73%|███████▎  | 319/435 [05:00<00:27,  4.29it/s]
Epoch 2/50 - Training:  74%|███████▎  | 320/435 [05:07<04:06,  2.15s/it]
Epoch 2/50 - Training:  74%|███████▍  | 322/435 [05:07<02:16,  1.21s/it]
Epoch 2/50 - Training:  74%|███████▍  | 323/435 [05:07<01:46,  1.05it/s]
Epoch 2/50 - Training:  74%|███████▍  | 324/435 [05:07<01:22,  1.35it/s]
Epoch 2/50 - Training:  75%|███████▍  | 325/435 [05:07<01:03,  1.73it/s]
Epoch 2/50 - Training:  75%|███████▍  | 326/435 [05:07<00:49,  2.19it/s]
Epoch 2/50 - Training:  75%|███████▌  | 327/435 [05:08<00:39,  2.72it/s]
Epoch 2/50 - Training:  75%|███████▌  | 328/435 [05:08<00:32,  3.32it/s]
Epoch 2/50 - Training:  76%|███████▌  | 329/435 [05:08<00:27,  3.89it/s]
Epoch 2/50 - Training:  76%|███████▌  | 330/435 [05:14<03:18,  1.90s/it]
Epoch 2/50 - Training:  76%|███████▋  | 332/435 [05:14<01:51,  1.08s/it]
Epoch 2/50 - Training:  77%|███████▋  | 333/435 [05:14<01:26,  1.19it/s]
Epoch 2/50 - Training:  77%|███████▋  | 334/435 [05:14<01:06,  1.53it/s]
Epoch 2/50 - Training:  77%|███████▋  | 335/435 [05:14<00:50,  1.97it/s]
Epoch 2/50 - Training:  77%|███████▋  | 336/435 [05:14<00:39,  2.50it/s]
Epoch 2/50 - Training:  77%|███████▋  | 337/435 [05:14<00:31,  3.13it/s]
Epoch 2/50 - Training:  78%|███████▊  | 338/435 [05:15<00:25,  3.86it/s]
Epoch 2/50 - Training:  78%|███████▊  | 339/435 [05:15<00:21,  4.50it/s]
Epoch 2/50 - Training:  78%|███████▊  | 340/435 [05:21<03:18,  2.09s/it]
Epoch 2/50 - Training:  79%|███████▊  | 342/435 [05:21<01:49,  1.17s/it]
Epoch 2/50 - Training:  79%|███████▉  | 343/435 [05:22<01:24,  1.09it/s]
Epoch 2/50 - Training:  79%|███████▉  | 344/435 [05:22<01:04,  1.41it/s]
Epoch 2/50 - Training:  79%|███████▉  | 345/435 [05:22<00:49,  1.82it/s]
Epoch 2/50 - Training:  80%|███████▉  | 346/435 [05:22<00:38,  2.34it/s]
Epoch 2/50 - Training:  80%|███████▉  | 347/435 [05:22<00:30,  2.92it/s]
Epoch 2/50 - Training:  80%|████████  | 348/435 [05:22<00:23,  3.64it/s]
Epoch 2/50 - Training:  80%|████████  | 349/435 [05:22<00:19,  4.38it/s]
Epoch 2/50 - Training:  80%|████████  | 350/435 [05:30<03:24,  2.40s/it]
Epoch 2/50 - Training:  81%|████████  | 352/435 [05:30<01:51,  1.34s/it]
Epoch 2/50 - Training:  81%|████████  | 353/435 [05:30<01:25,  1.05s/it]
Epoch 2/50 - Training:  81%|████████▏ | 354/435 [05:30<01:05,  1.25it/s]
Epoch 2/50 - Training:  82%|████████▏ | 355/435 [05:30<00:49,  1.62it/s]
Epoch 2/50 - Training:  82%|████████▏ | 356/435 [05:31<00:38,  2.08it/s]
Epoch 2/50 - Training:  82%|████████▏ | 357/435 [05:31<00:29,  2.63it/s]
Epoch 2/50 - Training:  82%|████████▏ | 358/435 [05:31<00:23,  3.25it/s]
Epoch 2/50 - Training:  83%|████████▎ | 360/435 [05:39<02:27,  1.97s/it]
Epoch 2/50 - Training:  83%|████████▎ | 362/435 [05:39<01:29,  1.23s/it]
Epoch 2/50 - Training:  83%|████████▎ | 363/435 [05:39<01:10,  1.01it/s]
Epoch 2/50 - Training:  84%|████████▎ | 364/435 [05:39<00:55,  1.29it/s]
Epoch 2/50 - Training:  84%|████████▍ | 366/435 [05:39<00:34,  1.98it/s]
Epoch 2/50 - Training:  84%|████████▍ | 367/435 [05:40<00:28,  2.36it/s]
Epoch 2/50 - Training:  85%|████████▍ | 368/435 [05:40<00:23,  2.82it/s]
Epoch 2/50 - Training:  85%|████████▍ | 369/435 [05:40<00:19,  3.35it/s]
Epoch 2/50 - Training:  85%|████████▌ | 370/435 [05:48<02:34,  2.37s/it]
Epoch 2/50 - Training:  86%|████████▌ | 372/435 [05:48<01:26,  1.38s/it]
Epoch 2/50 - Training:  86%|████████▌ | 373/435 [05:48<01:06,  1.08s/it]
Epoch 2/50 - Training:  86%|████████▌ | 374/435 [05:48<00:50,  1.20it/s]
Epoch 2/50 - Training:  86%|████████▌ | 375/435 [05:48<00:38,  1.56it/s]
Epoch 2/50 - Training:  86%|████████▋ | 376/435 [05:48<00:29,  2.02it/s]
Epoch 2/50 - Training:  87%|████████▋ | 377/435 [05:49<00:22,  2.60it/s]
Epoch 2/50 - Training:  87%|████████▋ | 378/435 [05:49<00:17,  3.30it/s]
Epoch 2/50 - Training:  87%|████████▋ | 379/435 [05:49<00:14,  3.99it/s]
Epoch 2/50 - Training:  87%|████████▋ | 380/435 [05:57<02:30,  2.73s/it]
Epoch 2/50 - Training:  88%|████████▊ | 382/435 [05:58<01:20,  1.53s/it]
Epoch 2/50 - Training:  88%|████████▊ | 383/435 [05:58<01:01,  1.18s/it]
Epoch 2/50 - Training:  88%|████████▊ | 384/435 [05:58<00:45,  1.11it/s]
Epoch 2/50 - Training:  89%|████████▊ | 385/435 [05:58<00:34,  1.46it/s]
Epoch 2/50 - Training:  89%|████████▊ | 386/435 [05:58<00:25,  1.90it/s]
Epoch 2/50 - Training:  89%|████████▉ | 387/435 [05:58<00:20,  2.39it/s]
Epoch 2/50 - Training:  89%|████████▉ | 388/435 [05:58<00:15,  3.00it/s]
Epoch 2/50 - Training:  89%|████████▉ | 389/435 [05:58<00:12,  3.68it/s]
Epoch 2/50 - Training:  90%|████████▉ | 390/435 [06:06<01:54,  2.55s/it]
Epoch 2/50 - Training:  90%|█████████ | 392/435 [06:07<01:01,  1.42s/it]
Epoch 2/50 - Training:  91%|█████████ | 394/435 [06:07<00:36,  1.11it/s]
Epoch 2/50 - Training:  91%|█████████ | 396/435 [06:07<00:23,  1.64it/s]
Epoch 2/50 - Training:  91%|█████████▏| 397/435 [06:07<00:19,  1.97it/s]
Epoch 2/50 - Training:  92%|█████████▏| 399/435 [06:07<00:12,  2.79it/s]
Epoch 2/50 - Training:  92%|█████████▏| 400/435 [06:15<01:05,  1.86s/it]
Epoch 2/50 - Training:  92%|█████████▏| 402/435 [06:15<00:39,  1.19s/it]
Epoch 2/50 - Training:  93%|█████████▎| 404/435 [06:15<00:25,  1.23it/s]
Epoch 2/50 - Training:  93%|█████████▎| 405/435 [06:15<00:20,  1.49it/s]
Epoch 2/50 - Training:  93%|█████████▎| 406/435 [06:15<00:15,  1.82it/s]
Epoch 2/50 - Training:  94%|█████████▎| 407/435 [06:16<00:12,  2.24it/s]
Epoch 2/50 - Training:  94%|█████████▍| 408/435 [06:16<00:09,  2.76it/s]
Epoch 2/50 - Training:  94%|█████████▍| 409/435 [06:16<00:07,  3.41it/s]
Epoch 2/50 - Training:  94%|█████████▍| 410/435 [06:23<00:55,  2.21s/it]
Epoch 2/50 - Training:  95%|█████████▍| 412/435 [06:23<00:29,  1.27s/it]
Epoch 2/50 - Training:  95%|█████████▌| 414/435 [06:23<00:17,  1.22it/s]
Epoch 2/50 - Training:  95%|█████████▌| 415/435 [06:24<00:13,  1.49it/s]
Epoch 2/50 - Training:  96%|█████████▌| 416/435 [06:24<00:10,  1.85it/s]
Epoch 2/50 - Training:  96%|█████████▌| 417/435 [06:24<00:07,  2.29it/s]
Epoch 2/50 - Training:  96%|█████████▌| 418/435 [06:24<00:05,  2.84it/s]
Epoch 2/50 - Training:  97%|█████████▋| 420/435 [06:29<00:20,  1.34s/it]
Epoch 2/50 - Training:  97%|█████████▋| 422/435 [06:29<00:11,  1.17it/s]
Epoch 2/50 - Training:  97%|█████████▋| 424/435 [06:30<00:06,  1.70it/s]
Epoch 2/50 - Training:  98%|█████████▊| 425/435 [06:30<00:04,  2.02it/s]
Epoch 2/50 - Training:  98%|█████████▊| 426/435 [06:30<00:03,  2.46it/s]
Epoch 2/50 - Training:  98%|█████████▊| 427/435 [06:30<00:02,  3.00it/s]
Epoch 2/50 - Training:  98%|█████████▊| 428/435 [06:30<00:01,  3.61it/s]
Epoch 2/50 - Training:  99%|█████████▉| 430/435 [06:34<00:04,  1.06it/s]
Epoch 2/50 - Training:  99%|█████████▉| 432/435 [06:34<00:01,  1.60it/s]
Epoch 2/50 - Training: 100%|█████████▉| 434/435 [06:34<00:00,  2.28it/s]
Epoch 2/50 - Training: 100%|██████████| 435/435 [06:34<00:00,  1.10it/s]

Epoch 2/50 - Validation:   0%|          | 0/48 [00:00<?, ?it/s]
Epoch 2/50 - Validation:   2%|▏         | 1/48 [00:04<03:43,  4.75s/it]
Epoch 2/50 - Validation:   4%|▍         | 2/48 [00:09<03:29,  4.55s/it]
Epoch 2/50 - Validation:   6%|▋         | 3/48 [00:13<03:28,  4.64s/it]
Epoch 2/50 - Validation:   8%|▊         | 4/48 [00:18<03:17,  4.48s/it]
Epoch 2/50 - Validation:  10%|█         | 5/48 [00:22<03:04,  4.29s/it]
Epoch 2/50 - Validation:  12%|█▎        | 6/48 [00:26<02:59,  4.27s/it]
Epoch 2/50 - Validation:  15%|█▍        | 7/48 [00:30<02:50,  4.15s/it]
Epoch 2/50 - Validation:  17%|█▋        | 8/48 [00:34<02:45,  4.15s/it]
Epoch 2/50 - Validation:  19%|█▉        | 9/48 [00:38<02:36,  4.01s/it]
Epoch 2/50 - Validation:  21%|██        | 10/48 [00:41<02:30,  3.96s/it]
Epoch 2/50 - Validation:  23%|██▎       | 11/48 [00:45<02:27,  3.98s/it]
Epoch 2/50 - Validation:  25%|██▌       | 12/48 [00:49<02:18,  3.83s/it]
Epoch 2/50 - Validation:  27%|██▋       | 13/48 [00:53<02:13,  3.82s/it]
Epoch 2/50 - Validation:  29%|██▉       | 14/48 [00:56<02:08,  3.79s/it]
Epoch 2/50 - Validation:  31%|███▏      | 15/48 [01:00<02:01,  3.70s/it]
Epoch 2/50 - Validation:  33%|███▎      | 16/48 [01:04<01:58,  3.70s/it]
Epoch 2/50 - Validation:  35%|███▌      | 17/48 [01:07<01:53,  3.66s/it]
Epoch 2/50 - Validation:  38%|███▊      | 18/48 [01:10<01:45,  3.52s/it]
Epoch 2/50 - Validation:  40%|███▉      | 19/48 [01:14<01:40,  3.46s/it]
Epoch 2/50 - Validation:  42%|████▏     | 20/48 [01:17<01:35,  3.39s/it]
Epoch 2/50 - Validation:  44%|████▍     | 21/48 [01:20<01:31,  3.39s/it]
Epoch 2/50 - Validation:  46%|████▌     | 22/48 [01:24<01:26,  3.34s/it]
Epoch 2/50 - Validation:  48%|████▊     | 23/48 [01:27<01:22,  3.29s/it]
Epoch 2/50 - Validation:  50%|█████     | 24/48 [01:30<01:19,  3.32s/it]
Epoch 2/50 - Validation:  52%|█████▏    | 25/48 [01:33<01:15,  3.27s/it]
Epoch 2/50 - Validation:  54%|█████▍    | 26/48 [01:36<01:11,  3.23s/it]
Epoch 2/50 - Validation:  56%|█████▋    | 27/48 [01:40<01:07,  3.22s/it]
Epoch 2/50 - Validation:  58%|█████▊    | 28/48 [01:43<01:04,  3.23s/it]
Epoch 2/50 - Validation:  60%|██████    | 29/48 [01:46<01:01,  3.24s/it]
Epoch 2/50 - Validation:  62%|██████▎   | 30/48 [01:49<00:57,  3.20s/it]
Epoch 2/50 - Validation:  65%|██████▍   | 31/48 [01:53<00:55,  3.24s/it]
Epoch 2/50 - Validation:  67%|██████▋   | 32/48 [01:56<00:51,  3.23s/it]
Epoch 2/50 - Validation:  69%|██████▉   | 33/48 [01:59<00:48,  3.21s/it]
Epoch 2/50 - Validation:  71%|███████   | 34/48 [02:02<00:44,  3.21s/it]
Epoch 2/50 - Validation:  73%|███████▎  | 35/48 [02:06<00:43,  3.33s/it]
Epoch 2/50 - Validation:  75%|███████▌  | 36/48 [02:09<00:39,  3.27s/it]
Epoch 2/50 - Validation:  77%|███████▋  | 37/48 [02:12<00:35,  3.23s/it]
Epoch 2/50 - Validation:  79%|███████▉  | 38/48 [02:16<00:33,  3.32s/it]
Epoch 2/50 - Validation:  81%|████████▏ | 39/48 [02:19<00:28,  3.21s/it]
Epoch 2/50 - Validation:  83%|████████▎ | 40/48 [02:22<00:25,  3.16s/it]
Epoch 2/50 - Validation:  85%|████████▌ | 41/48 [02:25<00:22,  3.18s/it]
Epoch 2/50 - Validation:  88%|████████▊ | 42/48 [02:28<00:19,  3.19s/it]
Epoch 2/50 - Validation:  90%|████████▉ | 43/48 [02:31<00:15,  3.18s/it]
Epoch 2/50 - Validation:  92%|█████████▏| 44/48 [02:34<00:12,  3.19s/it]
Epoch 2/50 - Validation:  94%|█████████▍| 45/48 [02:38<00:09,  3.23s/it]
Epoch 2/50 - Validation:  96%|█████████▌| 46/48 [02:41<00:06,  3.24s/it]
Epoch 2/50 - Validation:  98%|█████████▊| 47/48 [02:44<00:03,  3.18s/it]
Epoch 2/50 - Validation: 100%|██████████| 48/48 [02:45<00:00,  2.42s/it]
Epoch 2/50 - Validation: 100%|██████████| 48/48 [02:45<00:00,  3.44s/it]

Epoch 3/50 - Training:   0%|          | 0/435 [00:00<?, ?it/s]
Epoch 3/50 - Training:   0%|          | 1/435 [00:14<1:48:09, 14.95s/it]
Epoch 3/50 - Training:   0%|          | 2/435 [00:15<44:52,  6.22s/it]  
Epoch 3/50 - Training:   1%|          | 4/435 [00:15<17:07,  2.38s/it]
Epoch 3/50 - Training:   1%|▏         | 6/435 [00:15<09:19,  1.30s/it]
Epoch 3/50 - Training:   2%|▏         | 7/435 [00:15<07:08,  1.00s/it]
Epoch 3/50 - Training:   2%|▏         | 8/435 [00:15<05:25,  1.31it/s]
Epoch 3/50 - Training:   2%|▏         | 10/435 [00:15<03:21,  2.11it/s]
Epoch 3/50 - Training:   2%|▏         | 10/435 [00:26<03:21,  2.11it/s]
Epoch 3/50 - Training:   3%|▎         | 11/435 [00:26<20:33,  2.91s/it]
Epoch 3/50 - Training:   3%|▎         | 12/435 [00:26<15:38,  2.22s/it]
Epoch 3/50 - Training:   3%|▎         | 13/435 [00:26<11:42,  1.67s/it]
Epoch 3/50 - Training:   3%|▎         | 14/435 [00:27<08:42,  1.24s/it]
Epoch 3/50 - Training:   3%|▎         | 15/435 [00:27<06:33,  1.07it/s]
Epoch 3/50 - Training:   4%|▎         | 16/435 [00:27<04:59,  1.40it/s]
Epoch 3/50 - Training:   4%|▍         | 17/435 [00:27<03:51,  1.81it/s]
Epoch 3/50 - Training:   4%|▍         | 18/435 [00:27<03:02,  2.28it/s]
Epoch 3/50 - Training:   4%|▍         | 19/435 [00:27<02:27,  2.82it/s]
Epoch 3/50 - Training:   5%|▍         | 20/435 [00:28<02:03,  3.36it/s]
Epoch 3/50 - Training:   5%|▍         | 21/435 [00:36<17:53,  2.59s/it]
Epoch 3/50 - Training:   5%|▌         | 22/435 [00:36<13:43,  1.99s/it]
Epoch 3/50 - Training:   5%|▌         | 23/435 [00:37<10:37,  1.55s/it]
Epoch 3/50 - Training:   6%|▌         | 24/435 [00:37<07:40,  1.12s/it]
Epoch 3/50 - Training:   6%|▌         | 25/435 [00:37<05:38,  1.21it/s]
Epoch 3/50 - Training:   6%|▌         | 26/435 [00:37<04:12,  1.62it/s]
Epoch 3/50 - Training:   6%|▌         | 27/435 [00:37<03:12,  2.12it/s]
Epoch 3/50 - Training:   6%|▋         | 28/435 [00:37<02:32,  2.67it/s]
Epoch 3/50 - Training:   7%|▋         | 29/435 [00:37<02:03,  3.28it/s]
Epoch 3/50 - Training:   7%|▋         | 30/435 [00:38<01:43,  3.91it/s]
Epoch 3/50 - Training:   7%|▋         | 31/435 [00:45<16:30,  2.45s/it]
Epoch 3/50 - Training:   7%|▋         | 32/435 [00:46<13:26,  2.00s/it]
Epoch 3/50 - Training:   8%|▊         | 33/435 [00:47<10:16,  1.53s/it]
Epoch 3/50 - Training:   8%|▊         | 34/435 [00:47<07:25,  1.11s/it]
Epoch 3/50 - Training:   8%|▊         | 35/435 [00:47<05:28,  1.22it/s]
Epoch 3/50 - Training:   8%|▊         | 36/435 [00:47<04:07,  1.61it/s]
Epoch 3/50 - Training:   9%|▊         | 37/435 [00:47<03:11,  2.07it/s]
Epoch 3/50 - Training:   9%|▊         | 38/435 [00:47<02:32,  2.60it/s]
Epoch 3/50 - Training:   9%|▉         | 39/435 [00:47<02:05,  3.16it/s]
Epoch 3/50 - Training:   9%|▉         | 40/435 [00:48<01:44,  3.79it/s]
Epoch 3/50 - Training:   9%|▉         | 41/435 [00:56<17:38,  2.69s/it]slurmstepd: error: *** REASON: burst_buffer/lua: Stage-out in progress ***
slurmstepd: error: *** JOB 541349 ON ip-10-0-135-126 CANCELLED AT 2024-11-27T02:28:13 ***
slurmstepd: error: *** REASON: burst_buffer/lua: Stage-out in progress ***

Epoch 3/50 - Training:  10%|▉         | 42/435 [00:57<14:09,  2.16s/it]
Epoch 3/50 - Training:  10%|▉         | 43/435 [00:57<10:22,  1.59s/it]
Epoch 3/50 - Training:  10%|█         | 44/435 [00:57<07:27,  1.15s/it]
Epoch 3/50 - Training:  10%|█         | 45/435 [00:57<05:27,  1.19it/s]
Epoch 3/50 - Training:  11%|█         | 46/435 [00:57<04:04,  1.59it/s]
Epoch 3/50 - Training:  11%|█         | 47/435 [00:58<03:06,  2.08it/s]