Hello everyone,
I try to use pegasus on squad dataset. I am using notebook 2 for pegasus fine-tuning on xsum.
I use all hyper parameters as in the above notebook. After train, I got following loss values.

/usr/local/lib/python3.7/dist-packages/transformers/optimization.py:562: UserWarning: This overload of add_ is deprecated: add_(Number alpha, Tensor other) Consider using one of the following signatures instead: add_(Tensor other, *, Number alpha) (Triggered internally at /pytorch/torch/csrc/utils/python_arg_parser.cpp:1005.) exp_avg_sq_row.mul_(beta2t).add_(1.0 - beta2t, update.mean(dim=-1))
[2000/2000 46:21, Epoch 2000/2000]




Step
Training Loss
Validation Loss




100
11.515400
11.971568


200
9.583800
10.400596


300
7.782900
8.812467


400
1.653800
1.603077


500
0.006000
0.367762


600
0.049500
0.271221


700
0.000600
0.271198


800
0.000300
0.282740


900
0.000200
0.311012


1000
0.000200
0.308879


1100
0.000100
0.316374


1200
0.000100
0.329436


1300
0.000100
0.326996


1400
0.020200
0.291157


1500
0.002400
0.272543


1600
0.000100
0.291934


1700
0.035900
0.308235


1800
0.000100
0.308755


1900
0.000100
0.311096


2000
0.000000
0.311382




CPU times: user 1h 1min 52s, sys: 18min 49s, total: 1h 20min 41s Wall time: 46min 26s
TrainOutput(global_step=2000, training_loss=1.8480298345236805, metrics={‘train_runtime’: 2783.2468, ‘train_samples_per_second’: 0.719, ‘total_flos’: 0, ‘epoch’: 2000.0, ‘init_mem_cpu_alloc_delta’: 8192, ‘init_mem_gpu_alloc_delta’: 0, ‘init_mem_cpu_peaked_delta’: 0, ‘init_mem_gpu_peaked_delta’: 0, ‘train_mem_cpu_alloc_delta’: -4224876544, ‘train_mem_gpu_alloc_delta’: 2288025088, ‘train_mem_cpu_peaked_delta’: 4242300928, ‘train_mem_gpu_peaked_delta’: 8677123584})

My question is what values for hyperparameters (optimization, learning rate etc.) I should use for a QA task? And the training loss shown in TrainOutput does not match with the loss values on table. What can the reason be?