Thanks @tom. I made a simple test to calculate some ctc losses from @SeanNaren’s warp-ctc (with the option of size_average=True and length_average=True) and nn.CTCLoss as follows:
frame_len: [473, 225]  target_len: [100, 38]
	warp_ctc: 4.356187343597412	CTCLoss(elem_mean): 23.244503021240234	CTCLoss(sum.item()): 3040.61865234375	CTCLoss(none): [2054.897705078125, 985.7210693359375]

frame_len: [793, 203]  target_len: [125, 44]
	warp_ctc: 4.358583927154541	CTCLoss(elem_mean): 23.780426025390625	CTCLoss(sum.item()): 4341.14892578125	CTCLoss(none): [3469.86328125, 871.2857666015625]

frame_len: [431, 121]  target_len: [104, 20]
	warp_ctc: 4.288755416870117	CTCLoss(elem_mean): 22.03105926513672	CTCLoss(sum.item()): 2367.3935546875	CTCLoss(none): [1839.9967041015625, 527.3968505859375]

frame_len: [391, 128]  target_len: [77, 17]
	warp_ctc: 4.3624701499938965	CTCLoss(elem_mean): 27.947921752929688	CTCLoss(sum.item()): 2264.12255859375	CTCLoss(none): [1686.162841796875, 577.959594726562

As you can see, all values are with batch_size 2. I’m sure the values are correctly calculated as intended. I guess the elementwise_mean loss led my training to somewhat different from earlier. I’m not sure what the effect is, but now, at least I know how to make the losses from warp-ctc and from the nn.CTCLoss same. I’ll report here if I found something strange further. Thank you!