Hi everyone,
I am new to pytorch and after some tutorials, I tried to mimic a source github about Flor HTR architecture (you can search this git in the internet). This source used tensorflow and I want to test the flor architecture in pytorch. However, when I train the model, my CTCloss always goes from around 10 then down to around 3 and 2. I has tried some methods like:

Change zero infinity to True in the CTCloss (this was when I got negative values but now I only see positive value after some changes in the model so I think this is not the case).
Change the implement of Batchrenorm I found in the internet which are included  batchrenorm by ludvb, and batchrenorm by mf1024 but the situation is still the same.
I am very appreciate if somebody helps me in this case. To produce the situation, this is my colab link (https://drive.google.com/file/d/1mgJ-asR6iosANUI0goL697kLMYn8LJeX/view?usp=sharing) and my file data format hdf5 transformed from IAM dataset (https://drive.google.com/file/d/1I3cEdQmFF7loqj0r0XPjpO5RPgBamR5m/view?usp=sharing)

Thank you,
If you have any question, I am happy to answer.