Thanks tom for the fast reply. However, I am using Logsoftmax at the moment so the negative values have not appeared again. You can see it in my code that I assign self.softmax = nn.LogSoftmax(dim=-1) at the code I marked with model.py. So I think I use the right way for the CTCloss but my loss value still start at 10 and go down to 3. I am happy to hear if you find out anything may cause the problem.
Thank you