Solved by tom in post #2 
              
              
                You want encoded_label = torch.tensor([1, 2, 1, 3]), otherwise the CTC loss will tip over. The CTCLoss implementation does follow the alignment with the enlarged sequence, but does this “padding” on the fly. 
Best regards 
Thomas