Solved by tom in post #2 
              
              
                You almost certainly want permute here and not view. 
A loss of inf means your input sequence is too short to be aligned to your target sequence (ie the data has likelihood 0 given the model - CTC loss is a negative log likelihood after all). 
Best regards 
Thomas