I am using CTC  in an LSTM-OCR setup and was previously using a CPU implementation (from here 86). I am now looking to using the CTCloss function in pytorch, however I have some issues making it work properly. My test model is very simple and consists of a single BI-LSTM layer followed by a single linear layer.
 def make_model(ninput=48, noutput=97):
     return nn.Sequential(
        # run 1D LSTM
        layer.Lstm1(100),
        # reorder for Linear layer
        layer.Reorder("BDW", "BWD"),
        # run single linear layer
        layer.Linear(noutput),
        # reorder to CTC convention
        layer.Reorder("BWD", "WBD"))

My inputs are image batches in the form “BDW” (batch, depth, width). My targets are of the form “BL”
target[0] = tensor([18, 50, 43, 61, 39, 52, 42, 43, 56])

with the numbers going from 1 to C, reserving 0 for “blank”.
I then train the model like this:
def train(model, input, target, input_lengths, target_lengths):
    assert input.size(0) == target.size(0)
    logits = model.forward(input)
    probs = nn.functional.log_softmax(logits, dim=2)
    optimizer.zero_grad()
    loss = ctc(probs, target, input_lengths, target_lengths)
    loss.backward() 
    optimizer.step()
    return nn.functional.softmax(logits, dim=2)

For the optimizer I use SGD.
When training using my data set, it only predicts one letter in the beginning, but after a couple of epochs it only predicts blank for all inputs. If I only use a single sample for training and the one letter predicted in the beginning is part of the target, it will increase the probability for that output to 1 for any input, instead of predicting blank.
So far I am using a batch size of 1, because I have additional problems with how to provide the data for larger batches. If I provide the input as a “BDW” tensor, where “W” is the maximum input_length for all samples in the batch, zero-pad all other samples to the same length and provide the correct input_lengths, the model produces “NaN” after a few epochs.
I had reasonable outputs using the CTC implementation mentioned in the beginning, although it was a lot slower, so I assume I am using it somehow incorrectly.
UPDATE: I at least figured out why there didnt seem to be any training going on. I am not sure how pytorch scales the CTC loss, but the updates were just so much smaller compared to the implementation I used previously, that training stopped too early. Increasing the learning rate I noticed that training is happening.