ptrblck Hello, I would be grateful if someone can help me here. I have a model whose high level architecture is
CNN -> Transformer Encoder -> CTC.
When the batch size 2 the CTC loss works (using batch size 2 because of GPU limitations). Here is an example pipeline.
CNN output:
size [2, 168, 1024] # 2 videos, 168 frames/video, 1024 spatial embedding output
ctc_loss:
def ctc_loss(log_probs, targets, input_lengths, target_lengths, blank=0,
             reduction='mean', zero_infinity=False):
    # type: (Tensor, Tensor, Tensor, Tensor, int, str, bool) -> Tensor

    return torch.ctc_loss(log_probs, targets, input_lengths, target_lengths, blank, _Reduction.get_enum(reduction),
                          zero_infinity)


log_probs: size [226, 2, 70]
targets: size [2, 18]
input_lengths: size [2], value: tensor([78,78])
target_lengths: size [2]

During training in one of the epochs when the batch size is 1 the runtime error pops up.
CNN output:
size [1, 42, 1024] # 1 video, 42 frames, 1024 spatial embedding output
ctc_loss:
log_probs: size [42, 1, 70]
targets: size [1]
input_lengths: size [1], value: tensor([77])
target_lengths: size [1]

RuntimeError: Expected input_lengths to have value at most 42, but got value 77 (while checking arguments for ctc_loss_gpu)
When the batch size is 2, the input_lengths tensor has value [78, 78] and the ctc loss works but it doesn’t work when batch size is 1. Any idea what’s going on?
Thanks.