Thank you for your answer @SimonW.
It means that my training problem does not come from the way I use CTCLoss then as I think I used it right in the first place according to your description. The inputs of the log_softmax came from my last nn.Linear layer.
In the model I have (only kept the relevant parts):
...
self.dense = to_best_device(nn.Linear(in_features=50, out_features=len(characters)))
...
def forward(self, x):
   ...
   return self.dense(x)

And in the training loop I have:
    for i, batch_data in enumerate(dataloader):
        data, labels = batch_data
        optimizer.zero_grad()
        outputs = model(data)
        outputs = outputs.permute(1, 0, 2) # Because I have (batch_size, sequence_length, nb_features)
        bs = len(data)
        curr_loss = loss(outputs.log_softmax(2), labels, bs * [outputs.shape[0]], [get_sentence_length(label) for label in labels])
        curr_loss.backward()
        optimizer.step()