Hey !
I’m using CTCLoss to implement a speech recognition recurrent neural network.
       def forward(self, x):

        hidden = None
        rnn_output, hidden = self.recurrent(x, hidden)

        rnn_output_flat = rnn_output.view(-1, self.hidden_size)

        lin_output = self.output(rnn_output_flat)

        output_flat = self.softmax(lin_output)

        output = output_flat.view(rnn_output.size(0), rnn_output.size(1), output_flat.size(1))

        return output

This is my forward function, where self.softmax = nn.LogSoftmax, self.output = nn.Linear(…), and self.recurrent = nn.RNN(…)
Now, as input I have an audio files which I have pre-processed using MFCC feature extraction, and encoded using a simple alphabet encoding. I pad my inputs per batch in my DataLoader.
Training seems to work: the loss starts at about 30 for my first input, and then gradually goes down after every batch. But after 7 or 8 batches, I start getting losses, in the [-1, 0] range. At that point, obviously, training doesn’t actually seem to improve the model at all anymore.
I was wondering if I’m missing something obvious here. I’ve been scratching my head for a while now…
Thanks a lot for your help !