I guess a view or reshape operation might be wrong and in particular this line of code might be interesting to check:
        # Reshaping the outputs such that it can be fit into the fully connected layer
        out = out.contiguous().view(-1, self.hidden_dim)

Here you are pushing all “additional” dimension/shapes to dim0, which is the batch dimension and should stay constant.
Use: out = out.contiguous().view(out.size(0), -1) instead as it would keep the batch size equal and would then flatten the additional values into the feature dimension.
If a shape mismatch is raised in the linear layer afterwards, adapt the in_features to the expected value.
Also, remove the nn.Softmax layers, since nn.CrossEntropyLoss expects raw logits and will apply F.log_softmax internally.