What was the sentiment classification problem like? Was it a binary classification problem (Positive vs Negative)? In that case, sigmoid activation would work. But this is a multi-class classification. So, my guess is that you shouldn’t be using sigmoid here. Instead, replace it with a Linear layer (size = num_classes, input_dim).