Hi guys, I am new to deep learning models and pytorch. I have been working on a multiclass text classification with three output categories. I used LSTM model for 30 epochs, and batch size is 32, but the accuracy for the training data is fluctuating and the accuracy for validation data does not change. Here are my codes.
class AdvancedModel(nn.Module):
    def __init__(self, vocab_size, embedding_dim, hidden_dim, output_dim, n_layers, 
                 bidirectional, dropout, pad_idx):
        
        super().__init__()
        
        self.embedding = nn.Embedding(vocab_size, embedding_dim, padding_idx = pad_idx)
        
        # Define some parameters
        self.output_dim = output_dim
        self.n_layers = n_layers
        self.hidden_dim = hidden_dim
        
        # Define layers
        self.lstm = nn.LSTM(embedding_dim, hidden_dim, n_layers, bidirectional=bidirectional, batch_first=True)
        self.dropout = nn.Dropout(dropout)
        self.fc = nn.Linear(hidden_dim, output_dim)
    
        
    def forward(self, text, text_lengths):
        
        #text = [sent len, batch size]
        
        embedded = self.dropout(self.embedding(text))
        
        #embedded = [sent len, batch size, emb dim]
        
        #pack sequence. Clambing the lengths to avoid errors in some bugs
        packed_embedded = nn.utils.rnn.pack_padded_sequence(embedded, text_lengths.clamp(min=1, max=50))
        
        # Initializing hidden state for first input using method defined below
        hidden = self.init_hidden(BATCH_SIZE)
        lstm_out, hidden = self.lstm(packed_embedded, hidden)
        
        unpacked, unpacked_len = torch.nn.utils.rnn.pad_packed_sequence(lstm_out, batch_first=True)
        lstm_out=unpacked[:, -1, :]
        
        out = self.dropout(lstm_out)
        out = self.fc(out)
        
        return out
    
    def init_hidden(self, batch_size):
        weight = next(self.parameters()).data
        hidden = (weight.new(self.n_layers, batch_size, self.hidden_dim).zero_().to(device),
                      weight.new(self.n_layers, batch_size, self.hidden_dim).zero_().to(device))
        return hidden

Here is the parameters I use:
INPUT_DIM = len(TEXT.vocab)
EMBEDDING_DIM = 100
HIDDEN_DIM = 300
OUTPUT_DIM = len(LABEL.vocab)
N_LAYERS = 1
BIDIRECTIONAL = False
DROPOUT = 0.5

The rest of the codes are based on this:

  
      github.com
  
  
    bentrevett/pytorch-sentiment-analysis/blob/master/5 - Multi-class Sentiment Analysis.ipynb 9
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5 - Multi-class Sentiment Analysis\n",
    "\n",
    "In all of the previous notebooks we have performed sentiment analysis on a dataset with only two classes, positive or negative. When we have only two classes our output can be a single scalar, bound between 0 and 1, that indicates what class an example belongs to. When we have more than 2 examples, our output must be a $C$ dimensional vector, where $C$ is the number of classes.\n",
    "\n",
    "In this notebook, we'll be performing classification on a dataset with 6 classes. Note that this dataset isn't actually a sentiment analysis dataset, it's a dataset of questions and the task is to classify what category the question belongs to. However, everything covered in this notebook applies to any dataset with examples that contain an input sequence belonging to one of $C$ classes.\n",
    "\n",
    "Below, we setup the fields, and load the dataset. \n",
    "\n",
    "The first difference is that we do not need to set the `dtype` in the `LABEL` field. When doing a mutli-class problem, PyTorch expects the labels to be numericalized `LongTensor`s. \n",
    "\n",
    "The second different is that we use `TREC` instead of `IMDB` to load the `TREC` dataset. The `fine_grained` argument allows us to use the fine-grained labels (of which there are 50 classes) or not (in which case they'll be 6 classes). You can change this how you please."
   ]
  },
  {


  This file has been truncated. show original

  
  
    
    
  
  


Here is the training accuracy I get:
Capture547×761 28.4 KB
The validation accuracy never changes for the whole 30 epochs.
It would be really helpful if anyone knows what is wrong here. Thank you!