sequence_output = outputs[0] was initially # Extract the last hidden state of the token[CLS] for classification task         sequence_output = outputs[0][:, 0, :] but that turned the tensor into a 2 dimensional and i needed a 3 dimensional for lstm. I tried to use that and reshape the tensor so it will work with lstm but that brought  a lot of issues after.
Yes, i am using nn.CrossEntropyLoss.
By using torch.utils.checkpoint do i have to make modifications just in the forward function or in the entire code as well? Do you know any tutorial on using this?