Thank you so much @ptrblck ! I modified linear_output to linear_output = self.linear(sequence_output[:, -1]) and the training started, hopefully I won’t encounter other issue during training. I do have 2 more questions if you could also help with this. As I am new to the entire pytorch and bert, lstm, is the forward function computed fine for a binary classification? The training works, but I’m wondering if it is a correct way to do it.
The second one would be regarding CUDA. At the moment the training runs on CPU but it takes hours to train. I have the latest CUDA version and the problem I have is that it says it ran out of memory right before training. I already tried to lower the batch size as much as I could ( to 16,4,1) but this didn’t resolved the problem. I don’t know if it’s a relevant thing but when I run the code on GPU, I checked the GPU usage on task manager and from 0% it goes to 5%,9% and when it throws the error it jumps to 100% and the running code stops and throws the error right before training. Any idea what else i can do to resolve this problem? Thank you!