Of course, my hardware setup is a 6 core CPU(8400), and a 1060 GPU with 3 gb of vrams, so a tad bit limited in compute power and vram.
Any ideas on how to solve this issue will still running inplace=True? Switching inplace to False required me to scale down the model layers, since inplace reduced the vram usage by a decent amount for me, so that was nice to have. I could get rid of the dropout errors but that is a bit of a non ideal solution.