Assuming you are running this on jupyter kernel, try to restart the system or kernel and see if the error persists. Generally this error comes up when you have previously moved some data to GPU and that data remains on GPU as you have to do manual garbage collection when using pytorch for this issue.
You can refer to this fastai tutorial for some references link 43