I’m not sure, what went wrong, but try to check the device of all parameters after calling model.to('cuda'):
for name, param in model.named_parameters():
    if param.device.type != 'cuda':
        print('param {}, not on GPU'.format(name))

This should print, if a parameter is not on the GPU.