Perfect, I’m able to produce that error with Adagrad (see below). Thank you for your help! As a last question, how come Adagrad doesn’t initialize its buffer in step instead of __init__ like in Adam? Wouldn’t this remove the problem altogether? (assuming step isn’t called before moving the model to GPU)
TypeError: addcmul_ received an invalid combination of arguments - got (int, torch.cuda.FloatTensor, torch.cuda.FloatTensor), but expected one of:
 * (torch.FloatTensor tensor1, torch.FloatTensor tensor2)
 * (torch.SparseFloatTensor tensor1, torch.SparseFloatTensor tensor2)
 * (float value, torch.FloatTensor tensor1, torch.FloatTensor tensor2)
      didn't match because some of the arguments have invalid types: (int, torch.cuda.FloatTensor, torch.cuda.FloatTensor)
 * (float value, torch.SparseFloatTensor tensor1, torch.SparseFloatTensor tensor2)
      didn't match because some of the arguments have invalid types: (int, torch.cuda.FloatTensor, torch.cuda.FloatTensor)