It depends (if there is no buffer in your loss)
Usually, it works fine.
Here is an example:
import torch as t

# Crossentropy with weight
criterion = t.nn.CrossEntropyLoss(weight=t.Tensor([1, 3]))
input = t.autograd.Variable(t.randn(4, 2)).cuda()
target = t.autograd.Variable(t.Tensor([1, 0, 0, 1])).long().cuda()

# Error, since weight is still in cpu
# loss = criterion(input, target)

# it's ok
criterion.cuda()
loss = criterion(input, target)

print(criterion._buffers)


See celll around In[45] for more detail (it’s tutorials written in Chinese )

  
      github.com
  
  
    chenyuntc/pytorch-book/blob/master/chapter5-常用工具/chapter5.ipynb 21
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第五章  PyTorch常用工具模块"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在训练神经网络过程中，需要用到很多工具，其中最重要的三部分是：数据、可视化和GPU加速。本章主要介绍Pytorch在这几方面的工具模块，合理使用这些工具能够极大地提高编码效率。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [


  This file has been truncated. show original