img is no more a leaf variable since you unsqueeze it. Gradients are kept only for leaf variables. You can correct the code like so:
image = torch.Tensor(1,28,28)
image.requires_grad_(True)
image_unsqueezed = image.unsqueeze(0)
pred = model(image_unsqueezed)
target = 3
target_tensor = torch.LongTensor([target])
loss = criterion(pred, target_tensor)
loss.backward()

or using variables explicitly:
image = torch.Tensor(1,28,28)
image_var = Variable(image.unsqueeze(0), requires_grad=True)
pred = model(image_var)
target = 3
target_tensor = Variable(torch.LongTensor([target]))
loss = criterion(pred, target_tensor)
loss.backward()
print(image_var.grad)