Does PyTorch automatically delete GPU variables once they go out of scope?

Yes ! You would get a lot of memory leaks otherwise 

nn.Parameter

Note that nn.Parameter are only important if your work inside a nn.Module and you want them to be found by mod.parameters(). Otherwise, there is no need to have them.
Here is an updated example with optimizer:
full_params = torch.rand(num_classes, feature_size, requires_grad=True, device="cpu")
opt = optim.SGD([full_params,], lr=1)

def acc_grads(full_params, slice):
  param_subset = full_params[slice]
  param_subset = param_subset.cuda()
  # Compute that part of the loss
  local_loss = your_fn(param_subset)
  # Accumulate into full_params.grad
  local_loss.backward()
  # Use .item() to drop gradient history and get a python number
  return local_loss.item()

loss = 0.
# reset the gradients
opt.zero_grad()
for slice in slices:
  # Nothing outside `acc_grad` is on GPU. So the all the GPU memory will be freed
  # after it returns (use torch.cuda.memory_allocated() to see the used memory)
  partial_loss = acc_grad(full_params, slice)
  loss += partial_loss
opt.step()
print("Did a step for a total loss of {}".format(loss))

Note that with this, the gradients will only live on the CPU and the optimizer will work with CPU Tensors.
Also because we send the Tensor to cuda after slicing it, during the backward, the full gradient Tensor will only exist on the CPU.
Does that match what you want better?