If you begin with two separate tensors they will live in different places in memory so it is not possible to combine them into a single tensor without copying them both into new tensor.
One trick to do this could be to first begin with the big tensor, and then take small views of that tensor.
import torch
import torch.optim as optim

# some dummy tensors that have the same shape as `a` used to initialize `x`
_a = torch.tensor(1.)
_b = torch.tensor(2.)
x = torch.vstack((_a, _b))

# now we finally create `a`
a = x.select(0, 0)
a.requires_grad_(True) # a.is_leaf is True
opt = optim.Adam([a]) # optimize `a`

loss = a ** 2
loss.backward()

print(a)
print(x)

opt.step()

print(a)
print(x) # should be updated as well

The trick is basically to find the inverse operation of whatever you’re doing. So for arbitrary fn (where x = fn(a)), make sure you find fn2 such that `a = fn2(fn(a))