Also, another question: certain operations only change the view of a tensor rather than the underlying storage. What happens when we do tensor operations on those tensors? Are we doing operations on the view, allocating new memory of the same shape as the underlying storage, or allocating new memory of the shape of the view?
i.e. I have a tensor a. I do b = a.repeat(20). b is still using the same underlying storage. Now I do d = torch.cat(b, c). What memory does d occupy?