Yea, I thought it’s a good idea to buffer those tensors as otherwise, they won’t be transferred to the GPU - this would mean GPU to CPU transfer of data during the update of the hidden variables which seems inefficient.