Hi,
You seem to create a lot of intermediary results but it is indeed weird that it uses 12GB.
I can’t see anything obviously wrong with the code.
I would try and remove some of the computations and see how it changes things. You should also print the size of few Tensors to make sure they are what you expect.