I see, thank you for your answer and advice!
At this point is just out of curiosity.
Even though torch doesn’t materialize those matrices, this amount of operations needs to be done one way or another. I guess what you mean, is that this matrix doesn’t appear as a contiguous array in memory, is that right? If not, I don’t know what would be the alternative. I would love to know more details if it doesn’t waste too much of your time on this specific matter.