Yeah I am aware of that, the problem is that fully connected layer + mask are both big matrices and I ran into memory issues on my GPU. I was rather thinking whether there is faster indexing procedure than the one I’m using.
Is a masking/zeroing approach implementation available somewhere so I could refer to it and look for potential mistakes in my code?