I’m not sure how I’d implement it, that’s why I asked the question. On a GPU I imagine a block would run the necessary SVD or inversion operations for each of the set of problems. Is there something inherent to these matrix operations that prohibits this type of implementation?
For a reason why you don’t want to just iterate over the batch, perhaps you have an image with a matrix representing each pixel? Then you have BxHxWxKxK, for example. You then might want to invert theKxK elements for some reason. Or run SVD over them. In this case iterating over each would be absurdly slow, but the operations at each pixel are completely independent from one another which would lend itself to parallelizaton.