Hi George!



 mpalaourg:

The two results I get from those implementations is slightly different (max absolute error 2e-5). Is that because of numeric reasons?


I believe that your discrepancy is due to (accumulated) round-off
error.
An easy way to test this is to repeat your computation using double
precision (by using torch.double tensors).  If the discrepancy is due
to round-off error it should drop dramatically, say to about 1e-13.  If the
discrepancy were to remain around 2e-5, then it would be due to the
two computations not being mathematically equivalent (or to some bug
somewhere).


OR Is that because on the second approach I impact every item on the [T, T] matrix, even if outside the blocks the dependency factor I add is zero?


Based on your description, I believe that your two computations should
be mathematically equivalent (but without full details, I can’t really say
for sure).
As an aside, do you ever work with the off-block-diagonal parts of your
tensor X?  If not, you might consider creating, storing, and processing
only the [B, B]-shaped blocks themselves.  Given that B << T, doing
so could result in significant time and memory savings.
Best.
K. Frank