Dear all,
I am trying to do the following: Given I have some column vector of shape (5 x 1) I want to calculate the inverse of (q @ q.t()). This actually shouldn’t be such a difficult task, however, it seems that the inverse of this expression, torch.inverse(q @ q.t()) is way off:
In [358]: q = torch.randn(5,1)                                                                                                                                                                                 
In [359]: q                                                                                                                                                                                                    
Out[359]: 
tensor([[ 0.2144],
        [ 0.2899],
        [-1.0016],
        [-2.5544],
        [ 0.3549]])

In [360]: (q @ q.t()) @ torch.inverse(q @ q.t())                                                                                                                                                               
Out[360]: 
tensor([[ 0.2019, -1.0537,  0.2293, -0.3905, -0.7937],
        [-0.2614, -0.9735,  0.2418, -0.4921, -1.4343],
        [ 0.4410,  7.5348,  0.6504,  1.9400,  3.7739],
        [ 4.5718, 17.0569, -1.1591,  4.7328, 12.1593],
        [ 0.3166, -1.8443,  0.1681, -0.5732, -0.4462]])

So this calculation should obviously return something which is close to the identity. So is this some numerical issue?
If I just create a (5 x 5) matrix via w = torch.randn(5,5) and then calculate (w @ w.t()) @ torch.inverse(w @ w.t()) there is no issue and the result is close to the identity.
So is there something special by matrix multiplying a column vector with its transpose to obtain a matrix?
So maybe what I am missing is that q @ q.t() is often singular or close to singular? And hence the inverse is actually not defined?
Best regards