@tom
I understood and reproduced your code snippet.
I have an issue though in integrating it in a custom layer.
My goal is to compute gradients for Q (3x3 matrix) !
My problem is that I cannot compute gradients (None is shown)
I suspect there is something wrong with the computational graph or an inplace operation that I am missing.
Here is a sample of my custom layer: (I tried to keep it clear and simple      )
class AdaLayer(torch.nn.Module):
def __init__(self, Li):
    super(AdaLayer, self).__init__()
    self.Q = nn.Parameter(torch.rand((3, 3), dtype=torch.float))
    self.M = nn.Parameter(torch.mm(self.Q, self.Q.t()), requires_grad=True)  #symmetric
    self.W = None
    self.mat = None
    self.linear1 = torch.nn.Linear(re_size*re_size*3, out_size)

def forward(self, img_flat, uniq_dif):
    dim = img_flat.shape[0]
    self.M = nn.Parameter(torch.mm(self.Q, self.Q.t()))

    vector = Variable( torch.diag(  torch.mm( torch.mm(uniq_dif, self.M) , uniq_dif.t())), requires_grad=True )

    # as discussed
    self.mat = torch.zeros(dim, dim)
    self.mat[np.triu_indices(dim, 1)] = vector

    # auto-gradient reaches up to here !
    self.W = nn.Parameter(torch.exp(-1*(torch.sqrt((self.mat + self.mat.t())))/2), requires_grad=True)
    y = (torch.mm(self.W, img_flat)).t().contiguous()
    print(y.shape)
    y = self.linear1(y.view(1,-1))
    return y

Output:
------------------Q 3x3  gradient----------------------
None
------------------M symmetric positive semidefinite gradient----------------------
None
------------------ mat  gradient----------------------
None
------------------W gradient----------------------
tensor([[-1.7305e-05, -1.1781e-05, -1.3871e-05,  …, -5.0380e-06,
-1.9574e-05, -6.8704e-06],
[ 1.3375e-06,  1.0680e-06,  1.2287e-06,  …,  5.4148e-07,
1.3086e-06,  6.5805e-07],
[-3.6731e-06, -2.9817e-06, -3.5238e-06,  …, -1.0123e-06,
-2.3968e-06, -1.4685e-06],
…, etc…
Any extra help would be greatly appreciated. ( I am new in pytorch   )
Kind regards ,
black0017