Howdy Hoody!



 Hoodythree:


I want to know if this term can help to update the parameters of my network in the training process cause I did an assignment for model weights here weights = self.model.module.model.fc[-1].weight.



If you’re asking whether the assignment:
weights = self.model.module.model.fc[-1].weight

will prevent gradients from flowing back through the assignment
and break backpropagation, the answer is no.
In python, “variables” are references.
self.model.module.model.fc[-1].weight refers to a tensor
in memory somewhere.  The above assignment creates a new
reference that refers to the same tensor in memory.  No new
tensor is created, nor is any data copied from one place to
another.  Performing tensor operations on weights is essentially
identical to performing tensor operations on
self.model.module.model.fc[-1].weight, so backpropagation
will work identically.
To emphasize this point:
wtmp1 = self.model.module.model.fc[-1].weight
wtmp2 = wtmp1
wtmp3 = wtmp2
weights = wtmp3

would be almost equivalent to your assignment, with the only
difference being the three extra (and unnecessary) temporary
references that will be cleaned up when your script exits (or
when an enclosing code block goes out of scope).



If I want to get a weighed similar loss, is the below code based on yours correct?

...
    sim_mat = sim_mat * w_s
    sim_mat = sim_mat - torch.diag(torch.diag(sim_mat))
    max_val = torch.max(sim_mat, dim=1).values



This will do what I believe you want.  Each element of sim_mat
will be multiplied by the corresponding element of w_s.  This
will potentially change the result of torch.max(), depending on
the specific values involved.
Note, if you set up w_s so that it has zeros along its diagonal, the
multiplication will zero out the diagonal of sim_mat so you can forgo
sim_mat = sim_mat - torch.diag(torch.diag(sim_mat)).
Best.
K. Frank