Hi Jpj!



 jpj:

#custom initiliazation of weight
self.hidden_layer_1.weight = torch.nn.Parameter(torch.from_numpy(weights))



At this point, you (presumably) have already created your optimizer, so
your optimizer contains a collection of Parameters that it will be updating.
Because you have overwritten weight with a new Parameter (rather
than having overwritten weight.data with a new tensor) – and done so
after creating your optimizer – your optimizer does not contain the new
Parameter that is used (via the forward pass) to calculate the loss.
(The Parameter that is modified by the optimizer-- or would be if it had
a non-trivial gradient – is the old Parameter that you have, in a sense, “hidden.”)
So self.hidden_layer_1.weight doesn’t change, because it is not in
the optimizer’s list of Parameters to update.
[/quote]


I don’t understand how weights are the same after 10 epochs even though the loss is changing


Why the loss is changing even though self.hidden_layer_1.weight
is not:  I will assume that self.hidden_layer_1 is a Linear.  A Linear
has both a weight and a bias.  You haven’t overwritten bias, so it is
still updated by your optimizer, and still affects the loss you calculate.
Best.
K. Frank