Hi all,
Is there a quick way to access (then plot) the l2 norm of the distance between the initial set of weights w_0 and a set of weights at iteration t, w_t ?
I’d like to access this quantity to then plot it in 2D (along a random dimension).
I believe net.parameters() where net is defined as below
class MnistNetSmall(nn.Module):
    def __init__(self):
        super(MnistNetSmall, self).__init__()
        self.conv1 = nn.Conv2d(1, 20, 5, 1)
        self.fc1 = nn.Linear(20 * 12 * 12, 10)
        #self.fc1 = nn.Linear(28 * 28, 10)


    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = F.max_pool2d(x, 2, 2)
        x = x.view(-1, 20 * 12 * 12)
        #x = x.view(-1, 28 * 28)
        x = self.fc1(x)
        return F.log_softmax(x, dim=1)

net = MnistNetSmall()


is what I need. Though this quantity is a generator object.
Any ideas how to collect net.parameters() at each iteration for plotting purposes?
Thanks all