Hey everyone,
I have the following code set up.
sigm=torch…nn.Sigmoid()
z1 is an instance of a class that does batch normalization,relu  and maxpooling
def reparam(a,b,h):
weight_m= (2sigm(b)-(2sigm(a)sigm(b))-1+sigm(a))
weight_v=(1-sigm(a))-weight_m2
om=F.conv2d(h,weight_m,padding=1)
ov=F.conv2d(h2,weight_v,padding=1)
e=torch.randn(ov.shape).cuda()
z=om+ove
return z1(z)
loss=torch.nn.CrossEntropyLoss()
optimizer=torch.optim.Adam([a1,b1,a2,b2,…,a6,b6],learning_rate,weight_decay)
for epoch in range(num_epochs):
for images,labels in train_loader;
forward propagation
op1=reparam(a1,b1,images)
op2=reparam(a2,b2,op1)
…
…
op6=reparam(a6,b6,op5)
…# code for fully connected layers
y2=softmax(op6,dim=1)
lossi=loss(y2,labels)
optimizer.zero_grad()
lossi.backward()
optimizer.step()
Hence, the parameters to train are a1,b1,… a6,b6.
In the above piece of code,  my when I print my loss it does not decrease at all. It always stays the
same equal to 2.30
epoch 0 loss = 2.308579206466675
epoch 1 loss = 2.297269344329834
epoch 2 loss = 2.3083386421203613
epoch 3 loss = 2.3027005195617676
epoch 4 loss = 2.304455518722534
epoch 5 loss = 2.305694341659546
epoch 6 loss = 2.3002185821533203
epoch 7 loss = 2.304798126220703
epoch 8 loss = 2.3063807487487793
epoch 9 loss = 2.3052620887756348
epoch 10 loss = 2.2963013648986816
epoch 11 loss = 2.3032405376434326
epoch 12 loss = 2.293735980987549
epoch 13 loss = 2.30415940284729
epoch 14 loss = 2.3025383949279785
epoch 15 loss = 2.307767868041992
epoch 16 loss = 2.300485610961914
epoch 17 loss = 2.304170846939087
epoch 18 loss = 2.302550792694092
epoch 19 loss = 2.3051881790161133
epoch 20 loss = 2.3025758266448975
I am not sure where the problem is and i am trying hard to rectify the problem but not sure how to go about it. Any help in this would be highly appreciated,
I would like to know if there is something I am missing for the back-propagation to fail