Hi @ptrblck - I have one more query. I want to optimize a different cost using the activation from the bottleneck layer. I will calculate the cost using log of some quantity of bottleneck layer. This is what I’m thinking…


From the forward pass capture the output of the bottleneck layer and the output layer.
output_en,output_de = self.forward(trInput)


First I’m calculating the reconstruction loss as shown below.
loss = criterion(output_de,trOutput)


Then calculate the new cost using the bottleneck output.
M := some quantity I’m calculating from output_en
loss2 = -log(M)


Now add the second cost with the loss.
loss = loss + loss2


Backward and optimize
optimizer.zero_grad()
loss.backward()
optimizer.step()


The reconstruction loss is the standard MSE so I know that Torch can calculate the derivative. But the 2nd loss is a custom one. I’m not sure whether Torch can calculate the derivative. Do I need to write a new criterion for the 2nd cost? If so then how to do that?
Thank you,
Tomojit