I have tried to change nn.ReLU(True) or nn.LeakyReLU(True) from ‘Ture’ to ‘False’, but it didn’t work.
I am using Pytorch Version 1.4.0
Code is somewhat similar to this :
model_1 = my_model()
model_2 = my_model()
model_3 = my_model()

criterion = Loss()
optimizer = torch.optim.Adam(list(model_1.parameters())+list(model_2.parameters())+list(model_3.parameters()), lr=learning_rate)

for epoch in range(num_epochs):

	out_1 = model_1(a)
	out_2 = model_2(b)
	out_3 = model_3(out_1+out_2)

	loss = criterion(out_3,truth)
	loss.backward()
	optimizer.step()