Shivam_Saboo:

tr_loss += _loss


I guess, I’m not an expert in pytorch, that doing the cited piece of code you are saving the loss + the hist associated. if you wanna operate with the loss as a temporal recording you have to copy the data associated by doing tr_loss += _loss.data, even more i would do tr_loss += _loss.data.cpu() not to overload the gpu
and when u evaluate you should with torch.no_grad: else u will get out of memory too
edit: and training and evaluating at the same time feels strange for me
in addition, nn.module class has a training boolean by default. self.training
if u set model.train() self.training=true and dropout/BN layers will be on
if u set model.eval() self.training will be false for the whole nn. I would recommend to use that instatment instead of the self.architecture.train()
I also saw you do the following
        _op1 = self.act(self.conv1(input))
        _op2 = self.conv2(self.resl(_op1))
        op = self.conv3(_op1 + _op2)

i would recommend to use torch.add(op1,op2) since im not sure if op1+op2 properly create the graph