Hi, Thank you for the reply.
Here is my code:
optimizer = torch.optim.SGD(model.parameters(), args.lr,
                                    momentum=args.momentum,
                                    weight_decay=args.weight_decay)
scheduler = ReduceLROnPlateau(optimizer, 'min')

for epoch in xrange(args.start_epoch, args.epochs):
       train(train_loader, model, criterion, optimizer, epoch)
       result_avg, loss_val = validate(val_loader, model, criterion, epoch)
       scheduler.step(loss_val)

and here is how I got the validation loss:
out_cls = model(input_var)

loss_cls = [None] * 40
for i in range(len(loss_cls)):
    loss_cls[i] = criterion(out_cls[i], target_cls[i])

loss_val = 0
for i in range(len(loss_cls)):
    loss_val += loss_cls[i]

What the difference between training loss and validation loss? I mean they have the same data type right?