I trained a model on GPU with DataParallel.
I saved the best model to disk.
full_state = {'epoch': epoch, 'arch': args.arch, 'model_state_dict': model.state_dict(),
                    'best_val_acc1': best_model_val_acc1, 'optimizer_state_dict': optimizer.state_dict()}
torch.save(full_state, best_model_file_path)

After all epochs, I loaded the best model saved like above using the code below:
    if os.path.isfile(model_file):
        print("\n=> loading checkpoint '{}'".format(model_file))
        checkpoint = torch.load(model_file)
        model.load_state_dict(checkpoint['model_state_dict'])
        print("=> loaded checkpoint '{}' (epoch {})".format(model_file, checkpoint['epoch']))
    else:
        print("=> no checkpoint found at '{}'".format(model_file))
        return

I am passing mode=“cpu” to the code below.
Nevertheless, I am receiving the following error:
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
How can this be possible if I am using .cpu() to send my inputs to CPU?
I do not even understand why my input tensor is being loaded directly to GPU in the first place.
def compute_total_inference_time(model, val_loader, mode):

    total_inference_time = 0

    if mode == "cpu":
        model.cpu()
    elif mode == "gpu":
        model.cuda()

    # switch to evaluate mode
    model.eval()

    with torch.no_grad():

        for input_tensor, _ in tqdm(val_loader):

            if mode == "cpu":
                input_tensor = input_tensor.cpu()
            elif mode == "gpu":
                input_tensor = input_tensor.cuda()

            # compute output
            initial_time = time.time()
            _ = model(input_tensor)
            final_time = time.time()

            instance_inference_time = final_time - initial_time
            total_inference_time += instance_inference_time

    return total_inference_time