Hi, I don’t understand why this piece of code works not properly. I enabled the “required_grad” for the input dataset, then normalize it, and when I backward through the model (M_O) I expect that “data_batch_.grad.data” has some values other than zero but it always zero. Something is getting wrong in the normalize function which I don’t know what it is. I’m saying that because when I bypass the normalize function the gradients are not zero ( works correctly ).

def normalize(t, mean, std, dataset):

    if dataset == 'mnist':
        t[:, 0, :] = (t[:, 0, :] - mean[0]) / std[0]
    if dataset == 'cifar10':
        t[:, 0, :, :] = (t[:, 0, :, :].detach() - mean[0]) / std[0]
        t[:, 1, :, :] = (t[:, 1, :, :].detach() - mean[1]) / std[1]
        t[:, 2, :, :] = (t[:, 2, :, :].detach() - mean[2]) / std[2]

    return t

....
    criterion = nn.CrossEntropyLoss()
    criterion = criterion.to(device)
...
     for i, (data_batch, labels_batch) in enumerate(adv_iterator):
                data_batch = data_batch.to(device)
                labels_batch = labels_batch.to(device)

                data_batch_ = normalize(data_batch.clone().detach(), mean, std, chose_dataset)
                **data_batch_ = torch.tensor(data_batch_, requires_grad=True, device=device)**
               # **data_batch_ = data_batch**

                out_adv, _, _ = M_O(data_batch_)
                loss = criterion(out_adv, labels_batch)

                M_O.zero_grad()
                loss.backward()

                noise = torch.sign(data_batch_.grad.data)

                print(noise[0])
...