Hi.
I’m trying to make adversarial examples with iterative method but found out that resultant adversarial examples are very different from each other although they were all made under the same condition. Does anybody know what is the problem? Is it precision issue?
The following codes are just templates to show the mentioned problem, not the exact code
import torch
import torch.nn.functional as F
from torch.autograd import Variable
import torchvision.models as models

mean = torch.cuda.FloatTensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1)
std = torch.cuda.FloatTensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1)
def normalizer(tensor):
    return (tensor-mean)/std

res101 = models.resnet101(pretrained=True)
res101.eval()
res101.cuda();

base = torch.zeros(1,3,224,224).cuda()
input1 = base.clone()
input2 = base.clone()
K = 200

for k in range(K):
    input1 = Variable(input1.data, requires_grad=True)
    normed_input1 = normalizer(input1)
    loss1 = res101(normed_input1).sum()
    
    res101.zero_grad()
    if input1.grad is not None:
        input1.grad.fill_(0)
        
    loss1.backward()
    input1 = (input1 - input1.grad)
    

    input2 = Variable(input2.data, requires_grad=True)
    normed_input2 = normalizer(input2)
    loss2 = res101(normed_input2).sum()
    
    res101.zero_grad()
    if input2.grad is not None:
        input2.grad.fill_(0)
        
    loss2.backward()
    input2 = (input2 - input2.grad)
    
    if k% 20 == 0:
        print('step :', k+1, 'close :', torch.allclose(input1, input2), 'cosine similarity :', F.cosine_similarity(input1.view(-1), input2.view(-1), dim=0).item())

and here are the results
step : 1 close : True cosine similarity : 1.0
step : 21 close : False cosine similarity : 0.9865298271179199
step : 41 close : False cosine similarity : 0.9560290575027466
step : 61 close : False cosine similarity : 0.8762256503105164
step : 81 close : False cosine similarity : 0.7901834845542908
step : 101 close : False cosine similarity : 0.711552619934082
step : 121 close : False cosine similarity : 0.6323087811470032
step : 141 close : False cosine similarity : 0.5666524171829224
step : 161 close : False cosine similarity : 0.4950137734413147
step : 181 close : False cosine similarity : 0.44248107075691223