Hi,
I read multiple papers about implementing bitflips during training to make your neural network more resistant against random bit flips.
i found two tools:
  
      
      github.com
  
  
    

pytorchfi/pytorchfi 2


  A runtime fault injection tool for PyTorch :fire:. Contribute to pytorchfi/pytorchfi development by creating an account on GitHub.

  
  
    
    
  
  


and
  
      
      github.com
  
  
    

bfgoldstein/torchfi 2


  TorchFI is a fault injection framework build on top of PyTorch for research purposes.

  
  
    
    
  
  


they are mainly focussing on images.
I just have a small NN(one relu layer) with numerical inputs and outputs
I made a kind of algorithm that takes one bias/weight from the input/relu or output layer and converts that to binary. Do a bitflip and converts it back.
I was thinking of implementing this after the feedforward stage. Now everytime i get a runtime error:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [8, 4]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
This is my code:
for epoch in range(num_epochs):
    running_loss = 0
    for i, (inputs, labels) in enumerate(train_loader):
        inputs = torch.from_numpy(inputs).to(torch.float32)
        labels = torch.from_numpy(labels).to(torch.float32)

        optimizer.zero_grad()
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        
        net = bitflip(net)

        loss.backward()
        
        optimizer.step()

        running_loss += loss.item()

How can i implement the bitflip algorithm during training without getting the error?