Is the attack and defense methods constitute ‘adversarial training’?

Nope. Those were just very simple attack and defense (that’s not really a defense) examples to help people get started in that challenge using PyTorch.
Adversarial training is the training of a model to be more robust to attack by including adversarial examples in the training process, along side the unperturbed examples.
This is a while ago now, but I did release repo after the challenge that included an adversarial training setup I was working on at the end of the challenge. Results were promising but I had some issues with GCP killing my long running GPU instances, so lost the weights.
The idea was:

create a list of attacks with associated config (ie some basic iterative, optimizer based PGD, etc)
create a list of ensembles of models to run the attacks with
create a defense model (the model being trained) that can be single or ensembled (at the feature or logits level)
training involves an ‘adversarial generator’ that’s running on a subset of GPUs that cycles through the combos of attacks and attack models (moving them on/off GPU as needed) to create adversarial examples that are fed into the batches along with normal examples used in the training loop (which runs the remaining GPUs)
on occasion the model being trained will be used for generating adversarial examples (dogfooding)

The code needs updating to more recent PyTorch API, cleanup, etc. Two main files related to the adversarial training idea were:

  
      github.com
  
  
    rwightman/pytorch-nips2017-adversarial/blob/master/python/train_adversarial_defense.py 1
import os
import argparse
import math
import time
import torch
import torch.utils.data
import numpy as np
from copy import deepcopy
from torchvision import transforms
from torchvision import datasets
from torchvision import utils

from models import create_ensemble, create_model_from_cfg, Ensemble
from models.model_configs import config_from_string
from adversarial_generator import AdversarialGenerator
from mp_feeder import MpFeeder
from defenses import multi_task

parser = argparse.ArgumentParser(description='PyTorch ImageNet Training')
parser.add_argument('data', metavar='DIR',


  This file has been truncated. show original

  
  
    
    
  
  



  
      github.com
  
  
    rwightman/pytorch-nips2017-adversarial/blob/master/python/adversarial_generator.py 2
import torch
import torch.utils.data
import numpy as np
from copy import deepcopy

from models import create_ensemble, create_model
from models.model_configs import config_from_string
from attacks.iterative import AttackIterative
from attacks.cw_inspired import CWInspired
from attacks.selective_universal import SelectiveUniversal
import processing


def attack_factory(model, cfg):
    cfg = deepcopy(cfg)
    attack_name = cfg.pop('attack_name')
    print('Creating attack (%s), with args: ' % attack_name, cfg)
    if attack_name == 'iterative':
        attack = AttackIterative(model, **cfg)
    elif attack_name == 'cw_inspired':


  This file has been truncated. show original