summaries: This paper tackles the problem of designing neural network architectures that are robust to adversarial attacks. Several defense techniques against adversarial attacks have been proposed, mainly adversarial training (train on perturbed inputs) and introducing random perturbation to the weights or activations of the network. The paper claims that one limitation of the second approach is that it introduces artifacts (e.g. sparsity). The authors propose a simple but original idea to address this issue: parameterize the network's weight matrices as low rank tensors (in the Tucker format) and randomize the weights by sketching the core tensor of the Tucker decomposition (in effect, the sketching amounts to randomly setting fibers of the core tensor to 0).  In this paper, the authors propose a randomization-based tensorization framework towards robust network learning. The high-level idea of this work is to reparameterize the network parameters W of each layer with low-rank tensors, where the factor matrices are injected with randomization through randomly sampled sketching matrices. Since the randomization is is done within a subspace than directly on the weight matrix itself, the authors claim that this brings certain advantages such as less sparsity.
abstractText: As deep neural networks become widely adopted for solving most problems in computer vision and audio-understanding, there are rising concerns about their potential vulnerability. In particular, they are very sensitive to adversarial attacks, which manipulate the input to alter models’ predictions. Despite large bodies of work to address this issue, the problem remains open. In this paper, we propose defensive tensorization, a novel adversarial defence technique that leverages a latent high order factorization of the network. Randomization is applied in the latent subspace, therefore resulting in dense reconstructed weights, without the sparsity or perturbations typically induced by the randomization. Our approach can be easily integrated with any arbitrary neural architecture and combined with techniques like adversarial training. We empirically demonstrate the effectiveness of our approach on standard image classification benchmarks. We further validate the generalizability of our approach across domains and low-precision architectures by considering an audio classification task and binary networks. In all cases, we demonstrate superior performance compared to prior works in the target scenario.
references.authors: Naveed Akhtar Ajmal Mian
references.title: Threat of adversarial attacks on deep learning in computer vision: A survey
references.venue: IEEE Access,
references.authors: Dario Amodei Sundaram Ananthanarayanan Rishita Anubhai Jingliang Bai Eric Battenberg Carl Case Jared Casper Bryan Catanzaro Qiang Cheng Guoliang Chen
references.title: Deep speech 2: End-toend speech recognition in english and mandarin
references.venue: In International conference on machine learning,
references.authors: Dario Amodei Chris Olah Jacob Steinhardt Paul Christiano John Schulman Dan Mané
references.title: Concrete problems in ai safety
references.venue: arXiv preprint arXiv:1606.06565,
references.authors: Marcella Astrid Seung-Ik Lee
references.title: Cp-decomposition with tensor power method for convolutional neural networks
references.venue: compression. CoRR,
references.authors: Anish Athalye Nicholas Carlini David Wagner
references.title: Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples
references.venue: arXiv preprint arXiv:1802.00420,
references.authors: Yusuf Aytar Carl Vondrick Antonio Torralba
references.title: Soundnet: Learning sound representations from unlabeled video
references.venue: In Advances in Neural Information Processing Systems,
references.authors: Casey Battaglino Grey Ballard Tamara G Kolda
references.title: A practical randomized cp tensor decomposition
references.venue: SIAM Journal on Matrix Analysis and Applications,
references.authors: Adrian Bulat Georgios Tzimiropoulos
references.title: Binarized convolutional landmark localizers for human pose estimation and face alignment with limited resources
references.authors: Nicholas Carlini David Wagner
references.title: Towards evaluating the robustness of neural networks
references.venue: IEEE Symposium on Security and Privacy (SP),
references.authors: Matthieu Courbariaux Yoshua Bengio Jean-Pierre David
references.title: Binaryconnect: Training deep neural networks with binary weights during propagations
references.venue: In NIPS,
references.authors: Matthieu Courbariaux Itay Hubara Daniel Soudry Ran El-Yaniv Yoshua Bengio
references.title: Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1
references.authors: Guneet S Dhillon Kamyar Azizzadenesheli Zachary C Lipton Jeremy Bernstein Jean Kossaifi Aran Khanna Anima Anandkumar
references.title: Stochastic activation pruning for robust adversarial defense
references.venue: arXiv preprint arXiv:1803.01442,
references.authors: Yinpeng Dong Fangzhou Liao Tianyu Pang Hang Su Jun Zhu Xiaolin Hu Jianguo Li
references.title: Boosting adversarial attacks with momentum
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
references.authors: Simon S Du Jason D Lee
references.title: On the power of over-parametrization in neural networks with quadratic activation
references.venue: In ICML,
references.authors: N Benjamin Erichson Krithika Manohar Steven L Brunton J Nathan Kutz
references.title: Randomized cp tensor decomposition
references.venue: arXiv preprint arXiv:1703.09074,
references.authors: Angus Galloway Graham W Taylor Medhat Moussa
references.title: Attacking binarized neural networks
references.venue: arXiv preprint arXiv:1711.00449,
references.authors: Ian J Goodfellow Jonathon Shlens Christian Szegedy
references.title: Explaining and harnessing adversarial examples
references.venue: arXiv preprint arXiv:1412.6572,
references.authors: A. Graves A. Mohamed G. Hinton
references.title: Speech recognition with deep recurrent neural networks
references.venue: IEEE International Conference on Acoustics, Speech and Signal Processing,
references.authors: Alex Graves Navdeep Jaitly
references.title: Towards end-to-end speech recognition with recurrent neural networks
references.venue: Proceedings of the 31st International Conference on Machine Learning,
references.authors: Chuan Guo Mayank Rana Moustapha Cisse Laurens van der Maaten
references.title: Countering adversarial images using input transformations
references.venue: arXiv preprint arXiv:1711.00117,
references.authors: Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun
references.title: Deep residual learning for image recognition
references.venue: In Proceedings of the IEEE conference on computer vision and pattern recognition,
references.authors: Sandy Huang Nicolas Papernot Ian Goodfellow Yan Duan Pieter Abbeel
references.title: Adversarial attacks on neural network policies
references.venue: arXiv preprint arXiv:1702.02284,
references.authors: Elias B Khalil Amrita Gupta Bistra Dilkina
references.title: Combinatorial attacks on binarized neural networks
references.venue: arXiv preprint arXiv:1810.03538,
references.authors: Diederik P Kingma Jimmy Ba
references.title: Adam: A method for stochastic optimization
references.venue: arXiv preprint arXiv:1412.6980,
references.authors: Tamara G. Kolda Brett W. Bader
references.title: Tensor decompositions and applications
references.venue: SIAM REVIEW,
references.authors: Jean Kossaifi Adrian Bulat Georgios Tzimiropoulos Maja Pantic
references.title: T-net: Parametrizing fully convolutional nets with a single high-order tensor
references.venue: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
references.authors: Jean Kossaifi Yannis Panagakis Anima Anandkumar Maja Pantic
references.title: Tensorly: Tensor learning in python
references.venue: The Journal of Machine Learning Research,
references.authors: Alex Krizhevsky Geoffrey Hinton
references.title: Learning multiple layers of features from tiny images
references.venue: Technical report, Citeseer,
references.authors: Alexey Kurakin Ian Goodfellow Samy Bengio
references.title: Adversarial examples in the physical world
references.venue: arXiv preprint arXiv:1607.02533,
references.authors: Vadim Lebedev Yaroslav Ganin Maksim Rakhuba Ivan V. Oseledets Victor S. Lempitsky
references.title: Speeding-up convolutional neural networks using fine-tuned cp-decomposition
references.venue: In ICLR,
references.authors: Ji Lin Chuang Gan Song Han
references.title: Defensive quantization: When efficiency meets robustness
references.venue: arXiv preprint arXiv:1904.08444,
references.authors: Xuanqing Liu Yao Li Chongruo Wu Cho-Jui Hsieh
references.title: Adv-bnn: Improved adversarial defense through robust bayesian neural network
references.venue: arXiv preprint arXiv:1810.01279,
references.authors: Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras Adrian Vladu
references.title: Towards deep learning models resistant to adversarial attacks
references.venue: arXiv preprint arXiv:1706.06083,
references.authors: Alexander Novikov Dmitry Podoprikhin Anton Osokin Dmitry Vetrov
references.title: Tensorizing neural networks
references.venue: In Proceedings of the 28th International Conference on Neural Information Processing Systems,
references.authors: Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga Adam Lerer
references.title: Automatic differentiation in PyTorch
references.venue: In NIPS Autodiff Workshop,
references.authors: Mohammad Rastegari Vicente Ordonez Joseph Redmon Ali Farhadi
references.title: Xnor-net: Imagenet classification using binary convolutional neural networks
references.venue: In ECCV,
references.authors: Jonas Rauber Wieland Brendel Matthias Bethge. Foolbox v
references.title: 8.0: A python toolbox to benchmark the robustness of machine learning models
references.venue: arXiv preprint arXiv:1707.04131,
references.authors: Pouya Samangouei Maya Kabkab Rama Chellappa
references.title: Defense-gan: Protecting classifiers against adversarial attacks using generative models
references.venue: arXiv preprint arXiv:1805.06605,
references.authors: Nicholas D Sidiropoulos Evangelos E Papalexakis Christos Faloutsos
references.title: Parallel randomly compressed cubes: A scalable distributed architecture for big tensor decomposition
references.venue: IEEE Signal Processing Magazine,
references.authors: Mahdi Soltanolkotabi Adel Javanmard Jason D Lee
references.title: Theoretical insights into the optimization landscape of over-parameterized shallow neural networks
references.venue: IEEE Transactions on Information
references.authors: Yang Song Taesup Kim Sebastian Nowozin Stefano Ermon Nate Kushman
references.title: Pixeldefend: Leveraging generative models to understand and defend against adversarial examples
references.venue: arXiv preprint arXiv:1710.10766,
references.authors: Florian Tramèr Alexey Kurakin Nicolas Papernot Ian Goodfellow Dan Boneh Patrick McDaniel
references.title: Ensemble adversarial training: Attacks and defenses
references.venue: arXiv preprint arXiv:1705.07204,
references.authors: Charalampos E Tsourakakis
references.title: Mach: Fast randomized tensor decompositions
references.venue: In Proceedings of the 2010 SIAM International Conference on Data Mining,
references.authors: Nico Vervliet Otto Debals Laurent Sorber Lieven De Lathauwer
references.title: Breaking the curse of dimensionality using decompositions of incomplete tensors: Tensor-based scientific computing in big data analysis
references.venue: IEEE Signal Processing Magazine,
references.authors: Siyue Wang Xiao Wang Pu Zhao Wujie Wen David Kaeli Peter Chin Xue Lin
references.title: Defensive dropout for hardening deep neural networks under adversarial attacks
references.venue: In Proceedings of the International Conference on Computer-Aided Design,
references.authors: Yining Wang Hsiao-Yu Tung Alexander J Smola Anima Anandkumar
references.title: Fast and guaranteed tensor decomposition via sketching
references.venue: In Advances in Neural Information Processing Systems,
references.authors: Pete Warden
references.title: Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition
references.venue: arXiv e-prints, art
references.authors: Cihang Xie Jianyu Wang Zhishuai Zhang Zhou Ren Alan Yuille
references.title: Mitigating adversarial effects through randomization
references.venue: In International Conference on Learning Representations,
references.authors: Weilin Xu David Evans Yanjun Qi
references.title: Feature squeezing: Detecting adversarial examples in deep neural networks
references.venue: arXiv preprint arXiv:1704.01155,
references.authors: Longhao Yuan Chao Li Jianting Cao Qibin Zhao
references.title: Randomized tensor ring decomposition and its application to large-scale data reconstruction
references.authors: Guoxu Zhou Andrzej Cichocki Shengli Xie
references.title: Decomposition of big tensors with low multilinear rank
references.venue: arXiv preprint arXiv:1412.1885,
sections.heading: 1 INTRODUCTION
sections.text: Deep neural networks (DNNs) are powerful predictive models that achieve impressive accuracy across a wide range of artificial intelligence tasks, including image classification (He et al., 2016; Krizhevsky & Hinton, 2009) and speech recognition (Amodei et al., 2016a; Graves et al., 2013; Graves & Jaitly, 2014). The popularity of DNNs in production-ready systems has raised a serious security concern as DNNs are found to be susceptible to a wide range of adversarial attacks (Madry et al., 2017; Akhtar & Mian, 2018; Dong et al., 2018; Huang et al., 2017; Goodfellow et al., 2014; Kurakin et al., 2016), where small and imperceptible perturbations of the input data lead to incorrect predictions by the networks. These shortcomings pose an obstacle in wide-scale adoption of DNNs and expose an inherent weakness in their reliability. This is especially important when such models become part of security and safety related solutions (Amodei et al., 2016b).
  This susceptibility of DNNs to adversarial perturbations has led to a large volume of work that attempts to design robust networks (Dhillon et al., 2018; Lin et al., 2019; Samangouei et al., 2018; Song et al., 2017; Guo et al., 2017). However, advances in designing robust DNNs have been followed with stronger perturbation schemes that defeat such defences (Athalye et al., 2018).
  Most defenses that rely on randomization, either apply randomized transformations to the input, e.g. (Xie et al., 2018), or randomization applied within the network, e.g., on the activations (Dhillon et al., 2018) or on the weights directly (Wang et al., 2018). However, all these approaches typically introduce artificats (e.g. sparsity in the weights or activations) and can be defeated by carefully crafted attacks (Athalye et al., 2018). In addition, it is preferable to not to rely on modifications of the inputs but have a network that is inherently robust against attacks. In this paper, we take a different approach to randomization. We first parametrize the network using tensor factorization, effectively introducing a latent subspace spanning the weights. We then apply randomization in that latent subspace, enabling us to create models that are robust to adversarial attacks, without modifying directly the weights, activations or inputs. In summary, we make the following contributions:
  • We propose a novel adversarial defence technique that relies on a latent randomized tensor parametrization of each layer in the network and can be seamlessly integrated within any network architecture.
  • We thoroughly evaluate the robustness of our method against various adversarial attacks and show that it consistently and significantly improves over the current state-of-the-art especially when combined with adversarial training.
  • We show that our method successfully hardens the models against these attacks for both quantized and real-valued nets.
  • We verify that our strategy works across domains, by experimenting on both image-based and audio-based classification.
sections.heading: 2 RELATED WORK
sections.text: In this Section, we review the related work on adversarial attacks (Section 2.1) and defences (Section 2.2), followed by an overview of tensor methods for deep learning (Section 2.3).
sections.heading: 2.1 ADVERSARIAL ATTACKS
sections.text: First, we review a few of the most popular adversarial attacks alongside the current defense strategies employed. Given a data sample, e.g., an image X , an adversary will try to find a small perturbation, often imperceptible to a human eye, but that, added to the input sample, will cause it to be misclassified by the target model, with high confidence. Mathematically, the attacked generates a perturbation ∆ bounded in terms of some `p norm, i.e. ‖∆‖p ≤ , typically with p = 2 or p = ∞. The adversarial sample is obtained by adding the perturbation to the input sample Xadv = X + ∆. Several ways of obtaining this adversarial perturbations have been proposed (Goodfellow et al., 2014; Athalye et al., 2018; Carlini & Wagner, 2017). Among them, black box attacks, consider the network as a black-box in which the attacker has no information regarding its architecture or the gradients. White-box attacks on the contrary, assume complete access to the network architecture and all its parameters. Moreover, attacks can either be untargeted, in which case the goal is simply to make the network predict any wrong label, or targeted, in which case the aim is to force the network to predict a specific label, independently from the input sample. Next, we introduce the main white-box attacks used in this paper.
  Fast Gradient Sign Method (FGSM) is a single-step, gradient based technique, introduced by Goodfellow et al. (2014) to generate `∞-bounded adversarial perturbations as follows:
  Xadv = X + · sgn(∇XL(θ,X , y)) (1)
  where θ is the parameters of the target neural network. While the single gradient-step nature of FGSM makes it better for transferability attacks, this can also lead to a suboptimal ascent direction.
  Basic Iterative Method (BIM) and Projected Gradient Descend (PGD) aims to address the shortcoming of FGSM by running it for several iterations. Kurakin et al. (2016) propose BIM, in which the FGSM is run for for several iteration, clipping the values of the perturbation at each step to be inside the bounds. Madry et al. (2017) further improve upon this by prepending BIM with a random start and replacing clipping with a projection onto the acceptable set:
  X t+1adv = ΠX+S(X t adv + α · sgn ( ∇XL(θ,Xtadv, y) ) , (2)
  where α is the step size and ΠX+S is a projection operation forcing the generated adversarial samples to be in the `p ball S around X. A model resilient to PGD attacks is considered to be reasonably resistant to all first order attacks (Madry et al., 2017).
sections.heading: 2.2 ADVERSARIAL DEFENCES
sections.text: Despite recent advances, developing robust neural networks remains an open, challenging problem (Athalye et al., 2018). Current defense strategies typically attempt to either detect the adversarial samples and denoise them, or inject adversarial samples during training. The latter is known as adversarial training (Goodfellow et al., 2014; Kurakin et al., 2016; Madry et al., 2017) and is considered the most resilient defense technique. However, the above mentioned defences typically do not increase the robustness to black box attacks. In addition they can typically be defeated using
  two-step approaches (Tramèr et al., 2017). Feature squeezing is another model hardening approach introduced by Xu et al. (2017). This technique proposes to reduce the complexity of the data representation which in turn causes the adversarial perturbations to disappear due to lower sensitivity. Guo et al. (2017) proposes a set of five transformation, namely image cropping and re-scaling, bitdepth reduction, JPEG compression, pixel dropping and image quilting, that applied to the image to increase the robustness to adversarial attacks of a given model. However, even the combination of all these transformations was shown to be vulnerable to carefully tuned attacks (Athalye et al., 2018). Samangouei et al. (2018) introduces the so-called Defense-GAN technique. The main idea is to project the samples into the manifold of a generator before classifying them. Similarly, Song et al. (2017) uses a PixelCNN instead of a generative model. Despite the variety of recently proposed defence strategies, in (Athalye et al., 2018) the authors show that most of the existing defense techniques rely on one form of gradient obfuscation (gradients shattering, stochastic gradients and vanishing/exploding gradients), proposing both a method to detect such class of defences and to defeat them.
  As opposed to all the aforementioned works which either manipulate the data samples (Samangouei et al., 2018; Song et al., 2017; Guo et al., 2017) or introduce stochasticity on the activations (Dhillon et al., 2018) or weights Wang et al. (2018) of each layer, we propose a novel defense strategy that leverages tensor factorization of the weights in order to apply randomization in that latent space, before reconstructing the weights the weights. The approach is introduced in details in section 3.
sections.heading: 2.3 TENSOR METHODS IN DEEP LEARNING AND RANDOMIZED TENSOR DECOMPOSITION
sections.text: Tensors are high dimensional generalizations of matrices (Kolda & Bader, 2009). Recently, tensor decompositions have found a surge of applications in deep learning, mainly focusing on networks compression and acceleration. By parametrizing layers of neural networks using tensor decomposition, or even whole networks (Kossaifi et al., 2019a), the number of parameters can be reduced with little to no loss of performance, and in some cases the operations can be done more efficiently (Lebedev et al., 2015; Novikov et al., 2015; Kim et al., 2016; Astrid & Lee, 2017).
  In some cases, tensor decompositions can exhibit high computational cost and possibly low convergence rates when applied to massive data. To accelerate computation, and enable them to scale, several randomized tensor decompositions have been developed. In this way, CP decomposition can be done by selecting randomly elements from the original tensor (Battaglino et al., 2018), or using randomization to solve the problem on one or several smaller tensors before projecting back the result to the original space (Erichson et al., 2017; Sidiropoulos et al., 2014; Vervliet et al., 2014). Wang et al. (2015) proposed a fast yet provable randomized CP decomposition using FFT to perform tensor contraction. Randomization approaches have also be explored for fast approximation of other tensor decompositions, e.g., Tucker decomposition via sketching (Tsourakakis, 2010; Zhou et al., 2014) and tensor ring using tensor random projections (Yuan et al., 2019).
  These methods are orthogonal to our approach and can be combined with it. As opposed to the aforementioned works, which focus on compression or efficiency, we explore tensor factorization methods in the context of adversarial defense, proposing a novel approach in which the weight tensor of each convolutional layer are parametrized using a randomized tensor decomposition that significantly hardens the model, increasing its robustness to a wide range of adversarial attacks. Our method is generic and can be applied to both real-valued networks and binary ones. In addition, it is orthogonal to the existing defence methods and can be combined with existing defenses such as adversarial training. The method is introduced in details in section 3.
sections.heading: 3 DEFENSIVE TENSORIZATION
sections.text: In this section, we introduce our method for the defense against adversarial attack. Our defense leverages a randomized higher-order factorization method, which is used as the basis for our defense. Typically, defensive methods relying on randomization do so by introducing sparsity in either the weights or the input activation tensors of the layers of the deep neural neural network. For instance, Dhillon et al. (2018) sparsify the input tensor, by stochastically pruning some of the activations and scaling up the remaining ones. Wang et al. (2018) apply sparsification to the weights directly using dropout both during training and testing. However, all these approaches degrade per-
  formance by setting activations to zero, and, while rescaling the non-zero entries can mitigate the issue, increasing the sparsity (and therefore the efficiency of the defense) translates into large losses in performance. By contrast, we propose to rely on a latent parametrization of the layers using tensor decomposition. Intuitively, a latent subspace spanning the weights is learnt, along with a non-linear projection to and from that subspace. The sparsity inducing randomization is applied in the latent space. Upon projection, the resulting weights are dense and yet preserve the robustness against adversarial attacks. This allows us to built models that are both more robust to adversarial methods than existing works, while preserving high classification accuracy.
  Notation: Throughout the papers, we denote vectors (1storder tensors) as small bold letters v, matrices (2ndorder tensors) as bold capital letters M and tensors, which generalize the concept of matrices for orders (number of dimensions) higher than 2, in capital calligraphic letters X . The n–mode product is defined, for a given tensor X ∈ RD0×D1×···×DN and a matrix M ∈ RR×Dn , as the tensor T = X ×n M ∈ RD0×···×Dn−1×R×Dn+1×···×DN , with: Ti0,i1,··· ,in = ∑Dn k=0 Min,kXi0,i1,··· ,in .
  Latent high-order parametrization of the network: We introduce tensor factorization in the context of deep neural networks. Note that this method is independent of the dimensionality of the input but we introduce it here, without loss of generality, for the case of a 4 dimensional kernel of 2D convolutions. Specifically, we consider a deep neural network composed of L layers convolutional layers, interlaced with non-linearities Φl, l ∈ [1 . . L]. Let’s consider a convolutional layer l ∈ [1 . . L], taking as input an activation tensor Xl and parametrized by a weight tensor Wl ∈ RF,C,H,W , where F,C,H,W correspond respectively to number of Filters (e.g. output channels), input Channels, Height and Width of the convolutional kernel. The output of that layer, after applying non-linearity, will be Φ (Xl ?Wl). We introduce a latent parametrization of the weight kernelWl by expressing it as a low-rank tensor, in this paper using a Tucker decomposition Kolda & Bader (2009). In other words we expressWl in a latent subspace as a core tensor Gl. The mapping to and from this subspace is done via factor matrices UFl ,U C l ,U H l and U W l : Wl = Gl ×0 UFl ×1 UCl ×2 UHl ×3 UWl .
  Randomizing in the latent subspace: In addition to the above deterministic decomposition, we introduce, for each layer, a randomization term stochastically controlling the rank of the decomposition. To do so, we introduce diagonal sketching matrices MF ,MC ,MH and MW , the diagonal entries of which are i.i.d. and follow a Bernoulli distribution parametrized by probability θ ∈ [0, 1]. Specifically, we samples random vectors λF ∈ RO,λC ∈ RC ,λH ∈ RH and λW ∈ RW , the entries of which are i.i.d. following a Bernoulli distribution parametrized by probability θ. We can then define the sketching matrices as MO = diag(λF ),MC = diag(λC),MH = diag(λH) and MW = diag(λW ).
  This randomization is then applied not directly to the weight tensor W , but rather in the low-rank subspace, effectively randomizing the rank of the convolutional kernel:
  W̃l = (Gl ×0 MF × · · · ×3 MW )︸ ︷︷ ︸ randomized core Ĝl
  ×0 ( UFl M > F ) × · · · ×3 ( UWl M > W )︸ ︷︷ ︸ randomized factors ÛFl ,Û I l ,Û H l ,Û W l
  (3)
  This stochastic reduction of the rank can be done without affecting performance thanks to the overparametrization of deep networks, which, while crucial for learning (Du & Lee, 2018; Soltanolkotabi et al., 2018), create large amounts of redundancies. In addition, since Gl ×0 MF ×0 ( UFl M > F ) =
  Gl ×0 ( UFl M > FMF ) , and MF ,MI ,MH and MW are idempotent, eq. 3 can be simplified to:
  W̃l = (Gl ×0 MF × · · · ×3 MW )×0 UFl × · · · ×3 UWl = G̃l ×0 UFl × · · · ×3 UWl (4) In other words, we sketch the core tensor, then project it back using the original factors. An important detail is that the randomization terms from the above equation, G̃ = G ×0 MF × · · · ×N MW is never explicitly computed using actual tensor contractions. Instead, the elements sampled are directly selected from the core and the corresponding factors, which is much more computationally effective. For the binary case, we additionally apply the process from eq. (5) by plugging in eq. (4).
  The randomization being done in the latent subspace, it induces no sparsity, unlike pruning or dropout based methods and the reconstructed weights are dense. Since the weights are learnt endto-end with randomization on the latent cores, the network naturally learns not to rely on any single latent component for prediction, thus learning intrinsically more robust representations. The result is a network that is naturally more robust to perturbations in the inputs.
sections.heading: 4 EXPERIMENTAL SETTING
sections.text: In this section, we detail the experimental setting used for the experiments for image-based classification and audio-based classification.
sections.heading: 4.1 DATASETS
sections.text: We conducted experiments on two widely used databases for image-based classification and audiobased classification, respectively:
  CIFAR-10 (Krizhevsky & Hinton, 2009) is a widely used image classification dataset consisting of 60, 000 images of size 32 × 32px in 10 classes, equally represented. The dataset is divided into 50, 000 images for training and 10, 000 for testing. We did not use any particular data augmentation, beside random horizontal flipping.
  Speech Command (Warden, 2018) is an audio recognition dataset comprised of 105, 000 1-second utterances of words from a large number of users spanning over a small vocabulary. The objective is to recognize among ten spoken words: yes, no, up, down, left, right, on, off, stop, go, in addition to recognizing words outside the vocabulary as unknown, and detecting silence. The dataset is balanced and all audio recordings are captured with a sampling frequency of 16 KHz. We use a 80%-10%-10% splits for training, validation and testing respectively.
sections.heading: 4.2 IMPLEMENTATION DETAILS
sections.text: Training the model All our CIFAR-10 (Krizhevsky & Hinton, 2009) experiments were conducted using a ResNet-18 (He et al., 2016) architecture. The network was trained for 350 epochs using SGD with momentum (0.9) and a starting learning rate of 0.1 that was dropped at epoch 150 and 250 by a factor of 0.1. The weight decay was set to 1e − 6. In order to accelerate the training process the models with θ < 1 were initialized from a pretrained model that was trained without stochasticity (θ = 1). The binary counterpart models were trained following the method proposed by Rastegari et al. (2016) using the same optimizer and learning scheduler as for the real-valued ones. For the experiments conducted on the Speech Command dataset we build on the SoundNet5 (Aytar et al., 2016) architecture containing 5 convolutional layers [in channels, out channels, kernel, stride, padding]: [1, 16, (1×64), (1×2), (0×32)], [16, 32, (1×32), (1×2), (0×16)], [32, 64, (1×16), (1×2), (0× 8)], [64, 128, (1×8), (1×2), (0×4)], [128, 256, (1×4), (1×2), (0×2)] and 2 linear ones: [512, 256] and [256, 12]. Each convolutional layer was followed by a max-pooling operation. We trained all of the audio models using Adam (Kingma & Ba, 2014) for 50 epochs with an initial learning rate set to 0.01 that was dropped by 0.1× at epoch 25 and 35. The binarization and re-parametrization follows the same procedure as for CIFAR-10.
  Attacking the model For FGSM, we run the attack for various values of = {1, 2, 4, 8, 16, 32, 64, 128} (for an image range [0 . . 255]) across the entire validation/testing set averaging the results over 10 runs. On the Speech Command dataset, since the raw data is in the range [−1, 1], we scaled the value of accordingly, running it for the following values = {0.008, 0.032, 0.063}. For the iterative methods BIM and PGD, we follow Kurakin et al. (2016) and Song et al. (2017) setting the step size to 1 and the number of iterations to bmin( + 4, 1.25 )c. Threat Model: we assume that the attacker has access to everything (e.g. the architecture of the network, its
  weights, inputs, outputs, training process and gradients, etc) except the random seed used for sampling the Bernoulli random variables.
  All of our models were implemented using PyTorch (Paszke et al., 2017) and trained on a single Titan X GPU. The latent, randomized tensor factorization was implemented using TensorLy (Kossaifi et al., 2019b). For the adversarial attacks we used the FoolBox (Rauber et al., 2017) package.
sections.heading: 5 RESULTS
sections.text: In this section, we empirically demonstrate the robustness property of our proposed method against adversarial examples by extensively evaluating it on CIFAR-10 and comparing it against existing state-of-the-art defense techniques. Moreover, we show that our approach can be combined with adversarial training based techniques leading to further robustness gains. All the experiments are run using the experimental setup described in Section 4.
  Robustness to adversarial attacks: When evaluated using the FGSM attack on the CIFAR-10 dataset our method is significantly more robust than the baseline approach, especially for high values of (see Table 1 and Fig. 1). Furthermore, the results presented in Table 1 show that for lower values of θ = 0.8, our network significantly decreases the typical high attack success-rate achieved by strong iterative attack algorithms such as BIM and PGD.
  Comparison to the State-of-the-Art. To better understand the performance of our method, we compare it against existing state-of-the-art defense techniques such as: DQ (Lin et al., 2019) in which the authors attempt to reduce the propagation of the adversarial attacks inside the network by controlling the Lipschitz constant, features squeezing (Xu et al., 2017) that simple reduces the dimensionality of the search space by controlling the color bit depth of each pixel while applying spatial smoothing and finally, adversarial training in two variants: R+FGSM as proposed in (Lin et al., 2019) and PGD (Madry et al., 2017). Where the later (i.e. adversarial training) is considered to be one of the strongest defence techniques developed. For feature-squeezing we used 5 bits for image color reduction combined with a 2 × 2 median filter. For adversarial training as in (Madry et al., 2017; Kurakin et al., 2016) we sample the number of steps (for PGD) and randomly. As the results from Table 2 show, our method consistently outperforms existing defense strategies for various attacks (FGSM, BIM, PGD), across different values of = {2, 8, 16}. Furthermore, when combined with adversarial training our method can further increase its resilience to attacks.
  Defending against omniscient attackers Another interesting question is whether our defense strategy would still work against an omniscient attacker, i.e., an attacker with access to the full unrandomized weights. We trained a network for θ = 0.9. Then, during training, we first generate an adversarial example using the full, un-randomized weights (i.e. θ = 1) and test it using the stochasticity (i.e. θ = 0.9). As can be seen in Table 3, our network is still robust against these
  attacks, despite the attacker having full access to the weights. Similar behaviour can be observed for other values of θ. Note that this is an extreme scenario and in general, the weights could be stored separately (and safely) with the network getting, at each time, the randomly reconstructed weights.
  6 ABLATION STUDIES
  To further validate our findings we test out approach on two different scenarios: on fully binarized networks (Section 6.1) and audio classification task (Section 6.2).
sections.heading: 6.1 ATTACKING BINARY NEURAL NETWORKS
sections.text: With the growing popularity of deep learning based methods and the constant need of incorporating such approaches on mobile devices, network quantization has emerged as a potential hardware-friendly solution, which squeezes the original network by reducing the number of bits required to represent the model parameters. In its most extreme form, binarization, the weights and features are represented using a single bit (Courbariaux et al., 2016; Rastegari et al., 2016; Bulat & Tzimiropoulos,
  2017). The typical approach quantizes the network using the sign function (Courbariaux et al., 2015), however this introduces high quantization errors that hinder the learning process. To alleviate this, a real-valued scaling factor is introduced by Rastegari et al. (2016). In this work we binarize the network following Rastegari et al. (2016):
  I ∗W = (sgn(I)©∗ sgn(W)) Kα, (5) where I ∈ Rc×win×hin andW ∈ Rc×h×w denote the input and respectively the weight of the L-th convolutional layer, α ∈ Rc×1×1 represent the weight scaling factor and K ∈ R1×hout×wout the input scaling factor. Both α andK are computed analytically as proposed by Rastegari et al. (2016). While it was previously thought that such binarized networks are more resilient to adversarial attacks (Khalil et al., 2018; Liu et al., 2018; Galloway et al., 2017) than their real-valued counterpart, in this work we confirm the recent findings of (Lin et al., 2019) by showing that in fact it is the opposite, i.e., binary networks are more susceptible to adversarial attacks. Typically, during the training phase the derivative of the quantization function (sgn) is approximated using a STE (e.g., an identity function clipped to [−1, 1] in this case). The same estimator can be used during the attacking phase and it often leads to a high rate of success of the attacks. Interestingly, the results in Table 4 show that we can go one step further by approximating the derivative of the sgn function using tanh(x) and tanh(0.75 ∗ x) respectively. The use of these approximations make the binary networks become more sensitive to the attacks. When evaluated on a binary network on the CIFAR-10 dataset, in accordance with the behaviour found on real-valued models, our method shows significant improvements across the entire range of values and attacks tested (see Table 4 and Fig. 1). We note that for the binary case, since such networks have a lower representational capacity, we set θ = 0.99.
sections.heading: 6.2 DEFENSIVE TENSORIZATION FOR AUDIO CLASSIFICATION
sections.text: To further demonstrate the generalizability of our approach, we next consider adversarial attacks on the audio domain. In-line with the latest success in audio recognition, we consider an end-to-end audio model following SoundNet (Aytar et al., 2016) architecture, that operates directly on the raw audio signal, without requiring any feature extractions (e.g., MFCC or log mel-spectrogram). We found that the end-to-end models show higher degree of vulnerability to the adversarial attacks, e.g., around 6% absolute degradation compared to the model operating on log mel-spectrogram. In case of the small vocabulary audio recognition task, we only consider FGSM attack and summarize our findings in Table 5. With a higher degree of stochasticity (i.e. θ = 0.9), both the real and the binarized model exhibit much higher resilience to the adversarial attacks.
sections.heading: 7 CONCLUSION
sections.text: In this paper, we propose defensive tensorization, a novel adversarial defence technique that leverages a latent high order factorization of the network. Randomization is applied in the latent subspace, therefore resulting in dense reconstructed weights, without the sparsity or perturbations typically induced by the randomization. We empirically demonstrate that our approach makes the network significantly more robust to adversarial attacks. Contrarily to a widely spread belief, we observe that binary networks are more sensitive to adversarial attacks than their real-valued counter-part. We show that our method significantly improves robustness in the face of adversarial attacks for both binary and real-valued networks. We demonstrate this empirically through thorough experimentation on image and audio classification.
sections.heading: A APPENDIX
sections.text: Algorithm for defensive tensorization We summarize in Algorithm. 1 the steps for inference using our proposed defensive tensorization.
  Algorithm 1 Defensive tensorization 1: Input sample X0. 2: for each layer l do 3: Sample random vector λF , · · · ,λW ∼ Bernouilli (θ) 4: Create sketch matrices MF = diag(λF ), · · · ,MW = diag(λW ) 5: Sketch the latent Core G̃l = Gl ×0 MF × · · · ×3 MW 6: ReconstructWk using Equation (3) . Randomized dense reconstruction 7: Xl+1 ← Φl(Wl ? Xl) . Convolve Xl with the randomized reconstruction and apply Φl 8: return XL
  Low-dimensional visualization of loss landscape: To visually assess the impact of both our randomization scheme and adversarial training on the optimization landscape, we visualise the evaluation of the loss in a fixed neighbourhood around an unseen data point. Specifically, given a model and a data point x, we visualize the loss function learned by the model in the close proximity of x. For clarity, we visualise the loss in a 2–dimensional space, by selecting the two relevant axis: the direction of the gradient at x (x-axis) and a randomly chosen direction, orthogonal to the direction of gradient (y-axis). Next, a mesh-grid is constructed by sampling uniformly points along these two directions for the range [−0.5, 0.5]. Then a contour plot is constructed by evaluating the losses for all points on the mesh-grid. Examples of loss-surfaces are shown in Fig. 2, 3 and 4.
  Effect on the optimization landscape: Intuitively, the randomization (which is done in the latent subspace of the decomposition, not on the weights themselves), changes the loss function, at each pass, making it hard to converge to a fixed attack due to the presence of many spurious minimums. This can be seen by looking at the landscape of the loss function around an arbitrary sample in Fig. 2, 3. For the adversarial case, the landscape is inline with the finding of Madry et al. (2017), where the authors show the adversarial training smooths the loss space around 0. This is even more noticeable for the method that combines our approach with adversarial training (see Fig. 4).
title: DEFENSIVE TENSORIZATION: RANDOMIZED TENSOR PARAMETRIZATION