Answers.Body: <p>Smoothness here is the mathematical <a href="https://en.wikipedia.org/wiki/Smoothness" rel="nofollow noreferrer">definition</a>, so as you implied smoothness is ruled out by output data with sharp spikes or discontinuous jumps (and possibly the data of the gradient, the gradient's gradient, ad infinitum, depending on who defines smoothness).</p>
  <p>By any definition a lot of activation functions are not smooth, for example RELU. This means neural networks in general are not smooth.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-19T16:12:40.593Z
Answers.LastActivityDate: 2021-01-19T16:12:40.593Z
Answers.UserDisplayName: Chris Young
Body: <p>In <a href="https://arxiv.org/pdf/1412.5068.pdf" rel="nofollow noreferrer">this research paper</a>, we have the following claim</p>
  <blockquote>
  <p>the smoothness assumption that underlies many kernel methods such as Support Vector Machines (SVMs) does not hold for deep neural networks trained through backpropagation</p>
  </blockquote>
  <p>Does <em>smoothness</em> here refer to no sharp rise/fall in gradients?</p>
  
Tags: <machine-learning><terminology><papers><support-vector-machine>
Title: What is the smoothness assumption in SVMs?
UserDisplayName: Hrushi
ThreadText: # What is the smoothness assumption in SVMs?
  
  (Asked by: username_0 on 2021-01-12)
  
  In [this research paper](https://arxiv.org/pdf/1412.5068.pdf), we have the following claim
  
  > the smoothness assumption that underlies many kernel methods such as Support Vector Machines (SVMs) does not hold for deep neural networks trained through backpropagation
  
  Does *smoothness* here refer to no sharp rise/fall in gradients?
  
  **Tags:** machine-learning, terminology, papers, support-vector-machine
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-01-19. Score: 2 | Accepted answer
  
  Smoothness here is the mathematical [definition](https://en.wikipedia.org/wiki/Smoothness), so as you implied smoothness is ruled out by output data with sharp spikes or discontinuous jumps (and possibly the data of the gradient, the gradient's gradient, ad infinitum, depending on who defines smoothness).
  
  By any definition a lot of activation functions are not smooth, for example RELU. This means neural networks in general are not smooth.
  
  
  ---
  
  