Body: <p>I came across the concept of <strong>&quot;deep learning primitives&quot;</strong> from the Nvidia talk <a href="https://info.nvidia.com/rs/156-OFN-742/images/Jetson_AGX_Xavier_New_Era_Autonomous_Machines.pdf" rel="nofollow noreferrer">Jetson AGX Xavier New Era Autonomous Machines</a> (on slide 44).</p>
  <p>There doesn't seem to be a lot of articles in the community on this concept. I was able to find one definition from <a href="https://www.oreilly.com/library/view/tensorflow-for-deep/9781491980446/ch01.html" rel="nofollow noreferrer">here</a>, where it defined deep learning primitives as the &quot;fundamental building blocks of deep networks&quot; like fully connected layers, convolutions layers, etc.</p>
  <p>I was curious to find out if a <strong>self-attention layer</strong> is a primitive, I came across this <a href="https://github.com/oneapi-src/oneDNN/issues/533" rel="nofollow noreferrer">OpenDNN issue</a> and one person explained that self-attention layers can be built by other primitives like inner product, concat, etc.</p>
  <p>So my question is what exactly are <strong>primitives</strong> in deep learning? What makes a convolution layer a primitive and a self-attention layer not a primitive?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-05T15:59:43.020Z
Comments.Text: Slide 44 only says "DEEP VISION PRIMITIVES" and then they provide examples of computer vision tasks, so I am not sure where, in that slide, you saw the term "deep learning primitives".
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-05T17:21:54.130Z
Comments.Text: I thought the concept of primitives would generalise to beyond just vision tasks? The second link gives a definition of deep learning primitives: https://www.oreilly.com/library/view/tensorflow-for-deep/9781491980446/ch01.html
Comments.UserDisplayName: Frank
Tags: <deep-learning><architecture>
Title: What exactly are deep learning primitives?
UserDisplayName: Frank
ThreadText: # What exactly are deep learning primitives?
  
  (Asked by: username_0 on 2020-11-05)
  
  I came across the concept of **"deep learning primitives"** from the Nvidia talk [Jetson AGX Xavier New Era Autonomous Machines](https://info.nvidia.com/rs/156-OFN-742/images/Jetson_AGX_Xavier_New_Era_Autonomous_Machines.pdf) (on slide 44).
  
  There doesn't seem to be a lot of articles in the community on this concept. I was able to find one definition from [here](https://www.oreilly.com/library/view/tensorflow-for-deep/9781491980446/ch01.html), where it defined deep learning primitives as the "fundamental building blocks of deep networks" like fully connected layers, convolutions layers, etc.
  
  I was curious to find out if a **self-attention layer** is a primitive, I came across this [OpenDNN issue](https://github.com/oneapi-src/oneDNN/issues/533) and one person explained that self-attention layers can be built by other primitives like inner product, concat, etc.
  
  So my question is what exactly are **primitives** in deep learning? What makes a convolution layer a primitive and a self-attention layer not a primitive?
  
  **Tags:** deep-learning, architecture
  
  > **Comments:**
  > **username_1**: Slide 44 only says "DEEP VISION PRIMITIVES" and then they provide examples of computer vision tasks, so I am not sure where, in that slide, you saw the term "deep learning primitives".  
  > **username_0**: I thought the concept of primitives would generalise to beyond just vision tasks? The second link gives a definition of deep learning primitives: https://www.oreilly.com/library/view/tensorflow-for-deep/9781491980446/ch01.html  
  
  ---
  
  ## 0 answers
  
  