Answers.Body: <p>The gradient is a generalization of the derivative of a 1D single-variable function to 1D multi-variable functions. It gives the direction (i.e., a vector containing some change for each variable) in which the function increases the most. This is a fundamental and essential concept to master in calculus, and for understanding gradient descent-based methods.</p>
  <p>Due to its fundamentality, there are lots of resources about it.</p>
  <ul>
  <li>For very visual results, I recommend the very popular series from 3Blue1Brown, which you can find <a href="https://www.youtube.com/watch?v=WUvTyaaNkzM&amp;list=PLZHQObOWTQDMsr9K-rj53DwVRMYO3t5Yr" rel="nofollow noreferrer">here</a>.</li>
  <li>For a more direct introduction in the deep learning context, you can read the <a href="https://d2l.ai/chapter_appendix-mathematics-for-deep-learning/single-variable-calculus.html" rel="nofollow noreferrer">single-variable</a> and <a href="https://d2l.ai/chapter_appendix-mathematics-for-deep-learning/multivariable-calculus.html" rel="nofollow noreferrer">multi-variable</a> mathematics appendices of <a href="https://d2l.ai" rel="nofollow noreferrer">Dive into Deep Learning</a>.</li>
  <li>If you desire to acquire a better and more general understanding of calculus, reading or following a more standard calculus course is definitely more adequate. Renowned books include &quot;Calculus&quot; by James Stewart, &quot;Calculus&quot; by Michael Spivak, &quot;Thomas' Calculus&quot; by George B. Thomas, for example.</li>
  </ul>
  <p>The adequate resource depends on your current background, what skills you want to acquire, and what tasks you would like to perform with AI. Developing some visual intuition with 3B1B's series might be enough for developing models with libraries like PyTorch which include a whole optimization library which handles the gradient descent process for you.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-29T13:05:53.177Z
Answers.Text: Great answer, exactly what I was looking for. I think I will start with the 3blue1brown link, then probably will go through Caclulus by Spivak, as I already bought that book and planned to study it. If you have other suggestions about Deep Learning books apart from that Dive into Deep Learning one, and focused on this topic, please let me know . Many thanks!
Answers.UserDisplayName: zerunio
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-29T13:24:05.663Z
Answers.Text: My pleasure. There are other great resources. If you're into books, a popular resource is the "Deep Learning" book by Ian Goodfellow. For a more practical approach, there is "Deep Learning with Python" by Francois Chollet. For a more course-style reference, I can recommend the course I personally followed, deep learning by Gilles Louppe (on GitHub). Other excellent courses are available online, such as EE559 Deep Learning (François Fleuret, EPFL), and CS230 Deep Learning (Andrew Ng, Stanford).
Answers.UserDisplayName: Sacha Lewin
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-29T15:25:55.743Z
Answers.Text: thank you again. I am more into paper books rather thank online only onces. I already heard about the 1st one
Answers.UserDisplayName: zerunio
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-29T13:00:50.770Z
Answers.LastActivityDate: 2023-09-29T13:00:50.770Z
Answers.UserDisplayName: Sacha Lewin
Body: <p>I have read some resources about AI, and they all speak about the gradient.</p>
  <p>Is there any book focused on this? maybe with tons of images / diagrams?</p>
  <p>Cheers</p>
  
Tags: <deep-learning><deep-rl><gradient-descent><books><gradient>
Title: Gradient: any resource on how to understand everything about it?
UserDisplayName: zerunio
ThreadText: # Gradient: any resource on how to understand everything about it?
  
  (Asked by: username_0 on 2023-09-29)
  
  I have read some resources about AI, and they all speak about the gradient.
  
  Is there any book focused on this? maybe with tons of images / diagrams?
  
  Cheers
  
  **Tags:** deep-learning, deep-rl, gradient-descent, books, gradient
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-09-29. Score: 1 | Accepted answer
  
  The gradient is a generalization of the derivative of a 1D single-variable function to 1D multi-variable functions. It gives the direction (i.e., a vector containing some change for each variable) in which the function increases the most. This is a fundamental and essential concept to master in calculus, and for understanding gradient descent-based methods.
  
  Due to its fundamentality, there are lots of resources about it.
  
  * For very visual results, I recommend the very popular series from 3Blue1Brown, which you can find [here](https://www.youtube.com/watch?v=WUvTyaaNkzM&list=PLZHQObOWTQDMsr9K-rj53DwVRMYO3t5Yr).
  * For a more direct introduction in the deep learning context, you can read the [single-variable](https://d2l.ai/chapter_appendix-mathematics-for-deep-learning/single-variable-calculus.html) and [multi-variable](https://d2l.ai/chapter_appendix-mathematics-for-deep-learning/multivariable-calculus.html) mathematics appendices of [Dive into Deep Learning](https://d2l.ai).
  * If you desire to acquire a better and more general understanding of calculus, reading or following a more standard calculus course is definitely more adequate. Renowned books include "Calculus" by James Stewart, "Calculus" by Michael Spivak, "Thomas' Calculus" by George B. Thomas, for example.
  
  The adequate resource depends on your current background, what skills you want to acquire, and what tasks you would like to perform with AI. Developing some visual intuition with 3B1B's series might be enough for developing models with libraries like PyTorch which include a whole optimization library which handles the gradient descent process for you.
  
  > **Comments:**
  > **username_0**: Great answer, exactly what I was looking for. I think I will start with the 3blue1brown link, then probably will go through Caclulus by Spivak, as I already bought that book and planned to study it. If you have other suggestions about Deep Learning books apart from that Dive into Deep Learning one, and focused on this topic, please let me know . Many thanks!  
  > **username_1**: My pleasure. There are other great resources. If you're into books, a popular resource is the "Deep Learning" book by Ian Goodfellow. For a more practical approach, there is "Deep Learning with Python" by Francois Chollet. For a more course-style reference, I can recommend the course I personally followed, deep learning by Gilles Louppe (on GitHub). Other excellent courses are available online, such as EE559 Deep Learning (François Fleuret, EPFL), and CS230 Deep Learning (Andrew Ng, Stanford).  
  > **username_0**: thank you again. I am more into paper books rather thank online only onces. I already heard about the 1st one  
  
  ---
  
  