Answers.Body: <p>Linear Algebra Done Right by Axler seems to be the best book on linear algebra, with a brisk and modern approach.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-28T16:30:47.773Z
Answers.LastActivityDate: 2019-10-28T16:30:47.773Z
Answers.UserDisplayName: k.c. sayz 'k.c sayz'
Answers.Body: <p>If you already have two years of a bachelor's of mathematics, I recommend part I of the book that you're mentioning. That part of the book reviews the main mathematics used in the optimization of neural nets (in part 1), and then actually goes through the various models in detail in the later parts. The review is done at a level that is suitable for someone who has already studied these topics, but needs a refresher.</p>
  <p>The book <a href="https://www.janmagnus.nl/misc/mdc-ch18.pdf" rel="nofollow noreferrer">Matrix Differential Calculus with Applications in Statistics and Econometrics</a> covers more advanced topics, which might also be what you are looking for. There is also <a href="https://en.wikipedia.org/wiki/Matrix_calculus" rel="nofollow noreferrer">the related Wikipedia article</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-25T17:59:35.297Z
Answers.Text: There's also this book [Mathematics for Machine Learning](https://mml-book.github.io/), which contains a chapter (chapter 5: Vector Calculus) that seems to cover (at least partially) what the OP was originally looking for. The [The Matrix Cookbook](http://www2.imm.dtu.dk/pubdb/edoc/imm3274.pdf).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-28T17:29:22.603Z
Answers.LastActivityDate: 2021-01-22T01:02:50.043Z
Answers.LastEditDate: 2021-01-22T01:02:50.043Z
Answers.UserDisplayName: John Doucette
Body: <p>I am reading <a href="http://www.deeplearningbook.org/" rel="nofollow noreferrer">Goodfellow's</a> book about neural networks, but I am stuck in the mathematical calculus of the back-propagation algorithm. I understood the principle, and some Youtube videos explaining this algorithm shown step-by-step, but now I would like to understand the <strong>matrix calculus</strong> (so not basic calculus!), that is, calculus with matrices and vectors, but especially everything related to the derivatives with respect to a matrix or a vector, and so on.</p>
  <p>Which math book could you advise me to read?</p>
  <p>I specify I studied 2 years after the bachelor in math school (in French: mathématiques supérieures et spéciales), but did not practice for years.</p>
  
Tags: <neural-networks><reference-request><linear-algebra><books><calculus>
Title: Which linear algebra book should I read to understand vectorized operations?
UserDisplayName: lolveley
ThreadText: # Which linear algebra book should I read to understand vectorized operations?
  
  (Asked by: username_0 on 2019-10-28)
  
  I am reading [Goodfellow's](http://www.deeplearningbook.org/) book about neural networks, but I am stuck in the mathematical calculus of the back-propagation algorithm. I understood the principle, and some Youtube videos explaining this algorithm shown step-by-step, but now I would like to understand the **matrix calculus** (so not basic calculus!), that is, calculus with matrices and vectors, but especially everything related to the derivatives with respect to a matrix or a vector, and so on.
  
  Which math book could you advise me to read?
  
  I specify I studied 2 years after the bachelor in math school (in French: mathématiques supérieures et spéciales), but did not practice for years.
  
  **Tags:** neural-networks, reference-request, linear-algebra, books, calculus
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2019-10-28. Score: 3 | Accepted answer
  
  If you already have two years of a bachelor's of mathematics, I recommend part I of the book that you're mentioning. That part of the book reviews the main mathematics used in the optimization of neural nets (in part 1), and then actually goes through the various models in detail in the later parts. The review is done at a level that is suitable for someone who has already studied these topics, but needs a refresher.
  
  The book [Matrix Differential Calculus with Applications in Statistics and Econometrics](https://www.janmagnus.nl/misc/mdc-ch18.pdf) covers more advanced topics, which might also be what you are looking for. There is also [the related Wikipedia article](https://en.wikipedia.org/wiki/Matrix_calculus).
  
  > **Comments:**
  > **username_3**: There's also this book [Mathematics for Machine Learning](https://mml-book.github.io/), which contains a chapter (chapter 5: Vector Calculus) that seems to cover (at least partially) what the OP was originally looking for. The [The Matrix Cookbook](http://www2.imm.dtu.dk/pubdb/edoc/imm3274.pdf).  
  
  ---
  
  ### Answer by username_1 on 2019-10-28. Score: 1
  
  Linear Algebra Done Right by Axler seems to be the best book on linear algebra, with a brisk and modern approach.
  
  
  ---
  
  