The Matrix Calculus You Need for Deep Learning
So I have a question somewhat related to this that I never knew where&#x2F;who to ask (well actually I asked a few mathematicians at a university I work with whose answers I couldn&#x27;t understand - their answers were almost as impenetrable as the Wikipedia page, and some engineering scientists who I thought would be more into &#x27;applied math&#x27; but they didn&#x27;t know. So I&#x27;m hoping some data science people reading this would better understand where I&#x27;m coming from, and be able to explain at my primitive math skills level).<p>So, some time ago I contracted out writing some code for fitting a logistic regression onto a given set of observations. There were some specific requirements, but I think I should have been able to piece something myself together using mainstream LA libraries; some of them even hint at &#x27;you could fit a logistic regression using these functions&#x27; but no complete examples. But I didn&#x27;t understand well enough so I contracted it out.<p>The woman who ended up writing the code used a &#x27;Hessian&#x27; matrix to do so (she actually wrote two functions doing the same thing, one used this Hessian approach - I think the idea was that it would be faster but there wasn&#x27;t a lot of time and it never got tweaked enough to make a difference).<p>So my question - is there a layman&#x27;s explanation for what a Hessian matrix is, and how it applies to fitting a logistic model? Also (with an eye to the future of my project), does it have applications for non-linear regressions?<p>Alternatively, are there any books where this is covered? I have most standard stats&#x2F;applied stats&#x2F;operations research books, as well as a few like the no bullshit guide to linear algebra, but none cover this specific issue - or even how to fit a logistic regression at all, on a practical level (so not just &#x27;conceptually you do xyz, implementation is left as an exercise for the reader&#x27;).