Confession as an AI researcher; seeking advice
Being a grad student, this person is at a perfect place to build up their math background. Any school almost certainly offers the following:<p>1. Convex Optimization -- not all problems are convex, but solutions for nonconvex problems end up primarily using convex methods with slight adaptations.<p>2. Stochastic Optimization -- ML is pretty much all stochastic optimization. No surprise there.<p>3. Statistical&#x2F;Theoretical Machine Learning -- courses built around concentration bounds, PAC learnability, and the Valiant&#x2F;Vapnik school of thought. This gives you what you need to talk about generalizability and sample complexity.<p>4. Numerical Linear Algebra -- being smart about linear algebra is most of efficient machine learning. Knowing which kinds of factorizations help you solve problems efficiently. Can you do a Gram-Schmidt factorization? Cholesky decomposition? LU factorization? When do these things fail? When do you benefit from sparse representations?<p>5. Graphical Models -- Markov chains, Markov fields, causal relationships, HMMs, factor graphs, forward-backward algorithm, sum-product algorithms.<p>If you&#x27;re in school, <i>take advantage of the fact that you&#x27;re in school</i>.<p>Once you have a grasp on these things (and you&#x27;ll have to catch up on real analysis, matrix calculus, and a few other fields of math), you&#x27;ll be able to start reasoning about ways to improve existing methods or come up with your own. I think a lot of it is just developing mathematical maturity to give you a vocabulary to think about things with.