Stanford A.I. Courses
Are there project-based tutorial that talks more about neural net architecture, hyperparameters selection and debugging? Something that walks through getting poor results and make explicit the reasoning for tweaking?<p>When I try to use transformers or any AI thing on a toy problem I come up with, it never works. Even Fizz-Buzz which I thought was easy doesn&#x27;t work (because division or modulo is apparently hard to represent for NNs). And there&#x27;s this blackbox of training that&#x27;s hard to debug into. Yes, for the available resources, if you pick the exact same problem, the exact same NN architecture and exact same hyperparameters, it all works out. But surely they didn&#x27;t get that on the first try. So what&#x27;s the tweaking process?<p>Somehow this point isn&#x27;t often talked about in courses and consequently the ones who&#x27;ve passed this hurdle don&#x27;t get their experience transferred. I&#x27;d follow an entire course on this if it were available. An HN commenter linked me to this<p><a href="https:&#x2F;&#x2F;karpathy.github.io&#x2F;2019&#x2F;04&#x2F;25&#x2F;recipe&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;karpathy.github.io&#x2F;2019&#x2F;04&#x2F;25&#x2F;recipe&#x2F;</a><p>which is exactly on point. But it&#x27;d be great if it were one or more tutorials with a specific example, wrapped in code and peppered with many failures.