Answers.Body: <p>Although I have only partially read or not read at all some of the following resources and some of these resources may not cover more advanced topics than the ones presented in the book you are reading, I think they can still be useful for your purposes, so I will share them with you.</p>
  <p>I would also like to note that if you understand the contents of the book you are currently reading, you are probably already prepared for reading some (if not most of) the research papers you wish to read. Initially, you may find them a little bit too succinct and sometimes unclear or complex, but you need to get used to this format, so there's nothing stopping you from trying to read them and learn even more by doing this exercise.</p>
  <h3>Books</h3>
  <ul>
  <li><a href="https://mitpress.mit.edu/books/introduction-computational-learning-theory" rel="nofollow noreferrer">An Introduction to Computational Learning Theory</a> (1994) by Kearns and Vazirani (no free PDF is available, afaik)</li>
  <li><a href="https://statisticalsupportandresearch.files.wordpress.com/2017/05/vladimir-vapnik-the-nature-of-statistical-learning-springer-2010.pdf" rel="nofollow noreferrer">The Nature of Statistical Learning Theory</a> (1995, 2000) by Vapnik</li>
  <li><a href="http://profsite.um.ac.ir/%7Emonsefi/machine-learning/pdf/Machine-Learning-Tom-Mitchell.pdf" rel="nofollow noreferrer">Machine Learning</a> (1997) by Mitchell</li>
  <li><a href="http://read.pudn.com/downloads161/ebook/733192/Statistical-Learning-Theory.pdf" rel="nofollow noreferrer">Statistical Learning Theory</a> (1998) by Vapnik</li>
  <li><a href="http://www.ii.uni.wroc.pl/%7Elukstafi/pmwiki/uploads/AGT/Prediction_Learning_and_Games.pdf" rel="nofollow noreferrer">Prediction, Learning, and Games</a> (2006) by Cesa-Bianchi and Lugosi</li>
  <li><a href="https://cs.nyu.edu/%7Emohri/mlbook/" rel="nofollow noreferrer">Foundations of Machine Learning</a> (2012) by Mohri et al.</li>
  <li><a href="https://doc.lagout.org/science/0_Computer%20Science/2_Algorithms/Boosting_%20Foundations%20and%20Algorithms%20%5BSchapire%20%26%20Freund%202012-05-18%5D.pdf" rel="nofollow noreferrer">Boosting: Foundations and Algorithms</a> (2012) by Schapire and Freund</li>
  <li><a href="https://tor-lattimore.com/downloads/book/book.pdf" rel="nofollow noreferrer">Bandit Algorithms</a> (2020) by Lattimore and Szepesvári (with associated <a href="https://tor-lattimore.com/downloads/book/solutions.pdf" rel="nofollow noreferrer">solutions to the exercises</a>)</li>
  </ul>
  <h3>Papers</h3>
  <ul>
  <li><p><a href="https://www.sciencedirect.com/science/article/pii/S0019995867911655" rel="nofollow noreferrer">Language identification in the limit</a> (1967) by E. Mark Gold (this paper introduces algorithmic learning theory, i.e. the non-statistical approach to CLT)</p>
  </li>
  <li><p><a href="http://web.mit.edu/6.435/www/Valiant84.pdf" rel="nofollow noreferrer">A theory of the learnable</a> (1984) by Valiant (this paper introduces PAC learning)</p>
  </li>
  <li><p><a href="http://www.mit.edu/%7E6.454/www_spring_2001/emin/slt.pdf" rel="nofollow noreferrer">An overview of statistical learning theory</a> (1999) by Vapnik</p>
  </li>
  <li><p><a href="http://www.econ.upf.edu/%7Elugosi/mlss_slt.pdf" rel="nofollow noreferrer">Introduction to Statistical Learning Theory</a> (2014) by Bousquet et al.</p>
  </li>
  </ul>
  <h3>Courses (videos)</h3>
  <ul>
  <li><a href="https://www.youtube.com/watch?v=mbyG85GZ0PI&amp;list=PLnIDYuXHkit4LcWjDe0EwlE57WiGlBs08" rel="nofollow noreferrer">Learning from Data (Caltech)</a> (2012) by Yaser Abu-Mostafa</li>
  <li><a href="https://www.youtube.com/watch?v=aILazXK059Y&amp;list=PLPW2keNyw-usgvmR7FTQ3ZRjfLs5jT4BO" rel="nofollow noreferrer">Machine Learning Theory</a> (2015) by Shai Ben-David</li>
  </ul>
  <h3>Lecture notes</h3>
  <ul>
  <li><p><a href="https://www.cs.ox.ac.uk/people/varun.kanade/teaching/CLT-MT2018/" rel="nofollow noreferrer">Computational Learning Theory - Michaelmas Term 2018</a> by Varun Kanade</p>
  </li>
  <li><p><a href="https://www.cse.huji.ac.il/%7Eshais/Handouts.pdf" rel="nofollow noreferrer">Introduction to Machine Learning</a> (2009) by Shai Shalev-Shwartz</p>
  </li>
  <li><p><a href="http://www.econ.upf.edu/%7Elugosi/anu.pdf" rel="nofollow noreferrer">Concentration-of-measure inequalities</a> (2009) by Gábor Lugosi</p>
  </li>
  <li><p><a href="http://stat.wharton.upenn.edu/%7Eskakade/courses/stat928/" rel="nofollow noreferrer">Stat 928: Statistical Learning Theory</a> (2011) by Sham Kakades</p>
  </li>
  <li><p><a href="http://www.cs.columbia.edu/%7Erocco/Teaching/S12/" rel="nofollow noreferrer">COMS E6253, Advanced Topics in Computational Learning Theory</a> (2012) Rocco Servedio</p>
  </li>
  <li><p><a href="https://www.mit.edu/%7E9.520/spring12/" rel="nofollow noreferrer">9.520: Statistical Learning Theory and Applications, Spring 2012</a> by Tomaso Poggio et al.</p>
  </li>
  <li><p><a href="http://www-stat.wharton.upenn.edu/%7Erakhlin/courses/stat928/" rel="nofollow noreferrer">STAT928: Statistical Learning Theory and Sequential Prediction</a> (2014) by Alexander Rakhlin and Karthik Sridharan</p>
  </li>
  <li><p><a href="http://maxim.ece.illinois.edu/teaching/SLT/" rel="nofollow noreferrer">ECE 543, Statistical Learning Theory</a> (2019) by Bruce Hajek and Maxim Raginsky</p>
  </li>
  </ul>
  <h3>Other</h3>
  <p>See also this list of resources <a href="https://kiranvodrahalli.github.io/links/#resources-notes-textbooks-monographs-classes-etc" rel="nofollow noreferrer">https://kiranvodrahalli.github.io/links/#resources-notes-textbooks-monographs-classes-etc</a> compiled by Kiran Vodrahalli.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-16T19:41:12.727Z
Answers.Text: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/118541/discussion-on-answer-by-nbro-what-are-some-resources-on-computational-learning-t).
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-17T17:07:08.537Z
Answers.LastActivityDate: 2022-01-16T23:13:23.253Z
Answers.LastEditDate: 2022-01-16T23:13:23.253Z
Answers.UserDisplayName: nbro
Body: <p>Pretty soon I will be finishing up <a href="https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf" rel="noreferrer">Understanding Machine Learning: From Theory to Algorithms</a> by Shai Ben-David and Shai Shalev-Shwartz. I absolutely love the subject and want to learn more, the only issue is I'm having trouble finding a book that could come after this. Ultimately, my goal is to read papers in <a href="http://proceedings.mlr.press" rel="noreferrer">JMLR's COLT</a>.</p>
  
  <ol>
  <li><p>Is there a book similar to "Understanding Machine Learning: From Theory to Algorithms" that would progress my knowledge further and would go well after reading UML?</p></li>
  <li><p>Is there any other materials (not a book) that could allow me to learn more or prepare me for reading a journal like the one mentioned above?</p></li>
  </ol>
  
  <p>(Also, taking courses in this is not really an option, so this will be for self-study).</p>
  
  <p>(Note that I have also asked this question here on <a href="https://cstheory.stackexchange.com/q/46650/34637">TCS SE</a>, but it was recommended I also ask here.)</p>
  
Tags: <reference-request><papers><computational-learning-theory><pac-learning><books>
Title: What are some resources on computational learning theory?
UserDisplayName: PMaynard
ThreadText: # What are some resources on computational learning theory?
  
  (Asked by: username_0 on 2020-04-17)
  
  Pretty soon I will be finishing up [Understanding Machine Learning: From Theory to Algorithms](https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf) by Shai Ben-David and Shai Shalev-Shwartz. I absolutely love the subject and want to learn more, the only issue is I'm having trouble finding a book that could come after this. Ultimately, my goal is to read papers in [JMLR's COLT](http://proceedings.mlr.press).
  
  1. Is there a book similar to "Understanding Machine Learning: From Theory to Algorithms" that would progress my knowledge further and would go well after reading UML?
  2. Is there any other materials (not a book) that could allow me to learn more or prepare me for reading a journal like the one mentioned above?
  
  (Also, taking courses in this is not really an option, so this will be for self-study).
  
  (Note that I have also asked this question here on [TCS SE](https://cstheory.stackexchange.com/q/46650/34637), but it was recommended I also ask here.)
  
  **Tags:** reference-request, papers, computational-learning-theory, pac-learning, books
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-04-17. Score: 11 | Accepted answer
  
  Although I have only partially read or not read at all some of the following resources and some of these resources may not cover more advanced topics than the ones presented in the book you are reading, I think they can still be useful for your purposes, so I will share them with you.
  
  I would also like to note that if you understand the contents of the book you are currently reading, you are probably already prepared for reading some (if not most of) the research papers you wish to read. Initially, you may find them a little bit too succinct and sometimes unclear or complex, but you need to get used to this format, so there's nothing stopping you from trying to read them and learn even more by doing this exercise.
  
  ### Books
  
  * [An Introduction to Computational Learning Theory](https://mitpress.mit.edu/books/introduction-computational-learning-theory) (1994) by Kearns and Vazirani (no free PDF is available, afaik)
  * [The Nature of Statistical Learning Theory](https://statisticalsupportandresearch.files.wordpress.com/2017/05/vladimir-vapnik-the-nature-of-statistical-learning-springer-2010.pdf) (1995, 2000) by Vapnik
  * [Machine Learning](http://profsite.um.ac.ir/%7Emonsefi/machine-learning/pdf/Machine-Learning-Tom-Mitchell.pdf) (1997) by Mitchell
  * [Statistical Learning Theory](http://read.pudn.com/downloads161/ebook/733192/Statistical-Learning-Theory.pdf) (1998) by Vapnik
  * [Prediction, Learning, and Games](http://www.ii.uni.wroc.pl/%7Elukstafi/pmwiki/uploads/AGT/Prediction_Learning_and_Games.pdf) (2006) by Cesa-Bianchi and Lugosi
  * [Foundations of Machine Learning](https://cs.nyu.edu/%7Emohri/mlbook/) (2012) by Mohri et al.
  * [Boosting: Foundations and Algorithms](https://doc.lagout.org/science/0_Computer%20Science/2_Algorithms/Boosting_%20Foundations%20and%20Algorithms%20%5BSchapire%20%26%20Freund%202012-05-18%5D.pdf) (2012) by Schapire and Freund
  * [Bandit Algorithms](https://tor-lattimore.com/downloads/book/book.pdf) (2020) by Lattimore and Szepesvári (with associated [solutions to the exercises](https://tor-lattimore.com/downloads/book/solutions.pdf))
  
  ### Papers
  
  * [Language identification in the limit](https://www.sciencedirect.com/science/article/pii/S0019995867911655) (1967) by E. Mark Gold (this paper introduces algorithmic learning theory, i.e. the non-statistical approach to CLT)
  * [A theory of the learnable](http://web.mit.edu/6.435/www/Valiant84.pdf) (1984) by Valiant (this paper introduces PAC learning)
  * [An overview of statistical learning theory](http://www.mit.edu/%7E6.454/www_spring_2001/emin/slt.pdf) (1999) by Vapnik
  * [Introduction to Statistical Learning Theory](http://www.econ.upf.edu/%7Elugosi/mlss_slt.pdf) (2014) by Bousquet et al.
  
  ### Courses (videos)
  
  * [Learning from Data (Caltech)](https://www.youtube.com/watch?v=mbyG85GZ0PI&list=PLnIDYuXHkit4LcWjDe0EwlE57WiGlBs08) (2012) by Yaser Abu-Mostafa
  * [Machine Learning Theory](https://www.youtube.com/watch?v=aILazXK059Y&list=PLPW2keNyw-usgvmR7FTQ3ZRjfLs5jT4BO) (2015) by Shai Ben-David
  
  ### Lecture notes
  
  * [Computational Learning Theory - Michaelmas Term 2018](https://www.cs.ox.ac.uk/people/varun.kanade/teaching/CLT-MT2018/) by Varun Kanade
  * [Introduction to Machine Learning](https://www.cse.huji.ac.il/%7Eshais/Handouts.pdf) (2009) by Shai Shalev-Shwartz
  * [Concentration-of-measure inequalities](http://www.econ.upf.edu/%7Elugosi/anu.pdf) (2009) by Gábor Lugosi
  * [Stat 928: Statistical Learning Theory](http://stat.wharton.upenn.edu/%7Eskakade/courses/stat928/) (2011) by Sham Kakades
  * [COMS E6253, Advanced Topics in Computational Learning Theory](http://www.cs.columbia.edu/%7Erocco/Teaching/S12/) (2012) Rocco Servedio
  * [9.520: Statistical Learning Theory and Applications, Spring 2012](https://www.mit.edu/%7E9.520/spring12/) by Tomaso Poggio et al.
  * [STAT928: Statistical Learning Theory and Sequential Prediction](http://www-stat.wharton.upenn.edu/%7Erakhlin/courses/stat928/) (2014) by Alexander Rakhlin and Karthik Sridharan
  * [ECE 543, Statistical Learning Theory](http://maxim.ece.illinois.edu/teaching/SLT/) (2019) by Bruce Hajek and Maxim Raginsky
  
  ### Other
  
  See also this list of resources <https://kiranvodrahalli.github.io/links/#resources-notes-textbooks-monographs-classes-etc> compiled by Kiran Vodrahalli.
  
  > **Comments:**
  > **username_1**: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/118541/discussion-on-answer-by-username_1-what-are-some-resources-on-computational-learning-t).  
  
  ---
  
  