Answers.Body: <p>I think one of the easiest ML algorithms to understand is <a href="https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm" rel="nofollow noreferrer">K-Nearest Neighbors</a> (KNN). We do a KNN project in my data structures class in the first week of school so that they can refresh their skills from AP CS and demystify ML a little bit. </p>
  
  <p>What I like about KNN is that it's conceptually easy to understand for high school students (I showcased it at Open House to the parents), it works for both classification and regression, and has its advantages and disadvantages. It's one of my favorite projects we do all year. </p>
  
  <p>I'd be happy to chat offline if you want some specifics. </p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-22T16:27:22.083Z
Answers.LastActivityDate: 2018-01-22T16:27:22.083Z
Answers.UserDisplayName: iFrame
Answers.Body: <p>There are all sorts of things you <em>could</em> cover — machine learning is an extremely interesting and growing field, with many different approaches and tools you could explore.</p>
  
  <p>Some of the more popular tasks for machine learning algorithms include <strong>classification</strong>, <strong>regression</strong> and <strong>clustering</strong>. Not every application fits into this grouping (e.g. <a href="https://en.wikipedia.org/wiki/AlphaGo" rel="noreferrer">AlphaGo</a>, the Go playing program which beat Lee Se-dol).</p>
  
  <p>I might suggest starting with <a href="https://en.wikipedia.org/wiki/Naive_Bayes_classifier" rel="noreferrer">Naive Bayes classifiers</a>. Your motivating example here could be <a href="https://en.wikipedia.org/wiki/Naive_Bayes_spam_filtering" rel="noreferrer">spam filtering</a>; Naive Bayes classifiers were used in some of the first 'learning' spam filters, dating back to the 1990s. If your students are familiar with Bayes' theorem, the assumptions of NB classifiers shouldn't be a huge leap (you may find <a href="http://scikit-learn.org/stable/modules/naive_bayes.html" rel="noreferrer">this derivation</a> interesting).</p>
  
  <p>There are also various other interesting options such as <a href="https://en.wikipedia.org/wiki/Random_forest" rel="noreferrer">random forests</a> (apparently used by <a href="https://www.quora.com/" rel="noreferrer">Quora</a> to <a href="https://www.kaggle.com/c/quora-question-pairs" rel="noreferrer">find duplicate questions</a>), <a href="https://en.wikipedia.org/wiki/Support_vector_machine" rel="noreferrer">support vector machines</a> and so forth which could be explored potentially.</p>
  
  <p>Many of the most 'fashionable' techniques involve <a href="https://en.wikipedia.org/wiki/Artificial_neural_network" rel="noreferrer">neural networks</a>, and I would recommend spending a decent amount of time with the theory. Unlike some of the classifiers and regressors I mentioned earlier, neural networks tend to be a bit more involved — a Naive Bayes classifier essentially just needs to be given some data in an appropriate form and is "plug and play", so to speak.</p>
  
  <p>The (free) online book <a href="http://neuralnetworksanddeeplearning.com/index.html" rel="noreferrer">Neural Networks and Deep Learning</a> gives a reasonably accessible explanation of neural networks in their various forms (starting with perceptrons, leading towards sigmoid neurons, gradient descent, backpropagation, deep learning, etc.). As you can imagine, there is an immense amount of content even in just the field of neural networks. </p>
  
  <p>Likely you won't want to cover all of the book's topics, but you don't need to go too far to reach another interesting motivating example: <a href="http://yann.lecun.com/exdb/mnist/" rel="noreferrer">classifying the MNIST dataset</a> of handwritten digits. That's something which is relatively hard traditionally, but much simpler with the usage of a neural network.</p>
  
  <hr>
  
  <p>Note that I've discussed a lot without ever actually explaining how you could <em>write any programs</em>. Of course, any machine learning class would be a little incomplete if your students had never actually applied their skills. </p>
  
  <p>Since your students know Python, I can highly recommend <a href="http://scikit-learn.org/stable/" rel="noreferrer">scikit-learn</a>. It's <a href="http://scikit-learn.org/stable/testimonials/testimonials.html" rel="noreferrer">widely regarded</a> as an exceptionally good ML library, and the API is very friendly — you can almost treat it as LEGO, and plug together the pieces needed to do your classification/regression/clustering... even without really knowing how each piece works entirely. You can get functional (though not exactly <em>effective</em>) solutions simply by connecting the appropriate pieces in a pipeline.</p>
  
  <p>For example, if you wanted to classify some texts, you'd just:</p>
  
  <ul>
  <li>collect some data</li>
  <li>setup a <code>Pipeline</code> of a <code>CountVectorizer</code> and a classifer such as <code>LinearSVC</code></li>
  <li>train it, and then test it.</li>
  </ul>
  
  <p>There's an awful lot of stuff included, but then again, all the batteries you'll ever need are included, too.</p>
  
  <p>For playing with neural networks, <a href="https://keras.io/" rel="noreferrer">Keras</a> might be worthwhile.</p>
  
  <p>That said, your students may derive some value from implementing the algorithms themselves, at first. The key is to be able to find truly motivating problems:</p>
  
  <blockquote>
    <p>Struggling with a project you care about will teach you far more than working through any number of set problems. Emotional commitment is a key to achieving mastery. <sup>(<a href="http://neuralnetworksanddeeplearning.com/exercises_and_problems.html" rel="noreferrer">source</a>)</sup></p>
  </blockquote>
  
  <p>It's not always easy if you're supervising many students, but it is worthwhile advice to consider.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-18T09:54:25.313Z
Answers.Text: The course will be taught in high school. May I ask what math prerequisites would be needed for those topics you listed in your answer?
Answers.UserDisplayName: Nobody
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-18T15:10:08.163Z
Answers.Text: @scaaahu: It's mainly linear algebra that comes up in machine learning. A familiarity with vectors and working with them (such as the dot product, gradient of a vector) and some calculus (differentiation and partial derivatives) would be helpful. If you didn't want to get bogged down with the details, you could probably explain what you needed as you taught it, but certainly the vectors pop up *everywhere* so an awareness would be helpful.
Answers.UserDisplayName: Aurora0001
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-17T20:38:06.570Z
Answers.LastActivityDate: 2018-01-17T20:38:06.570Z
Answers.UserDisplayName: Aurora0001
Answers.Body: <blockquote>
    <p>What are some topics (Machine Learning) that could be covered in a year long timespan that could be done by students with a basic understanding of Python and basic statistics (correlation/regression, measures of central tendency, hypothesis testing)?</p>
  </blockquote>
  
  <p>With regards to Neural Networks the for an introduction, even before specking oneself or doing suggested reading would be to watch the YouTube series of videos <a href="https://www.youtube.com/watch?v=bxe2T-V8XRs" rel="nofollow noreferrer">Neural Networks Demystified</a>. When I first wanted to learn about Neural Networks I spent a few days combing the Internet and books and once I saw this series I stopped looking. I still get regular YouTube recommendations but this is the best introductory one by far. It is easy to understand, has great visuals, uses Python as an example programming language, is a real world problem, moves seamlessly back and forth between high level concepts and low level details as source code, etc. </p>
  
  <p>The next topic(s) are to use the free online book <a href="http://neuralnetworksanddeeplearning.com/index.html" rel="nofollow noreferrer">Neural Networks and Deep Learning</a> which is often recommended even in the other answer here. You can build a few weeks of the course around just this and it would be worth the effort. This book goes more into the math and explains why certain <a href="https://en.wikipedia.org/wiki/Activation_function" rel="nofollow noreferrer">activation functions</a> are used. While I am not actively writing Neural Networks at present I do keep my finger on the pulse and one of the more intriguing ideas on the street is that using <a href="https://en.wikipedia.org/wiki/Rectifier_(neural_networks)" rel="nofollow noreferrer">Rectified linear unit (ReLU)</a> are just as effective as the <a href="https://en.wikipedia.org/wiki/Sigmoid_function" rel="nofollow noreferrer">Sigmoid function</a>. The beauty of this is that using ReLU doesn't need the use of a floating point processor and so in theory should speed up the processing without sacrificing abilities. So as a set of labs do the book as written, but then try other activation functions and compare the results. </p>
  
  <p>Another topic is to run the code using a CPU and then using a GPU and compare the speed. If done correctly the <a href="http://timdettmers.com/2017/04/09/which-gpu-for-deep-learning/" rel="nofollow noreferrer">GPU</a> will be significantly faster then the CPU. As another option compare those against using a cloud service such as <a href="https://aws.amazon.com/" rel="nofollow noreferrer">AWS</a></p>
  
  <p>After the concepts of Neural Networks are understood at the Python source code level, abstract that away and use <a href="https://www.tensorflow.org/" rel="nofollow noreferrer">TensorFlow</a> which is what Google uses. </p>
  
  <p>Lastly one topic I am seeing pop up much to often with regards to Neural Networks is for classification how easily they are fooled, e.g. <a href="https://techxplore.com/news/2018-01-world-toaster-ai.html" rel="nofollow noreferrer">When all the world's a toaster, according to tricked AI</a> After seeing this I wanted to get a sticker and put it on a shirt saying "I am not a toaster" but then forgot that they already have shirts saying I am not a toaster, referring to <a href="https://en.wikipedia.org/wiki/Cylon_(Battlestar_Galactica)" rel="nofollow noreferrer">Cylons</a>.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-19T14:02:38.580Z
Answers.Text: Very interesting point about adversarial examples; students might also find examples like [this paper](https://arxiv.org/pdf/1710.08864.pdf) interesting in which deep neural nets were fooled by one pixel changes. It really does make it clear how neural nets don't really see things the way we do (and perhaps makes driverless cars and such sound far more dangerous, if such small changes can fool them!)
Answers.UserDisplayName: Aurora0001
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-19T17:07:47.617Z
Answers.Text: @Aurora0001 Thanks for the reference about the single pixel difference. I know about it but could not find it.
Answers.UserDisplayName: Guy Coder
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-01-19T11:45:04.147Z
Answers.LastActivityDate: 2018-01-19T11:52:44.487Z
Answers.LastEditDate: 2018-01-19T11:52:44.487Z
Answers.UserDisplayName: Guy Coder
Body: <p>I'm a computer science teacher and our department is thinking about opening a data analysis and machine learning course for upperclassmen at my high school.</p>
  
  <p>What are some topics that could be covered in a year long timespan that could be done by students with a basic understanding of Python and basic statistics (correlation/regression, measures of central tendency, hypothesis testing)?</p>
  
  <p>I'm thinking the total time commitment available to students is 3.5 hours a week of in-class time, and approximately 2 to 3 hours a week of coding, learning, or doing homework.</p>
  
  <p>Some topics that we were thinking might be relevant are image processing and natural language processing.</p>
  
  <p>Are there any projects that really hit the ground running in terms of showcasing a topic in machine learning?</p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-17T17:50:54.020Z
Comments.Text: Welcome to CSEducators. I hope you get good answers to your question and come back for more in the future.
Comments.UserDisplayName: Buffy
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-17T19:24:49.167Z
Comments.Text: Can you translate  “upperclassmen” for the international audience. I am from England and have no idea what it means. (may be a foot note, with age etc.)
Comments.UserDisplayName: ctrl-alt-delor
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-17T22:13:47.413Z
Comments.Text: You may also be interested in [this question](https://cseducators.stackexchange.com/questions/494/deep-learning-teaching-resources).
Comments.UserDisplayName: Ben I.
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-17T23:58:25.167Z
Comments.Text: The definition is loose, but upperclassmen are generally 11th and 12th graders (16 - 18 years old), but sometimes include 10th graders.
Comments.UserDisplayName: Jonathan Ma
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-18T01:04:25.610Z
Comments.Text: A machine learning course is a learning course for machines. Which machines exactly would attend such a course?
Comments.UserDisplayName: AlMa1r
Tags: <secondary-education><machine-learning>
Title: Opening a machine learning course in high school
ThreadText: # Opening a machine learning course in high school
  
  (Asked by: username_0 on 2018-01-17)
  
  I'm a computer science teacher and our department is thinking about opening a data analysis and machine learning course for upperclassmen at my high school.
  
  What are some topics that could be covered in a year long timespan that could be done by students with a basic understanding of Python and basic statistics (correlation/regression, measures of central tendency, hypothesis testing)?
  
  I'm thinking the total time commitment available to students is 3.5 hours a week of in-class time, and approximately 2 to 3 hours a week of coding, learning, or doing homework.
  
  Some topics that we were thinking might be relevant are image processing and natural language processing.
  
  Are there any projects that really hit the ground running in terms of showcasing a topic in machine learning?
  
  **Tags:** secondary-education, machine-learning
  
  > **Comments:**
  > **username_1**: Welcome to CSEducators. I hope you get good answers to your question and come back for more in the future.  
  > **username_2**: Can you translate  “upperclassmen” for the international audience. I am from England and have no idea what it means. (may be a foot note, with age etc.)  
  > **username_3**: You may also be interested in [this question](https://cseducators.stackexchange.com/questions/494/deep-learning-teaching-resources).  
  > **username_0**: The definition is loose, but upperclassmen are generally 11th and 12th graders (16 - 18 years old), but sometimes include 10th graders.  
  > **username_4**: A machine learning course is a learning course for machines. Which machines exactly would attend such a course?  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_6 on 2018-01-17. Score: 5
  
  There are all sorts of things you *could* cover — machine learning is an extremely interesting and growing field, with many different approaches and tools you could explore.
  
  Some of the more popular tasks for machine learning algorithms include **classification**, **regression** and **clustering**. Not every application fits into this grouping (e.g. [AlphaGo](https://en.wikipedia.org/wiki/AlphaGo), the Go playing program which beat Lee Se-dol).
  
  I might suggest starting with [Naive Bayes classifiers](https://en.wikipedia.org/wiki/Naive_Bayes_classifier). Your motivating example here could be [spam filtering](https://en.wikipedia.org/wiki/Naive_Bayes_spam_filtering); Naive Bayes classifiers were used in some of the first 'learning' spam filters, dating back to the 1990s. If your students are familiar with Bayes' theorem, the assumptions of NB classifiers shouldn't be a huge leap (you may find [this derivation](http://scikit-learn.org/stable/modules/naive_bayes.html) interesting).
  
  There are also various other interesting options such as [random forests](https://en.wikipedia.org/wiki/Random_forest) (apparently used by [Quora](https://www.quora.com/) to [find duplicate questions](https://www.kaggle.com/c/quora-question-pairs)), [support vector machines](https://en.wikipedia.org/wiki/Support_vector_machine) and so forth which could be explored potentially.
  
  Many of the most 'fashionable' techniques involve [neural networks](https://en.wikipedia.org/wiki/Artificial_neural_network), and I would recommend spending a decent amount of time with the theory. Unlike some of the classifiers and regressors I mentioned earlier, neural networks tend to be a bit more involved — a Naive Bayes classifier essentially just needs to be given some data in an appropriate form and is "plug and play", so to speak.
  
  The (free) online book [Neural Networks and Deep Learning](http://neuralnetworksanddeeplearning.com/index.html) gives a reasonably accessible explanation of neural networks in their various forms (starting with perceptrons, leading towards sigmoid neurons, gradient descent, backpropagation, deep learning, etc.). As you can imagine, there is an immense amount of content even in just the field of neural networks.
  
  Likely you won't want to cover all of the book's topics, but you don't need to go too far to reach another interesting motivating example: [classifying the MNIST dataset](http://yann.lecun.com/exdb/mnist/) of handwritten digits. That's something which is relatively hard traditionally, but much simpler with the usage of a neural network.
  
  ---
  
  Note that I've discussed a lot without ever actually explaining how you could *write any programs*. Of course, any machine learning class would be a little incomplete if your students had never actually applied their skills.
  
  Since your students know Python, I can highly recommend [scikit-learn](http://scikit-learn.org/stable/). It's [widely regarded](http://scikit-learn.org/stable/testimonials/testimonials.html) as an exceptionally good ML library, and the API is very friendly — you can almost treat it as LEGO, and plug together the pieces needed to do your classification/regression/clustering... even without really knowing how each piece works entirely. You can get functional (though not exactly *effective*) solutions simply by connecting the appropriate pieces in a pipeline.
  
  For example, if you wanted to classify some texts, you'd just:
  
  * collect some data
  * setup a `Pipeline` of a `CountVectorizer` and a classifer such as `LinearSVC`
  * train it, and then test it.
  
  There's an awful lot of stuff included, but then again, all the batteries you'll ever need are included, too.
  
  For playing with neural networks, [Keras](https://keras.io/) might be worthwhile.
  
  That said, your students may derive some value from implementing the algorithms themselves, at first. The key is to be able to find truly motivating problems:
  
  > Struggling with a project you care about will teach you far more than working through any number of set problems. Emotional commitment is a key to achieving mastery. ([source](http://neuralnetworksanddeeplearning.com/exercises_and_problems.html))
  
  It's not always easy if you're supervising many students, but it is worthwhile advice to consider.
  
  > **Comments:**
  > **username_7**: The course will be taught in high school. May I ask what math prerequisites would be needed for those topics you listed in your answer?  
  > **username_6**: @scaaahu: It's mainly linear algebra that comes up in machine learning. A familiarity with vectors and working with them (such as the dot product, gradient of a vector) and some calculus (differentiation and partial derivatives) would be helpful. If you didn't want to get bogged down with the details, you could probably explain what you needed as you taught it, but certainly the vectors pop up *everywhere* so an awareness would be helpful.  
  
  ---
  
  ### Answer by username_8 on 2018-01-19. Score: 2
  
  > What are some topics (Machine Learning) that could be covered in a year long timespan that could be done by students with a basic understanding of Python and basic statistics (correlation/regression, measures of central tendency, hypothesis testing)?
  
  With regards to Neural Networks the for an introduction, even before specking oneself or doing suggested reading would be to watch the YouTube series of videos [Neural Networks Demystified](https://www.youtube.com/watch?v=bxe2T-V8XRs). When I first wanted to learn about Neural Networks I spent a few days combing the Internet and books and once I saw this series I stopped looking. I still get regular YouTube recommendations but this is the best introductory one by far. It is easy to understand, has great visuals, uses Python as an example programming language, is a real world problem, moves seamlessly back and forth between high level concepts and low level details as source code, etc.
  
  The next topic(s) are to use the free online book [Neural Networks and Deep Learning](http://neuralnetworksanddeeplearning.com/index.html) which is often recommended even in the other answer here. You can build a few weeks of the course around just this and it would be worth the effort. This book goes more into the math and explains why certain [activation functions](https://en.wikipedia.org/wiki/Activation_function) are used. While I am not actively writing Neural Networks at present I do keep my finger on the pulse and one of the more intriguing ideas on the street is that using [Rectified linear unit (ReLU)](https://en.wikipedia.org/wiki/Rectifier_(neural_networks)) are just as effective as the [Sigmoid function](https://en.wikipedia.org/wiki/Sigmoid_function). The beauty of this is that using ReLU doesn't need the use of a floating point processor and so in theory should speed up the processing without sacrificing abilities. So as a set of labs do the book as written, but then try other activation functions and compare the results.
  
  Another topic is to run the code using a CPU and then using a GPU and compare the speed. If done correctly the [GPU](http://timdettmers.com/2017/04/09/which-gpu-for-deep-learning/) will be significantly faster then the CPU. As another option compare those against using a cloud service such as [AWS](https://aws.amazon.com/)
  
  After the concepts of Neural Networks are understood at the Python source code level, abstract that away and use [TensorFlow](https://www.tensorflow.org/) which is what Google uses.
  
  Lastly one topic I am seeing pop up much to often with regards to Neural Networks is for classification how easily they are fooled, e.g. [When all the world's a toaster, according to tricked AI](https://techxplore.com/news/2018-01-world-toaster-ai.html) After seeing this I wanted to get a sticker and put it on a shirt saying "I am not a toaster" but then forgot that they already have shirts saying I am not a toaster, referring to [Cylons](https://en.wikipedia.org/wiki/Cylon_(Battlestar_Galactica)).
  
  > **Comments:**
  > **username_6**: Very interesting point about adversarial examples; students might also find examples like [this paper](https://arxiv.org/pdf/1710.08864.pdf) interesting in which deep neural nets were fooled by one pixel changes. It really does make it clear how neural nets don't really see things the way we do (and perhaps makes driverless cars and such sound far more dangerous, if such small changes can fool them!)  
  > **username_8**: @username_6 Thanks for the reference about the single pixel difference. I know about it but could not find it.  
  
  ---
  
  ### Answer by username_5 on 2018-01-22. Score: 1
  
  I think one of the easiest ML algorithms to understand is [K-Nearest Neighbors](https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm) (KNN). We do a KNN project in my data structures class in the first week of school so that they can refresh their skills from AP CS and demystify ML a little bit.
  
  What I like about KNN is that it's conceptually easy to understand for high school students (I showcased it at Open House to the parents), it works for both classification and regression, and has its advantages and disadvantages. It's one of my favorite projects we do all year.
  
  I'd be happy to chat offline if you want some specifics.
  
  
  ---
  
  