Answers.Body: <p>Here's a definition by Tom Mitchel (1997): </p>
  
  <blockquote>
    <p>Computer program is said to learn from experience E with respect to
    some task T and some performance measure P, if its performance on T,
    as measured by P, improves with experience E.</p>
  </blockquote>
  
  <p>So, the programmer gives some instructions/rules to the computer, so that it can learn how to solve the problem from the given examples by themselves. </p>
  
  <p>In some tasks, the computer can perform better than humans. For example, the Dota 2 bot (made by OpenAI) defeated the world champion. </p>
  
  <p>With machine learning, many cases can be automated. They also have the ability to improve solutions by learning the given data from time to time. It can process and analyze large data well.</p>
  
  <p>Machine learning already applied in many fields such as machine translation in google translate and face recognition that is widely used by today's society for security.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-21T09:48:48.153Z
Answers.LastActivityDate: 2020-04-21T12:44:26.153Z
Answers.LastEditDate: 2020-04-21T12:44:26.153Z
Answers.UserDisplayName: Beladina Elfitri
Answers.Body: <h3>What is machine learning?</h3>
  <p>Machine learning (ML) has been defined by multiple people in similar (or related) ways.</p>
  <p>Tom Mitchell, in his book <a href="http://www.cs.ubbcluj.ro/%7Egabis/ml/ML-books/McGrawHill%20-%20Machine%20Learning%20-Tom%20Mitchell.pdf" rel="nofollow noreferrer">Machine Learning</a> (1997), defines an ML <strong>algorithm/program</strong> (or machine learner) as follows.</p>
  <blockquote>
  <p>A computer program is said to <em>learn</em> from <strong>experience</strong> <span class="math-container">$E$</span> with respect to some class of tasks <span class="math-container">$T$</span> and performance measure <span class="math-container">$P$</span>, if its performance at tasks in <span class="math-container">$T$</span>, as measured by <span class="math-container">$P$</span>, improves with experience <span class="math-container">$E$</span>.</p>
  </blockquote>
  <p>This is a quite reasonable definition, given that it describes algorithms such as gradient descent, Q-learning, etc.</p>
  <p>In his book <a href="http://noiselab.ucsd.edu/ECE228/Murphy_Machine_Learning.pdf" rel="nofollow noreferrer">Machine Learning: A Probabilistic Perspective</a> (2012), Kevin P. Murphy defines the machine learning <strong>field/area</strong> as follows.</p>
  <blockquote>
  <p>a set of methods that can automatically detect patterns in <strong>data</strong>, and then use the uncovered patterns to predict future data, or to perform other kinds of decision making under uncertainty (such as planning how to collect more data!)</p>
  </blockquote>
  <p>Without referring to algorithms or the field, <a href="https://www.cs.huji.ac.il/%7Eshais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf" rel="nofollow noreferrer">Shai Shalev-Shwartz and Shai Ben-David</a> define machine learning as follows</p>
  <blockquote>
  <p>The term machine learning refers to the automated detection of meaningful patterns in <strong>data</strong>.</p>
  </blockquote>
  <p>In all these definitions, the core concept is <strong>data</strong> or <strong>experience</strong>. So, any <a href="https://edutechlearners.com/download/Introduction_to_algorithms-3rd%20Edition.pdf" rel="nofollow noreferrer">algorithm</a> that <em>automatically</em> detects patterns in <strong>data</strong> (of any form, such as textual, numerical, or categorical) to solve some task/problem (which often involves more data) is a (machine) learning algorithm.</p>
  <p>The tricky part of this definition, which often causes a lot of misconceptions about what ML is or can do, is probably <em>automatically</em>: this does not mean that the learning algorithm is completely autonomous or independent from the human, given that the human, in most cases, still needs to define a <em>performance measure</em> (and other parameters, including the learning algorithm itself) that guides the learning algorithm towards a set of solutions to the problem being solved.</p>
  <p>As a <a href="https://ai.stackexchange.com/a/26664/2444">field</a>, ML could be defined as the study and application of ML algorithms (as defined by Mitchell's definition).</p>
  <h3>Sub-categories</h3>
  <p>Murphy and many others often divide machine learning into three main sub-categories</p>
  <ul>
  <li><p><strong>supervised learning</strong> (or predictive), where the goal is to learn a mapping from inputs <span class="math-container">$\textbf{x}$</span> to outputs <span class="math-container">$y$</span>, given a labeled set of input-output pairs</p>
  </li>
  <li><p><a href="http://mlg.eng.cam.ac.uk/zoubin/papers/ul.pdf" rel="nofollow noreferrer"><strong>unsupervised learning</strong></a> (or descriptive), where the goal is to find &quot;interesting patterns&quot; in the data</p>
  </li>
  <li><p><a href="http://incompleteideas.net/book/RLbook2020.pdf" rel="nofollow noreferrer"><strong>reinforcement learning</strong></a>, which is useful for learning how to act or behave when given an occasional reward or punishment signals</p>
  </li>
  </ul>
  <p>However, there are many other possible sub-categories (or taxonomies) of machine learning techniques, such as</p>
  <ul>
  <li><a href="https://www.sciencedirect.com/science/article/pii/S0893608014002135" rel="nofollow noreferrer"><strong>deep learning</strong></a> (i.e. the use of neural networks to approximate functions and related learning algorithms, such as gradient descent) or</li>
  <li><a href="https://www.nature.com/articles/nature14541" rel="nofollow noreferrer"><strong>probabilistic machine learning</strong></a> (machine learning techniques that provide uncertainty estimation)</li>
  <li><a href="https://academic.oup.com/nsr/article/5/1/44/4093912" rel="nofollow noreferrer"><strong>weakly supervised learning</strong></a> (i.e. SL where labeling information may not be completely accurate)</li>
  <li><a href="https://www.cs.huji.ac.il/%7Eshais/papers/OLsurvey.pdf" rel="nofollow noreferrer"><strong>online learning</strong></a> (i.e. learning from a single data point at a time rather than from a dataset of multiple data points)</li>
  </ul>
  <p>These sub-categories can also be combined. For example, deep learning can be performed online or offline.</p>
  <h3>Related fields</h3>
  <p>There is also a related field known as <strong>computational</strong> (or <strong>statistical</strong>) <strong>learning theory</strong>, which concerned with the <em>theory of learning</em> (from a computational and statistical point of view). So, in this field, we are interested in questions like &quot;How many samples do we need to approximately compute this function with a certain error?&quot;.</p>
  <p>Of course, given that machine learning is a set of algorithms and techniques that are data- or experience-driven, one may wonder what the difference between machine learning and <strong>statistics</strong> is. In fact, in many cases, they are very similar and ML adopts many statistical concepts, and you may even read on the web that machine learning is just <em>glorified statistics</em>. ML and statistics often tackle the same problem, but from a different perspective or with slightly different approaches (and the terminology may slightly change from one field to the other). If you are interested in a more detailed explanation of their difference, you could read <a href="https://www.nature.com/articles/nmeth.4642" rel="nofollow noreferrer">Statistics versus machine learning</a> (2018) by Danilo Bzdok et al.</p>
  <h3>What is machine learning good for?</h3>
  <p>ML can potentially be used to (at least partially) automate tasks that involve data and pattern recognition, which were previously performed only by humans (e.g. translation from one human language, such as English, to another, such as Italian). However, machine learning cannot automate all tasks: for example, it cannot infer causal relations from the data (which often must be done by humans), unless you include <strong>causal inference</strong> as part of machine learning. If you are interested in causal inference, you could take a look at the paper <a href="http://proceedings.mlr.press/v6/pearl10a/pearl10a.pdf" rel="nofollow noreferrer">Causal Inference</a> by Judea Pearl (<a href="https://amturing.acm.org/award_winners/pearl_2658896.cfm" rel="nofollow noreferrer">Turing Award for his work in causal inference!</a>).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-20T17:20:13.067Z
Answers.Text: With regards to the definition of Mitchell, a traditional spam filter can also learn from experience? Suppose that at some time $t$ the emails that arrived to the traditional spam filter are so hard to classify for it (its hard-wired rules don't match the underlying patterns). Now if at some time $t'$ the emails that arrive are easy to classify them, then its performance has improved. Can we say that it has learned from experience? I would argue not, since the improvement was not due to experience (it was just fed with easy to classify emails).
Answers.UserDisplayName: Antonios Sarikas
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-20T17:28:38.130Z
Answers.Text: However, it is still very difficult how one should apply Mitchell's definition. What "improves with experience mean"? Does it mean "with more data"?
Answers.UserDisplayName: Antonios Sarikas
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-23T22:19:38.020Z
Answers.Text: @adosar Essentially, yes, experience is data. If you're familiar with RL and, in particular, DQN, you have heard of the term _experience replay_, which is kind of a data structure that stores the data the agent has observed during the interaction with the environment.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-27T19:04:42.560Z
Answers.LastActivityDate: 2021-03-11T10:01:25.157Z
Answers.LastEditDate: 2021-03-11T10:01:25.157Z
Answers.UserDisplayName: nbro
Body: <p>What is the definition of machine learning? What are the advantages of machine learning? </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-16T18:03:32.070Z
Comments.Text: [Here](https://ai.stackexchange.com/q/4706/2444) is a related question.
Comments.UserDisplayName: nbro
Tags: <machine-learning><terminology><definitions>
Title: What is machine learning?
UserDisplayName: Israr Ali
ThreadText: # What is machine learning?
  
  (Asked by: username_0 on 2019-05-27)
  
  What is the definition of machine learning? What are the advantages of machine learning?
  
  **Tags:** machine-learning, terminology, definitions
  
  > **Comments:**
  > **username_1**: [Here](https://ai.stackexchange.com/q/4706/2444) is a related question.  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2019-05-27. Score: 8 | Accepted answer
  
  ### What is machine learning?
  
  Machine learning (ML) has been defined by multiple people in similar (or related) ways.
  
  Tom Mitchell, in his book [Machine Learning](http://www.cs.ubbcluj.ro/%7Egabis/ml/ML-books/McGrawHill%20-%20Machine%20Learning%20-Tom%20Mitchell.pdf) (1997), defines an ML **algorithm/program** (or machine learner) as follows.
  
  > A computer program is said to *learn* from **experience** $E$ with respect to some class of tasks $T$ and performance measure $P$, if its performance at tasks in $T$, as measured by $P$, improves with experience $E$.
  
  This is a quite reasonable definition, given that it describes algorithms such as gradient descent, Q-learning, etc.
  
  In his book [Machine Learning: A Probabilistic Perspective](http://noiselab.ucsd.edu/ECE228/Murphy_Machine_Learning.pdf) (2012), Kevin P. Murphy defines the machine learning **field/area** as follows.
  
  > a set of methods that can automatically detect patterns in **data**, and then use the uncovered patterns to predict future data, or to perform other kinds of decision making under uncertainty (such as planning how to collect more data!)
  
  Without referring to algorithms or the field, [Shai Shalev-Shwartz and Shai Ben-David](https://www.cs.huji.ac.il/%7Eshais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf) define machine learning as follows
  
  > The term machine learning refers to the automated detection of meaningful patterns in **data**.
  
  In all these definitions, the core concept is **data** or **experience**. So, any [algorithm](https://edutechlearners.com/download/Introduction_to_algorithms-3rd%20Edition.pdf) that *automatically* detects patterns in **data** (of any form, such as textual, numerical, or categorical) to solve some task/problem (which often involves more data) is a (machine) learning algorithm.
  
  The tricky part of this definition, which often causes a lot of misconceptions about what ML is or can do, is probably *automatically*: this does not mean that the learning algorithm is completely autonomous or independent from the human, given that the human, in most cases, still needs to define a *performance measure* (and other parameters, including the learning algorithm itself) that guides the learning algorithm towards a set of solutions to the problem being solved.
  
  As a [field](https://ai.stackexchange.com/a/26664/2444), ML could be defined as the study and application of ML algorithms (as defined by Mitchell's definition).
  
  ### Sub-categories
  
  Murphy and many others often divide machine learning into three main sub-categories
  
  * **supervised learning** (or predictive), where the goal is to learn a mapping from inputs $\textbf{x}$ to outputs $y$, given a labeled set of input-output pairs
  * [**unsupervised learning**](http://mlg.eng.cam.ac.uk/zoubin/papers/ul.pdf) (or descriptive), where the goal is to find "interesting patterns" in the data
  * [**reinforcement learning**](http://incompleteideas.net/book/RLbook2020.pdf), which is useful for learning how to act or behave when given an occasional reward or punishment signals
  
  However, there are many other possible sub-categories (or taxonomies) of machine learning techniques, such as
  
  * [**deep learning**](https://www.sciencedirect.com/science/article/pii/S0893608014002135) (i.e. the use of neural networks to approximate functions and related learning algorithms, such as gradient descent) or
  * [**probabilistic machine learning**](https://www.nature.com/articles/nature14541) (machine learning techniques that provide uncertainty estimation)
  * [**weakly supervised learning**](https://academic.oup.com/nsr/article/5/1/44/4093912) (i.e. SL where labeling information may not be completely accurate)
  * [**online learning**](https://www.cs.huji.ac.il/%7Eshais/papers/OLsurvey.pdf) (i.e. learning from a single data point at a time rather than from a dataset of multiple data points)
  
  These sub-categories can also be combined. For example, deep learning can be performed online or offline.
  
  ### Related fields
  
  There is also a related field known as **computational** (or **statistical**) **learning theory**, which concerned with the *theory of learning* (from a computational and statistical point of view). So, in this field, we are interested in questions like "How many samples do we need to approximately compute this function with a certain error?".
  
  Of course, given that machine learning is a set of algorithms and techniques that are data- or experience-driven, one may wonder what the difference between machine learning and **statistics** is. In fact, in many cases, they are very similar and ML adopts many statistical concepts, and you may even read on the web that machine learning is just *glorified statistics*. ML and statistics often tackle the same problem, but from a different perspective or with slightly different approaches (and the terminology may slightly change from one field to the other). If you are interested in a more detailed explanation of their difference, you could read [Statistics versus machine learning](https://www.nature.com/articles/nmeth.4642) (2018) by Danilo Bzdok et al.
  
  ### What is machine learning good for?
  
  ML can potentially be used to (at least partially) automate tasks that involve data and pattern recognition, which were previously performed only by humans (e.g. translation from one human language, such as English, to another, such as Italian). However, machine learning cannot automate all tasks: for example, it cannot infer causal relations from the data (which often must be done by humans), unless you include **causal inference** as part of machine learning. If you are interested in causal inference, you could take a look at the paper [Causal Inference](http://proceedings.mlr.press/v6/pearl10a/pearl10a.pdf) by Judea Pearl ([Turing Award for his work in causal inference!](https://amturing.acm.org/award_winners/pearl_2658896.cfm)).
  
  > **Comments:**
  > **username_3**: With regards to the definition of Mitchell, a traditional spam filter can also learn from experience? Suppose that at some time $t$ the emails that arrived to the traditional spam filter are so hard to classify for it (its hard-wired rules don't match the underlying patterns). Now if at some time $t'$ the emails that arrive are easy to classify them, then its performance has improved. Can we say that it has learned from experience? I would argue not, since the improvement was not due to experience (it was just fed with easy to classify emails).  
  > **username_3**: However, it is still very difficult how one should apply Mitchell's definition. What "improves with experience mean"? Does it mean "with more data"?  
  > **username_1**: @adosar Essentially, yes, experience is data. If you're familiar with RL and, in particular, DQN, you have heard of the term _experience replay_, which is kind of a data structure that stores the data the agent has observed during the interaction with the environment.  
  
  ---
  
  ### Answer by username_2 on 2020-04-21. Score: 3
  
  Here's a definition by Tom Mitchel (1997):
  
  > Computer program is said to learn from experience E with respect to
  > some task T and some performance measure P, if its performance on T,
  > as measured by P, improves with experience E.
  
  So, the programmer gives some instructions/rules to the computer, so that it can learn how to solve the problem from the given examples by themselves.
  
  In some tasks, the computer can perform better than humans. For example, the Dota 2 bot (made by OpenAI) defeated the world champion.
  
  With machine learning, many cases can be automated. They also have the ability to improve solutions by learning the given data from time to time. It can process and analyze large data well.
  
  Machine learning already applied in many fields such as machine translation in google translate and face recognition that is widely used by today's society for security.
  
  
  ---
  
  