Answers.Body: <p>Your professor is wrong (or maybe you misunderstood what he wanted to say, or he did not explain correctly what he wanted to say). It may be a good idea to ask your professor for some clarification and tell him about the info given in this answer.</p>
  <p>Probability theory is widely used to model problems in machine learning, including in the context of today's neural networks. There are many examples of research papers or books where you will see probabilities and probability distributions. For example, you can take a look at <a href="https://arxiv.org/pdf/1409.0473.pdf" rel="nofollow noreferrer">the neural machine translation paper</a>.</p>
  <p>More importantly, we often formulate the problem of learning (in the context of neural networks) as a minimization of an objective function, which is equivalent to the maximization of a likelihood of the parameters (given the data). It's also often the case that neural networks produce a <a href="https://en.wikipedia.org/wiki/Probability_vector" rel="nofollow noreferrer">probability vector</a> (from the so-called <em>logits</em>, often, by using a softmax function).</p>
  <p>There are many other (maybe clearer) examples of the use of probability theory to model problems in machine learning. For example, you can take a look at <a href="http://proceedings.mlr.press/v37/blundell15.pdf" rel="nofollow noreferrer">Bayesian neural networks</a>, <a href="https://arxiv.org/pdf/1312.6114v10.pdf" rel="nofollow noreferrer">variational auto-encoders</a>, or <a href="https://papers.nips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf" rel="nofollow noreferrer">generative adversarial networks</a>. You will see a lot of probabilities and probability distributions in these linked papers.</p>
  <p>You may also be interested in <a href="https://ai.stackexchange.com/a/12354/2444">this answer</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-11-13T14:28:49.937Z
Answers.LastActivityDate: 2021-11-13T14:36:13.703Z
Answers.LastEditDate: 2021-11-13T14:36:13.703Z
Answers.UserDisplayName: nbro
Body: <p>In the paper <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.335.3398&amp;rep=rep1&amp;type=pdf" rel="nofollow noreferrer"><em>The Perceptron: A probabilistic model for information storage and organization in the brain</em></a>, Rosenblatt used the probability theory to model his perceptron.</p>
  <p>My professor told me that today's neural networks are not modeled with probability theory anymore. Why is that?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-11-22T01:25:59.597Z
Comments.Text: I appreciate this query as an undergrad math major doing a project on probability theory in neural machine translations.
Comments.UserDisplayName: Christian Evans
Tags: <neural-networks><history><perceptron><probability-theory>
Title: Why are today's neural networks not modeled with probability theory?
UserDisplayName: Collo
ThreadText: # Why are today's neural networks not modeled with probability theory?
  
  (Asked by: username_0 on 2021-11-13)
  
  In the paper [*The Perceptron: A probabilistic model for information storage and organization in the brain*](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.335.3398&rep=rep1&type=pdf), Rosenblatt used the probability theory to model his perceptron.
  
  My professor told me that today's neural networks are not modeled with probability theory anymore. Why is that?
  
  **Tags:** neural-networks, history, perceptron, probability-theory
  > **Comments:**
  > **username_1**: I appreciate this query as an undergrad math major doing a project on probability theory in neural machine translations.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-11-13. Score: 1
  
  Your professor is wrong (or maybe you misunderstood what he wanted to say, or he did not explain correctly what he wanted to say). It may be a good idea to ask your professor for some clarification and tell him about the info given in this answer.
  
  Probability theory is widely used to model problems in machine learning, including in the context of today's neural networks. There are many examples of research papers or books where you will see probabilities and probability distributions. For example, you can take a look at [the neural machine translation paper](https://arxiv.org/pdf/1409.0473.pdf).
  
  More importantly, we often formulate the problem of learning (in the context of neural networks) as a minimization of an objective function, which is equivalent to the maximization of a likelihood of the parameters (given the data). It's also often the case that neural networks produce a [probability vector](https://en.wikipedia.org/wiki/Probability_vector) (from the so-called *logits*, often, by using a softmax function).
  
  There are many other (maybe clearer) examples of the use of probability theory to model problems in machine learning. For example, you can take a look at [Bayesian neural networks](http://proceedings.mlr.press/v37/blundell15.pdf), [variational auto-encoders](https://arxiv.org/pdf/1312.6114v10.pdf), or [generative adversarial networks](https://papers.nips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf). You will see a lot of probabilities and probability distributions in these linked papers.
  
  You may also be interested in [this answer](https://ai.stackexchange.com/a/12354/2444).
  
  
  ---
  
  