Answers.Body: <p>In general terms yes. Because what the ML algorithms do in general is to learn the hidden probability density function of the target examples (cats, dogs..). And that is done by learning the conditional probability function between inputs, <span class="math-container">$X$</span>, and target outputs, <span class="math-container">$y$</span>, for discriminative models or by learning the joint probability function for generative models.</p>
  <p>More on discriminative VS generative models and their relation with probabilities in <a href="https://towardsdatascience.com/introduction-to-generative-and-discriminative-models-9c9ef152b9af#:%7E:text=Generative%20models%20are%20a%20wide,P(y%7Cx)." rel="nofollow noreferrer">this nice article</a> from where I took the picture.
  <a href="https://i.sstatic.net/INixU.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/INixU.png" alt="enter image description here" /></a></p>
  <p>It also helps to think that given a set of different examples (say the image of cats). What the ML algorithm really learns is the invariant features between all the examples, i.e. the cats. So the ML algorithm learns to discriminate all the variability (backgrounds, occluding objects) and learn what is invariant: the representation of a cat.</p>
  <p>This representation, since images are pixels, can be described as a probability density function of values that would represent a cat. It would be like the ML algorithm said: <em>&quot;If those pixels values behave close to the probability density function I learned, then those pixels must represent a cat!&quot;</em></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-27T13:48:39.883Z
Answers.Text: You say "... hidden probability density function", but maybe you mean "... **unknown** probability density function"? Because $p(y \mid x)$ is typically unknown. Hidden refers to something different. For example, the hidden space in VAEs or hidden variable in Hidden Markov models.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-29T15:54:27.070Z
Answers.Text: You are right, thinking it through I think we could say: “unknown but existent”, like although it is unknown, we know it is there
Answers.UserDisplayName: JVGD
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-26T10:19:43.723Z
Answers.LastActivityDate: 2022-01-14T23:56:25.590Z
Answers.LastEditDate: 2022-01-14T23:56:25.590Z
Answers.UserDisplayName: JVGD
Body: <p>Consider the following statements from <a href="https://www.deeplearningbook.org/contents/ml.html" rel="nofollow noreferrer">Chapter 5: Machine Learning Basics</a> from the book titled <a href="https://www.deeplearningbook.org/" rel="nofollow noreferrer">Deep Learning</a> (by Aaron Courville et al.)</p>
  <blockquote>
  <p>Machine learning tasks are usually described in terms of how the
  machine learning system should process an example. An example is a
  collection of <strong>features</strong> that have been quantitatively measured from
  some object or event that we want the machine learning system to
  process. We typically represent an example as a vector <span class="math-container">$\mathbf{x} \in
   \mathbb{R}^n$</span> where <strong>each entry <span class="math-container">$x_i$</span> of the vector is another feature</strong>. For example,
  the features of an image are usually the values of the pixels in the image.</p>
  </blockquote>
  <p>Here, an example is described as a collection of features, which are real numbers. In probability theory, a random variable is also a real-valued function.</p>
  <p>Can I always interpret features in machine learning as random variables or are there any exceptions for this interpretation?</p>
  
Tags: <machine-learning><comparison><features><random-variable>
Title: Can I always interpret features as random variables in machine learning safely?
UserDisplayName: hanugm
ThreadText: # Can I always interpret features as random variables in machine learning safely?
  
  (Asked by: username_0 on 2021-08-26)
  
  Consider the following statements from [Chapter 5: Machine Learning Basics](https://www.deeplearningbook.org/contents/ml.html) from the book titled [Deep Learning](https://www.deeplearningbook.org/) (by Aaron Courville et al.)
  
  > Machine learning tasks are usually described in terms of how the
  > machine learning system should process an example. An example is a
  > collection of **features** that have been quantitatively measured from
  > some object or event that we want the machine learning system to
  > process. We typically represent an example as a vector $\mathbf{x} \in
  > \mathbb{R}^n$ where **each entry $x\_i$ of the vector is another feature**. For example,
  > the features of an image are usually the values of the pixels in the image.
  
  Here, an example is described as a collection of features, which are real numbers. In probability theory, a random variable is also a real-valued function.
  
  Can I always interpret features in machine learning as random variables or are there any exceptions for this interpretation?
  
  **Tags:** machine-learning, comparison, features, random-variable
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-08-26. Score: 2 | Accepted answer
  
  In general terms yes. Because what the ML algorithms do in general is to learn the hidden probability density function of the target examples (cats, dogs..). And that is done by learning the conditional probability function between inputs, $X$, and target outputs, $y$, for discriminative models or by learning the joint probability function for generative models.
  
  More on discriminative VS generative models and their relation with probabilities in [this nice article](https://towardsdatascience.com/introduction-to-generative-and-discriminative-models-9c9ef152b9af#:%7E:text=Generative%20models%20are%20a%20wide,P(y%7Cx).) from where I took the picture.
  [![enter image description here](https://i.sstatic.net/INixU.png)](https://i.sstatic.net/INixU.png)
  
  It also helps to think that given a set of different examples (say the image of cats). What the ML algorithm really learns is the invariant features between all the examples, i.e. the cats. So the ML algorithm learns to discriminate all the variability (backgrounds, occluding objects) and learn what is invariant: the representation of a cat.
  
  This representation, since images are pixels, can be described as a probability density function of values that would represent a cat. It would be like the ML algorithm said: *"If those pixels values behave close to the probability density function I learned, then those pixels must represent a cat!"*
  
  > **Comments:**
  > **username_2**: You say "... hidden probability density function", but maybe you mean "... **unknown** probability density function"? Because $p(y \mid x)$ is typically unknown. Hidden refers to something different. For example, the hidden space in VAEs or hidden variable in Hidden Markov models.  
  > **username_1**: You are right, thinking it through I think we could say: “unknown but existent”, like although it is unknown, we know it is there  
  
  ---
  
  