<p><a href="https://towardsdatascience.com/why-neural-nets-can-approximate-any-function-a878768502f0" rel="nofollow noreferrer">Neural Networks can approximate any function</a>. Quoting the essence in case the article is removed in the future.</p>
<blockquote>
<p>The key to neural networks’ ability to approximate any function is that they incorporate non-linearity into their architecture. Each layer is associated with an activation function that applies a non-linear transformation to the output of that layer.</p>
</blockquote>
<p>So no, knowing the class of the probability density function is not required to approximate it via Deep Learning. With a large enough number of samples you could construct an approximation.</p>

CC BY-SA 4.0
2021-08-28T15:14:21.553Z
2021-08-28T15:14:21.553Z
tnfru
<p>In deep learning, models may learn the probability distribution that generated the dataset. Observe the following paragraph from <a href="https://www.deeplearningbook.org/contents/ml.html" rel="nofollow noreferrer">Chapter 5: Machine Learning Basics</a> from the book titled <a href="https://www.deeplearningbook.org/" rel="nofollow noreferrer">Deep Learning</a> (by Aaron Courville et al.)</p>
<blockquote>
<p>Unsupervised learning algorithms experience a dataset containing many
features, then learn useful properties of the structure of this
dataset. <strong>In the context of deep learning, we usually want to learn the
entire probability distribution that generated a dataset, whether
explicitly, as in density estimation</strong>, or implicitly, for tasks like
synthesis or denoising. Some other unsupervised learning algorithms
perform other roles, like clustering, which consists of dividing the
dataset into clusters of similar examples.</p>
</blockquote>
<p>I read about density estimation in the same chapter, as given below</p>
<blockquote>
<p>In the density estimation problem, the machine learning algorithm is
asked to learn a function <span class="math-container">$p_{model} : R^n \rightarrow R$</span>, where
<span class="math-container">$p_{model}$</span>(x) <strong>can be interpreted as a probability density function
(if <span class="math-container">$x$</span> is continuous)</strong> or a probability mass function (if <span class="math-container">$x$</span> is
discrete) on the space that the examples were drawn from.</p>
</blockquote>
<p>This question is focused on explicit probability density estimation in continuous case i.e., learning density function <span class="math-container">$p_{model}$</span>  directly.</p>
<p>Suppose I have a dataset <span class="math-container">$D$</span> with <span class="math-container">$n$</span> continuous random variables (features) <span class="math-container">$X_1, X_2, X_3,\cdots, X_n$</span>. And I don't know anything about the probability density function of individual random variables. That is, I don't know about any information about any <span class="math-container">$X_i$</span> , such as, whether <span class="math-container">$X_i$</span> follows normal distribution or any other distribution. Then, is it possible to learn density function explicitly? Or do I need to provide some necessary information such as the class of probability distribution function to be learned?</p>
<p>I am thinking as follows:</p>
<p>If I have some information about <span class="math-container">$X_i$</span>, such as: <span class="math-container">$X_i$</span> falls to a well known distribution, then I can learn the parameters of the underlying density function from <span class="math-container">$D$</span>. So, is it mandatory to know some information about the underlying probability density function.</p>

<density-estimation>
Is knowing the class of probability density function mandatory for explicit density estimation?
hanugm
# Is knowing the class of probability density function mandatory for explicit density estimation?

(Asked by: username_0 on 2021-08-28)

In deep learning, models may learn the probability distribution that generated the dataset. Observe the following paragraph from [Chapter 5: Machine Learning Basics](https://www.deeplearningbook.org/contents/ml.html) from the book titled [Deep Learning](https://www.deeplearningbook.org/) (by Aaron Courville et al.)

> Unsupervised learning algorithms experience a dataset containing many
> features, then learn useful properties of the structure of this
> dataset. **In the context of deep learning, we usually want to learn the
> entire probability distribution that generated a dataset, whether
> explicitly, as in density estimation**, or implicitly, for tasks like
> synthesis or denoising. Some other unsupervised learning algorithms
> perform other roles, like clustering, which consists of dividing the
> dataset into clusters of similar examples.

I read about density estimation in the same chapter, as given below

> In the density estimation problem, the machine learning algorithm is
> asked to learn a function $p\_{model} : R^n \rightarrow R$, where
> $p\_{model}$(x) **can be interpreted as a probability density function
> (if $x$ is continuous)** or a probability mass function (if $x$ is
> discrete) on the space that the examples were drawn from.

This question is focused on explicit probability density estimation in continuous case i.e., learning density function $p\_{model}$ directly.

Suppose I have a dataset $D$ with $n$ continuous random variables (features) $X\_1, X\_2, X\_3,\cdots, X\_n$. And I don't know anything about the probability density function of individual random variables. That is, I don't know about any information about any $X\_i$ , such as, whether $X\_i$ follows normal distribution or any other distribution. Then, is it possible to learn density function explicitly? Or do I need to provide some necessary information such as the class of probability distribution function to be learned?

I am thinking as follows:

If I have some information about $X\_i$, such as: $X\_i$ falls to a well known distribution, then I can learn the parameters of the underlying density function from $D$. So, is it mandatory to know some information about the underlying probability density function.

**Tags:** density-estimation


---

## 1 answers

### Answer by username_1 on 2021-08-28. Score: 0

[Neural Networks can approximate any function](https://towardsdatascience.com/why-neural-nets-can-approximate-any-function-a878768502f0). Quoting the essence in case the article is removed in the future.

> The key to neural networks’ ability to approximate any function is that they incorporate non-linearity into their architecture. Each layer is associated with an activation function that applies a non-linear transformation to the output of that layer.

So no, knowing the class of the probability density function is not required to approximate it via Deep Learning. With a large enough number of samples you could construct an approximation.


---

