Answers.Body: <p>I think, that <span class="math-container">$\mathbb{R}^{D}$</span> is most natural choice in practical situations since many kinds of data can be described in a this way:</p>
  <ul>
  <li>Image is 2d array <span class="math-container">$H \times W$</span> with each pixel taking value in <span class="math-container">$\mathbb{R}^{c}$</span> (say, <span class="math-container">$c=3$</span>) or a continuous subset of <span class="math-container">$\mathbb{R}^{c} = [-1, 1]$</span></li>
  <li>In the sequence modeling problems one gives an embedding vector to each token in some <span class="math-container">$\mathbb{R}^{k}$</span>. There is no reason priori to put constraints on the embedding vector (probably one may want to clip norm of these vectors)</li>
  </ul>
  <p>However, for the applications in physics, or any other fields, by the virtue of the task arbitrary input in <span class="math-container">$\mathbb{R}^{D}$</span> may not make sense, and one can be restricted to consider functions only on a certain manifold <span class="math-container">$M$</span> (sphere <span class="math-container">$S^d$</span>, <a href="https://arxiv.org/abs/1805.09112" rel="nofollow noreferrer">hyperbolic spaces</a>)</p>
  <p>In applications of <em>Quantum Machine Learning</em> input is a vector in a Hilbert space - system of quits, whatever.
  You may be interested to consult <a href="https://www.nature.com/articles/nature23474" rel="nofollow noreferrer">this review</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-11-06T05:28:46.527Z
Answers.LastActivityDate: 2021-11-06T05:28:46.527Z
Answers.UserDisplayName: spiridon_the_sun_rotator
Body: <p>Consider the following paragraph from the chapter named <a href="https://mml-book.github.io/book/mml-book.pdf#page=145" rel="nofollow noreferrer">Vector Calculus</a> from the textbook titled <strong><a href="https://mml-book.github.io/book/mml-book.pdf" rel="nofollow noreferrer">Mathematics for Machine Learning</a></strong> by <em>Marc Peter Deisenroth et al.</em></p>
  <blockquote>
  <p>Central to this chapter is the concept of a function. A function <span class="math-container">$f$</span>
  is a quantity that relates two quantities to each other. In this book,
  these quantities are typically inputs <span class="math-container">$x \in \mathbb{R}^D$</span> and targets
  (function values) <span class="math-container">$f(x)$</span>, which we assume are real-valued if not
  stated otherwise. <strong>Here <span class="math-container">$\mathbb{R}^D$</span> is the domain of <span class="math-container">$f$</span></strong>, and the
  function values <span class="math-container">$f(x)$</span> are the image/codomain of <span class="math-container">$f$</span>.</p>
  </blockquote>
  <p>we can notice that the textbook is taking <span class="math-container">$\mathbb{R}^D$</span> as the domain for objective functions. I want to know whether it is valid in general cases.</p>
  <p>Do the objective functions that we generally use in artificial intelligence have <span class="math-container">$\mathbb{R}^D$</span> as the domain?</p>
  <p>I am guessing it would <em>not</em> be since the loss functions are generally defined on the datasets which can also have discrete attributes and hence the objective function cannot be defined on every point in <span class="math-container">$\mathbb{R}^D$</span>. So, I am guessing that the correct form of the bold statement from the quoted paragraph is &quot;<strong>Here the domain of <span class="math-container">$f$</span> should be a subset of <span class="math-container">$\mathbb{R}^D$</span></strong>&quot; if we are intended to deal with a general case. Am I correct or is there any arrangement such as defining <span class="math-container">$f$</span> as zero where the function is not defined?</p>
  
Tags: <math><objective-functions>
Title: Are the domains of objective functions in AI always equals to $\mathbb{R}^D$ or subset of it?
UserDisplayName: hanugm
ThreadText: # Are the domains of objective functions in AI always equals to $\mathbb{R}^D$ or subset of it?
  
  (Asked by: username_0 on 2021-11-05)
  
  Consider the following paragraph from the chapter named [Vector Calculus](https://mml-book.github.io/book/mml-book.pdf#page=145) from the textbook titled **[Mathematics for Machine Learning](https://mml-book.github.io/book/mml-book.pdf)** by *Marc Peter Deisenroth et al.*
  
  > Central to this chapter is the concept of a function. A function $f$
  > is a quantity that relates two quantities to each other. In this book,
  > these quantities are typically inputs $x \in \mathbb{R}^D$ and targets
  > (function values) $f(x)$, which we assume are real-valued if not
  > stated otherwise. **Here $\mathbb{R}^D$ is the domain of $f$**, and the
  > function values $f(x)$ are the image/codomain of $f$.
  
  we can notice that the textbook is taking $\mathbb{R}^D$ as the domain for objective functions. I want to know whether it is valid in general cases.
  
  Do the objective functions that we generally use in artificial intelligence have $\mathbb{R}^D$ as the domain?
  
  I am guessing it would *not* be since the loss functions are generally defined on the datasets which can also have discrete attributes and hence the objective function cannot be defined on every point in $\mathbb{R}^D$. So, I am guessing that the correct form of the bold statement from the quoted paragraph is "**Here the domain of $f$ should be a subset of $\mathbb{R}^D$**" if we are intended to deal with a general case. Am I correct or is there any arrangement such as defining $f$ as zero where the function is not defined?
  
  **Tags:** math, objective-functions
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-11-06. Score: 1
  
  I think, that $\mathbb{R}^{D}$ is most natural choice in practical situations since many kinds of data can be described in a this way:
  
  * Image is 2d array $H \times W$ with each pixel taking value in $\mathbb{R}^{c}$ (say, $c=3$) or a continuous subset of $\mathbb{R}^{c} = [-1, 1]$
  * In the sequence modeling problems one gives an embedding vector to each token in some $\mathbb{R}^{k}$. There is no reason priori to put constraints on the embedding vector (probably one may want to clip norm of these vectors)
  
  However, for the applications in physics, or any other fields, by the virtue of the task arbitrary input in $\mathbb{R}^{D}$ may not make sense, and one can be restricted to consider functions only on a certain manifold $M$ (sphere $S^d$, [hyperbolic spaces](https://arxiv.org/abs/1805.09112))
  
  In applications of *Quantum Machine Learning* input is a vector in a Hilbert space - system of quits, whatever.
  You may be interested to consult [this review](https://www.nature.com/articles/nature23474).
  
  
  ---
  
  