Answers.Body: <p>Manifold is basically a geometric object where every small region can be mapped to a euclidean space(means manifold is locally euclidean). Think of a donut, here any small region can  be mapped to a euclidean space shown in this image:</p>
  <p><a href="https://i.sstatic.net/aXdY9.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/aXdY9.png" alt="enter image description here" /></a></p>
  <p>In this above picture, <span class="math-container">$M$</span> is the manifold, <span class="math-container">$\phi_\alpha,\phi_\alpha$</span> is the mapping function, <span class="math-container">$U_\alpha, U_\beta$</span> is two open sets(small local regions). This donut is an example of a manifold. Similarly, we can think of circles, spheres, paraboloids, <span class="math-container">$\mathbb{R}^2$</span>, <span class="math-container">$\mathbb{R}^3$</span>, etc. as a manifold because they all satisfy the above criteria(they are locally euclidean).</p>
  <p>Now, the question is why we are interested in manifolds in machine learning. In many machine learning applications, the data we interpret is laying on a manifold or non-Euclidean domain. For example, in astrophysics the observational data often time lies on a spherical domain. If we want we perform convolution over this spherical manifold to extract features, we can't just apply 2D convolution since we have to take account of parallel transport, gauges, symmetries, etc. Similarly, we may want to perform convolution over more complex shapes like those figures to extract features.</p>
  <p><a href="https://i.sstatic.net/OsFEG.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/OsFEG.png" alt="enter image description here" /></a></p>
  <p>There are methods like <a href="https://arxiv.org/pdf/2003.05425.pdf" rel="nofollow noreferrer">guage equivalent mesh CNN</a>, <a href="https://arxiv.org/pdf/1501.06297.pdf" rel="nofollow noreferrer">geodesic CNN</a>, etc to deal with such kind of data distribution.</p>
  <p>Graphs also lie on a non-Euclidean domain since the distance between any two nodes is not a straight line we have to travel through the graph and count the number of edges to measure distance. There are many applications where data lies on a graph, for example, drug-drug interaction, community detection, molecule structure, friendship network, recommendation system, traffic forecasting, etc.</p>
  <p><a href="https://i.sstatic.net/1wleg.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/1wleg.png" alt="enter image description here" /></a></p>
  <p>To perform convolution over graphs we have methods like <a href="https://arxiv.org/pdf/1606.09375.pdf" rel="nofollow noreferrer">ChebNet</a>, <a href="https://arxiv.org/pdf/1706.02216.pdf" rel="nofollow noreferrer">GraphSAGE</a>, <a href="https://arxiv.org/pdf/1710.10903.pdf" rel="nofollow noreferrer">graph attention network</a>, etc.</p>
  <p><strong>Notes:</strong></p>
  <p><em>1) Parallel transport:</em>  One basic problem occurs when we try to compare two vectors of two different points over the same manifold is that those two vectors belong to different Euclidean spaces (see the first figure), thus we can not directly compare them. Parallel transport provides a mechanism to move vectors over a manifold and analysis them. But note that parallel transport depends on the path means the result of the parallel transport is path-dependent.</p>
  <p><em>2) Guage:</em>  Guage is like a measurement apparatus to specify the tangent vector on the tangent space of a manifold.</p>
  <p><a href="https://i.sstatic.net/3w41q.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/3w41q.png" alt="enter image description here" /></a></p>
  <p><strong>References:</strong></p>
  <ol>
  <li><p><a href="https://arxiv.org/pdf/2104.13478.pdf" rel="nofollow noreferrer">Geometric Deep Learning: Grids, Groups, Graphs, Geodesics</a></p>
  </li>
  <li><p><a href="https://geometricdeeplearning.com/" rel="nofollow noreferrer">To learn about geometric deep learning this is very good website, they also contain extraordinary video lectures on GDL</a></p>
  </li>
  </ol>
  <p>Note that: I intentionally skipped the rigorous mathematical definition of a manifold while trying to convey the underlying meaning. Please, let me know if you want to know more about open sets, closed sets, topological spaces, topological manifold, charts, atlas, etc.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-03T22:25:20.010Z
Answers.LastActivityDate: 2021-09-05T11:41:01.350Z
Answers.LastEditDate: 2021-09-05T11:41:01.350Z
Answers.UserDisplayName: Swakshar Deb
Answers.Body: <p>The definition is the same as in <em>Mathematics</em> and, I suppose, elsewhere:</p>
  <p>it is a topological space such that the vicinity of each point is <em>homeomorphic</em> to a disk in <span class="math-container">$\mathbb{R}^n$</span> (note, that dimension has to be the same for all points <span class="math-container">$x$</span>). This requirement is important, since not every imaginable geometric object satisfies this requirement:</p>
  <ul>
  <li>Sphere is a manifold, since one can draw a tangent plane in the vicinity of any point. It is &quot;locally flat&quot; and even we humans see Earth flat, since its radius of curvature is much greater than the visible distance (let the proponents of the flat Earth theory forgive me).</li>
  </ul>
  <p><a href="https://i.sstatic.net/NRodH.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/NRodH.png" alt="enter image description here" /></a></p>
  <ul>
  <li>Two intersecting lines are not a manifold, since for any point, except for the intersection we have a 1-dimensional space and for the intersection point, there are two non-collinear vectors belonging to them.</li>
  </ul>
  <p><a href="https://i.sstatic.net/DV31n.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/DV31n.png" alt="enter image description here" /></a></p>
  <p>Natural examples emerging in Machine Learning are images, videos, or arbitrary data. One usually treats, say, an image, as an object in the <span class="math-container">$\mathbb{R}^{H \times W \times 3}$</span>, where <span class="math-container">$H$</span> is the height, <span class="math-container">$W$</span> - width of the image, and <span class="math-container">$3$</span> - number of colors. But in fact, only a small subset of all objects in this high-dimensional space are real images, and they belong to some manifold of a lower dimension.</p>
  <p>It is a non-trivial question to tell what exactly the true dimensionality of data is. For MNIST, it is claimed that it is <span class="math-container">$3$</span> (instead of <span class="math-container">$28 \times 28 = 784$</span>).</p>
  <p>As a good material on this topic, I recommend <a href="https://www.youtube.com/watch?v=UjaoZE0GBpg&amp;t=2899s" rel="nofollow noreferrer">this lecture</a> from the recent workshop.</p>
  <p><a href="https://scikit-learn.org/stable/modules/manifold.html" rel="nofollow noreferrer">Scikit-learn has a nice exposition as well.</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-03T10:32:38.990Z
Answers.Text: I think the answer would be improved by a few examples and uses of manifolds in e.g. deep learning. For instance, the conceptual manifold of "natural images" within the space of all possible pixel values on a NxN grid.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-03T16:06:50.363Z
Answers.Text: For people not familiar with the concept of "homeomorphic" (i.e. most people that do not have a background in mathematics), maybe you should explain what it means to be "homeomorphic to $\mathbb{R}^n$". It may also be a good idea to explain what a _topological space_ is (how is it different from a set or a vector space?). Finally, it would be nice to have some intuition on why, for example, an image would be a "manifold" (and you probably mean the "space of all real images" and not "an image").
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-03T16:09:00.090Z
Answers.Text: You also provide a link to the scikit article on "manifold learning", but how is this different, for example, from "manifolds" in the context of "geometric deep learning" or "graph neural networks" (if you are familiar with these, of course)?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-03T16:41:51.607Z
Answers.Text: @nbro - you're right, I should have discussed the cases of learning on manifolds and the data distributions localized on manifolds separately. I'll update the answer.
Answers.UserDisplayName: spiridon_the_sun_rotator
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-03T22:33:21.563Z
Answers.Text: @nbro. I did not work with manifold learning but as far as I know, in manifold learning we want to learn the manifold underlying the data distribution but in geometric deep learning we extract features from the data where the data can also lie on the manifolds, graphs etc.
Answers.UserDisplayName: Swakshar Deb
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-03T09:22:47.470Z
Answers.LastActivityDate: 2021-09-03T15:49:32.920Z
Answers.LastEditDate: 2021-09-03T15:49:32.920Z
Answers.UserDisplayName: spiridon_the_sun_rotator
Body: <p>We come across the word &quot;manifold&quot; in artificial intelligence, especially in the domains where learning is done based on data instances.</p>
  <p>What is the formal definition for manifold?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-03T08:51:46.753Z
Comments.Text: https://en.wikipedia.org/wiki/Manifold
Comments.UserDisplayName: Criticize SE actions means ban
Tags: <machine-learning><definitions>
Title: What is the formal definition for manifold in artificial intelligence?
UserDisplayName: hanugm
ThreadText: # What is the formal definition for manifold in artificial intelligence?
  
  (Asked by: username_0 on 2021-09-03)
  
  We come across the word "manifold" in artificial intelligence, especially in the domains where learning is done based on data instances.
  
  What is the formal definition for manifold?
  
  **Tags:** machine-learning, definitions
  
  > **Comments:**
  > **username_1**: https://en.wikipedia.org/wiki/Manifold  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2021-09-03. Score: 6
  
  Manifold is basically a geometric object where every small region can be mapped to a euclidean space(means manifold is locally euclidean). Think of a donut, here any small region can be mapped to a euclidean space shown in this image:
  
  [![enter image description here](https://i.sstatic.net/aXdY9.png)](https://i.sstatic.net/aXdY9.png)
  
  In this above picture, $M$ is the manifold, $\phi\_\alpha,\phi\_\alpha$ is the mapping function, $U\_\alpha, U\_\beta$ is two open sets(small local regions). This donut is an example of a manifold. Similarly, we can think of circles, spheres, paraboloids, $\mathbb{R}^2$, $\mathbb{R}^3$, etc. as a manifold because they all satisfy the above criteria(they are locally euclidean).
  
  Now, the question is why we are interested in manifolds in machine learning. In many machine learning applications, the data we interpret is laying on a manifold or non-Euclidean domain. For example, in astrophysics the observational data often time lies on a spherical domain. If we want we perform convolution over this spherical manifold to extract features, we can't just apply 2D convolution since we have to take account of parallel transport, gauges, symmetries, etc. Similarly, we may want to perform convolution over more complex shapes like those figures to extract features.
  
  [![enter image description here](https://i.sstatic.net/OsFEG.png)](https://i.sstatic.net/OsFEG.png)
  
  There are methods like [guage equivalent mesh CNN](https://arxiv.org/pdf/2003.05425.pdf), [geodesic CNN](https://arxiv.org/pdf/1501.06297.pdf), etc to deal with such kind of data distribution.
  
  Graphs also lie on a non-Euclidean domain since the distance between any two nodes is not a straight line we have to travel through the graph and count the number of edges to measure distance. There are many applications where data lies on a graph, for example, drug-drug interaction, community detection, molecule structure, friendship network, recommendation system, traffic forecasting, etc.
  
  [![enter image description here](https://i.sstatic.net/1wleg.png)](https://i.sstatic.net/1wleg.png)
  
  To perform convolution over graphs we have methods like [ChebNet](https://arxiv.org/pdf/1606.09375.pdf), [GraphSAGE](https://arxiv.org/pdf/1706.02216.pdf), [graph attention network](https://arxiv.org/pdf/1710.10903.pdf), etc.
  
  **Notes:**
  
  *1) Parallel transport:* One basic problem occurs when we try to compare two vectors of two different points over the same manifold is that those two vectors belong to different Euclidean spaces (see the first figure), thus we can not directly compare them. Parallel transport provides a mechanism to move vectors over a manifold and analysis them. But note that parallel transport depends on the path means the result of the parallel transport is path-dependent.
  
  *2) Guage:* Guage is like a measurement apparatus to specify the tangent vector on the tangent space of a manifold.
  
  [![enter image description here](https://i.sstatic.net/3w41q.png)](https://i.sstatic.net/3w41q.png)
  
  **References:**
  
  1. [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics](https://arxiv.org/pdf/2104.13478.pdf)
  2. [To learn about geometric deep learning this is very good website, they also contain extraordinary video lectures on GDL](https://geometricdeeplearning.com/)
  
  Note that: I intentionally skipped the rigorous mathematical definition of a manifold while trying to convey the underlying meaning. Please, let me know if you want to know more about open sets, closed sets, topological spaces, topological manifold, charts, atlas, etc.
  
  
  ---
  
  ### Answer by username_3 on 2021-09-03. Score: 4
  
  The definition is the same as in *Mathematics* and, I suppose, elsewhere:
  
  it is a topological space such that the vicinity of each point is *homeomorphic* to a disk in $\mathbb{R}^n$ (note, that dimension has to be the same for all points $x$). This requirement is important, since not every imaginable geometric object satisfies this requirement:
  
  * Sphere is a manifold, since one can draw a tangent plane in the vicinity of any point. It is "locally flat" and even we humans see Earth flat, since its radius of curvature is much greater than the visible distance (let the proponents of the flat Earth theory forgive me).
  
  [![enter image description here](https://i.sstatic.net/NRodH.png)](https://i.sstatic.net/NRodH.png)
  
  * Two intersecting lines are not a manifold, since for any point, except for the intersection we have a 1-dimensional space and for the intersection point, there are two non-collinear vectors belonging to them.
  
  [![enter image description here](https://i.sstatic.net/DV31n.png)](https://i.sstatic.net/DV31n.png)
  
  Natural examples emerging in Machine Learning are images, videos, or arbitrary data. One usually treats, say, an image, as an object in the $\mathbb{R}^{H \times W \times 3}$, where $H$ is the height, $W$ - width of the image, and $3$ - number of colors. But in fact, only a small subset of all objects in this high-dimensional space are real images, and they belong to some manifold of a lower dimension.
  
  It is a non-trivial question to tell what exactly the true dimensionality of data is. For MNIST, it is claimed that it is $3$ (instead of $28 \times 28 = 784$).
  
  As a good material on this topic, I recommend [this lecture](https://www.youtube.com/watch?v=UjaoZE0GBpg&t=2899s) from the recent workshop.
  
  [Scikit-learn has a nice exposition as well.](https://scikit-learn.org/stable/modules/manifold.html)
  
  > **Comments:**
  > **username_4**: I think the answer would be improved by a few examples and uses of manifolds in e.g. deep learning. For instance, the conceptual manifold of "natural images" within the space of all possible pixel values on a NxN grid.  
  > **username_5**: For people not familiar with the concept of "homeomorphic" (i.e. most people that do not have a background in mathematics), maybe you should explain what it means to be "homeomorphic to $\mathbb{R}^n$". It may also be a good idea to explain what a _topological space_ is (how is it different from a set or a vector space?). Finally, it would be nice to have some intuition on why, for example, an image would be a "manifold" (and you probably mean the "space of all real images" and not "an image").  
  > **username_5**: You also provide a link to the scikit article on "manifold learning", but how is this different, for example, from "manifolds" in the context of "geometric deep learning" or "graph neural networks" (if you are familiar with these, of course)?  
  > **username_3**: @username_5 - you're right, I should have discussed the cases of learning on manifolds and the data distributions localized on manifolds separately. I'll update the answer.  
  > **username_2**: @username_5. I did not work with manifold learning but as far as I know, in manifold learning we want to learn the manifold underlying the data distribution but in geometric deep learning we extract features from the data where the data can also lie on the manifolds, graphs etc.  
  
  ---
  
  