Body: <p>I just started learning about AI and have been reading a book called &quot;Foundations of Machine Learning&quot; by Mehryar Mohri so that I can try to create my own. I had a question come up recently: Can I create a machine learning algorithm that can reasonably solve high dimensional problems?</p>
  <p>For example, say I want to find a local maximum within a specified range for <span class="math-container">$Y$</span>, but <span class="math-container">$Y$</span> is a function of <span class="math-container">$x_1,...x_{30}$</span> functions. And I call <span class="math-container">$x_i$</span> a function because it relies on all other <span class="math-container">$x$</span> functions such that for <span class="math-container">$x_{i=n}$</span>, this is a function of all <span class="math-container">$x_{i \neq n}$</span>. So there are 30 dimensions that the algo can alter and each variable is also a function of the other.</p>
  <p>I looked up online about dimensional issues with AI and found a pretty good, simple <a href="https://towardsdatascience.com/curse-of-dimensionality-a-curse-to-machine-learning-c122ee33bfeb" rel="nofollow noreferrer">article</a> outlining why AI accuracy decreases with dimensionality increasing. The article was written a couple of years ago though and I wanted to see if anyone knew if research since then has found a machine learning method to get around this problem. If not, what are ways to say interpolate or minimize the error, besides a ton of training data or what the article recommends: dimensionality reduction prior to training?</p>
  <p>PS - Please delve into the applied analysis that underlies whatever answer you may have!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-08T10:13:47.490Z
Comments.Text: "Solve" is kinda a harsh word here, but if it is "approximate", then I belive we have "kinda" solve it. For an image, it is reasonable to say that each pixel is one dimension, so a 224x244 image is already 50k dimensions. Recent advances in approximating the Image->Class function is pretty good.
Comments.UserDisplayName: Minh-Long Luu
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-09T12:47:29.137Z
Comments.Text: @Minh-LongLuu I can see that since each pixel is itself a function of the color map or RGB value. However, that is a more simple example since each pixel technically isn't dependent on each other when it's fed to an algo (I would think). But also, perhaps some are. Like the art AIs I've seen I would think that in order for them to even remotely work, they would need to take into account both the local (subset of pisxels) and global (total set of pixels in image) pixel RGB values to produce something coherent.
Comments.UserDisplayName: Sophia
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-03-09T15:55:03.620Z
Comments.Text: Assuming pixels have neighbor relationship is something we...assume. There is possibly no proof of it yet
Comments.UserDisplayName: Minh-Long Luu
Tags: <dimensionality-reduction><curse-of-dimensionality>
Title: Dimensionality Limitations
UserDisplayName: Sophia
ThreadText: # Dimensionality Limitations
  
  (Asked by: username_0 on 2023-03-08)
  
  I just started learning about AI and have been reading a book called "Foundations of Machine Learning" by Mehryar Mohri so that I can try to create my own. I had a question come up recently: Can I create a machine learning algorithm that can reasonably solve high dimensional problems?
  
  For example, say I want to find a local maximum within a specified range for $Y$, but $Y$ is a function of $x\_1,...x\_{30}$ functions. And I call $x\_i$ a function because it relies on all other $x$ functions such that for $x\_{i=n}$, this is a function of all $x\_{i \neq n}$. So there are 30 dimensions that the algo can alter and each variable is also a function of the other.
  
  I looked up online about dimensional issues with AI and found a pretty good, simple [article](https://towardsdatascience.com/curse-of-dimensionality-a-curse-to-machine-learning-c122ee33bfeb) outlining why AI accuracy decreases with dimensionality increasing. The article was written a couple of years ago though and I wanted to see if anyone knew if research since then has found a machine learning method to get around this problem. If not, what are ways to say interpolate or minimize the error, besides a ton of training data or what the article recommends: dimensionality reduction prior to training?
  
  PS - Please delve into the applied analysis that underlies whatever answer you may have!
  
  **Tags:** dimensionality-reduction, curse-of-dimensionality
  > **Comments:**
  > **username_1**: "Solve" is kinda a harsh word here, but if it is "approximate", then I belive we have "kinda" solve it. For an image, it is reasonable to say that each pixel is one dimension, so a 224x244 image is already 50k dimensions. Recent advances in approximating the Image->Class function is pretty good.  
  > **username_0**: @Minh-LongLuu I can see that since each pixel is itself a function of the color map or RGB value. However, that is a more simple example since each pixel technically isn't dependent on each other when it's fed to an algo (I would think). But also, perhaps some are. Like the art AIs I've seen I would think that in order for them to even remotely work, they would need to take into account both the local (subset of pisxels) and global (total set of pixels in image) pixel RGB values to produce something coherent.  
  > **username_1**: Assuming pixels have neighbor relationship is something we...assume. There is possibly no proof of it yet  
  
  ---
  
  ## 0 answers
  
  