Dimension reduction using space filling curve to avoid "Curse of dimensionality"? In machine learning, we want to train a model. While training, if the dimension of data is high, we have a problem (Curse of Dimensionality), so we want to reduce the dimension of our data.
Since we know $\mathbb{R}^n$ and $\mathbb{R}$ have the same cardinality. So we always have some space-filling curve that maps each point uniquely in both directions. i.e. we can always bijectively map any n-dimensional data to 1-dimension. So what is the problem we have in the first place?
I come up with two problems that can still be there:

*

*With this space-filling curve map, we can't reduce the size of data. i.e., we have to increase the precision when we are writing it in 1-dimension.

*This is the place where I have doubts. I am thinking that while representing data in $\mathbb{R}^n$ we have more information than representing it in $\mathbb{R}$. There is some structure when we write data in $\mathbb{R}^n$ like which point is near to which point. That is lost in this map (space-filling curve), i.e, the map is not homomorphic.

My question:

*

*Is it right what I am thinking?

*I don't know what information I am talking about in point 2. could you help me make it more rigorous is.

*Is there any other problem?

Example:
Suppose we have a training data with $x^i \in \mathbb{R}^n $ with label $y^i \in \mathbb{R}$  where $i \in \{1,2,..,N\}$. When we are training a neural network to fit this data. If we change the order of the bases i.e.
$$x^i = (x_1^i,x_2^i,..,x_n^i)$$
if we take
$$\tilde{x}^i = (x_{\rho(1)}^i,x_{\rho(2)}^i,..,x_{\rho(n)}^i)$$
where $\rho$ is some permutation function(same for all $i$). then the training of neural network doesn't affect. So when order doesn't matter. What if I transform all the data from $\mathbb{R}^n$ to $\mathbb{R}$? It should also don't matter. But it did matter. Otherwise, there is nothing like the curse of dimensions.
I think When we transform the data from $\mathbb{R}^n$ to $\mathbb{R}$, we lose some information or do something wrong. What is that?

This approach does work, to a degree, in low-dimensional $\mathbb{R}^n$ spaces, certainly in $\mathbb{R}^2$ and $\mathbb{R}^3$. A $k$-d tree is basically a data structure for ordering points along a space-filling curve.
The common reason why/when tree-based techniques work is because they turn a decision among $\mathcal{O}(m)$ choices into a sequence of $\mathcal{O}(\log m)$ decisions among $\mathcal{O}(n)$ choices. That's useful as long as $n\ll m$, but it becomes utterly counterproductive when $n$ is similar or bigger than $m$.
Oftentimes, the spaces for which the curse of dimensionality applies are not just high-dimensional, but conceptually infinite-dimensional, which makes a direct space-filling approach hopeless. Nevertheless, I wouldn't say it's completely futile to work in that direction, because even in such problems the data is in reality often confined to a finite-, perhaps even low-dimensional submanifold, and subdividing only that manifold in $k$-d fashion might well have its merits. The real problem is finding that manifold in the first place; that's what dimensionality reduction is all about.
