Dimension reduction using space filling curve to avoid "Curse of dimensionality"? In machine learning, we want to train a model. While training, if the dimension of data is high, we have a problem (Curse of Dimensionality), so we want to reduce the dimension of our data.
Since we know $\mathbb{R}^n$ and $\mathbb{R}$ have the same cardinality. So we always have some space-filling curve that maps each point uniquely in both directions. i.e. we can always bijectively map any n-dimensional data to 1-dimension. So what is the problem we have in the first place?
I come up with two problems that can still be there:

*

*With this space-filling curve map, we can't reduce the size of data. i.e., we have to increase the precision when we are writing it in 1-dimension.

*This is the place where I have doubts. I am thinking that while representing data in $\mathbb{R}^n$ we have more information than representing it in $\mathbb{R}$. There is some structure when we write data in $\mathbb{R}^n$ like which point is near to which point. That is lost in this map (space-filling curve), i.e, the map is not homomorphic.

My question:

*

*Is it right what I am thinking?

*I don't know what information I am talking about in point 2. could you help me make it more rigorous is.

*Is there any other problem?

Example:
Suppose we have a training data with $x^i \in \mathbb{R}^n $ with label $y^i \in \mathbb{R}$  where $i \in \{1,2,..,N\}$. When we are training a neural network to fit this data. If we change the order of the bases i.e.
$$x^i = (x_1^i,x_2^i,..,x_n^i)$$
if we take
$$\tilde{x}^i = (x_{\rho(1)}^i,x_{\rho(2)}^i,..,x_{\rho(n)}^i)$$
where $\rho$ is some permutation function(same for all $i$). then the training of neural network doesn't affect. So when order doesn't matter. What if I transform all the data from $\mathbb{R}^n$ to $\mathbb{R}$? It should also don't matter. But it did matter. Otherwise, there is nothing like the curse of dimensions.
I think When we transform the data from $\mathbb{R}^n$ to $\mathbb{R}$, we lose some information or do something wrong. What is that?

If you transform the data by some dimensional reduction then you will be affecting the distance metrics that you will be using implicitly in the higher dimensional training.
Try creating a 3d nearness metric from a 2d image - it's all distorted (see Fr Ted: "small, far away").
If you believe that all the N dimensions are useful, you could simply 'clump' the data based on clump size (a nearest neighbours metric over a reduced dimension, or a scaling matrix (eigenvalue/vectors), and either train each clump independently, or train against the clumps, depending on how the split happens/occurs/forced.
I.e. something along the lines of the RVM/IVM approaches What is the difference between Informative (IVM) and Relevance (RVM) vector machines
