I see, your thinking is actually quite interesting. So as I understand you believe that the loss is not that a problem but rather the network architecture.
Actually to give some more details regarding the architecture that I am using I am working on 3D deep analysis and point clouds. Imagine that each value that you see on the graph corresponds to an intensity value for each point in the 3D space, and these intensity values are what I am trying to regress. For example check on the image below:
image1062×734 182 KB
The red points are emitting elements and then I am trying to regress the effect of this emission to each point in the space respectively. As you can see the higher error is in the points closer to the emitting elements due to the smaller distance.
Currently the model that I am using is the RSConv implementation from torch-points3d with a “unet” architecture and the corresponding config file. The model initially is used for segmentation but I am trying to use it for regression. It indeed includes some MLPs and my batch size is quite relatively small imagine that for 400 point clouds my batch-size each time is 12 (maximum point clouds I can load in my gpu with the current size of 7000 points for each point cloud). Trying to downsample my point clouds and increase the batch size could help though.
Scaling/normalizing my values in the range 0-1 (initially my values can vary from 0-46000) is something I’ve tried but it doesn’t seem to work. At the end in the normalization the discripancy gap remains, so for example values that were before in the range of 0-1000 will go to the range 0-0.02 with the majority being in the 3 digits precision i.e. 0.00x so I am not sure whether this could help.