ttsesm:

I see, your thinking is actually quite interesting. So as I understand you believe that the loss is not that a problem but rather the network architecture.


Huber loss or some similar step balancing method may still be necessary, but yea, I said “MLP” but meant deep networks in general; and more complex architectures are more prone to introduce bias.



 ttsesm:

batch-size each time is 12 (maximum point clouds I can load in my gpu


Note that you can do gradient accumulation (i.e. do backward() without optimizer.step()), to increase effective batch size. Scheduled increasing may be optimal for faster convergence.


Scaling/normalizing my values in the range 0-1


Scaling is not ideal here, as you’re using symmetric losses with positive only values, good thing about it is that it makes optimization paths shorter. So, log transform is the simplest correcting (and variance stabilizing) transformation for positive values, not necessarily the best one though…