Hi again @ptrblck
I have been thinking about this answer and I’m confused about a few situations where this solution could potentially be problematic.
Consider a situation where a few samples have a weight of 0. That is, intuitively, those samples (or observations or entries) are meaningless.
loss = loss * sample_weight could result in a loss of 0 for those particular samples. Hence during gradient computation, wouldn’t the network technically be looking at a loss of 0 (perfect classification / regression) for samples which we weren’t confident to begin with? Do you think backprop on a loss of 0 could result in network crashes?
Furthermore, wouldn’t sample_weight.requires_grad be true? Do you think it could create complications during backprop?
I’ve been doing something similar and my network almost always crashes mid training.