ttsesm:

I will have to apply the clustering to my whole training dataset (I guess I do not need to do that for the testing dataset) or only to each corresponding batch size points each time?


I would perform the clustering on the training dataset to reuse the same cluster indices for the input samples. If you use it on the batch-level the clusters would “move around”.
You wouldn’t need it for the test set, as the weighting should be only applied during training, if I understand the use case correctly.



 ttsesm:

Now do I need to assign these weights to each value separately or what?


Yes, now you would need to map the contiguous input features to each cluster and select the corresponding weight for this input feature. This weight can then be multiplied with the unreduced loss before calculating the gradients.