banikr:

And are there ways to optimize weights?


Yes, it seems to be possible. I found this thread which explains how you can learn the weights for the cross-entropy loss: Is that possible to train the weights in CrossEntropyLoss? 1.1k
Good luck!