Hello Anil!



 ayadav01:

My question is should i calculate weights for each batch or per dataset.


I prefer calculating the weight based on the entire dataset,
rather than on a per-batch basis, although it shouldn’t matter
if your batches are of reasonable size.  The point is that the
weights aren’t magic numbers that have to be just right.  They
are approximate numbers that are used to (partially) account
for having a significantly unbalanced dataset.


Also, assuming my label per batch is [0, 0, 1, 0, 1] and weight_0 is 0.4 and weight_1 is 0.6, so does the weight tensor passed to nn.BCEWithLogitsLoss will be [0.4, 0.4, 0.6, 0.4, 0.6] ?


This would be reasonable, but the disadvantage is that you would
have to construct a new instance of BCEWithLogitsLoss for
every batch, because your weight tensor depends on the batch.
I am assuming here that by “weight tensor” you mean the tensor
you pass into BCEWithLogitsLoss's constructor as its named
weight argument, e.g.:
criterion = torch.nn.BCEWithLogitsLoss (weight = my_weight_tensor)
loss = criterion (preds, targets)

It is more convenient to use the named pos_weight constructor
argument:
criterion = torch.nn.BCEWithLogitsLoss (pos_weight = torch.FloatTensor ([count_of_lbl0 / count_of_lbl1]))

You can now use the same criterion loss object, constructed
once, over and over again for each batch.
I would also note that if your relative weights are, in fact, 0.4
and 0.6, your dataset isn’t really very unbalanced, and I probably
wouldn’t bother using weights in the loss function.


My input data shape is : 1 x 52 x 52 x 52 (3D Volume) … and i am using a batch size of 5. So, at each epoch, input is 5 x 1 x 52 x 52 x 52 and label is 1 x 5.


As an aside, your shapes look a little confused.  I assume that
“5 x 1 x 52 x 52 x 52” is the shape of the input to your model
(not your loss function).  The shape of your label is probably [5]
(although it could be [5, 1]), but a shape of [1, 5] would be
wrong.
Best.
K. Frank