Thank you for your response. I have over-written compute_loss function as follows -
class CustomTrainer(Trainer):        
    def compute_loss(self, model, inputs, return_outputs=False):
        class_weights = torch.FloatTensor([1./5542, 1./36587]).cuda()
        labels = inputs.pop("labels")
        outputs = model(**inputs)
        logits = outputs.logits
        loss_fct = CrossEntropyLoss(weight=class_weights)
        #loss = loss_fct(outputs,labels)
        loss = loss_fct(logits.view(-1, self.model.config.num_labels),
                        labels.float().view(-1, self.model.config.num_labels))
        return (loss, outputs) if return_outputs else loss

Is it all that I need to change? because I get an ValueError: Expected input batch_size (16) to match target batch_size (8). How can I change shapes of output and labels here?