Hi,
Why BCE was used in
  
      github.com
  
  
    huggingface/transformers/blob/8e74eca7f2b3235f8d5340d66361ea656c67bac7/examples/research_projects/mm-imdb/run_mmimdb.py#L524

logger.info("Training/evaluation parameters %s", args)# Trainingif args.do_train:    train_dataset = load_examples(args, tokenizer, evaluate=False)    label_frequences = train_dataset.get_label_frequencies()    label_frequences = [label_frequences[l] for l in labels]    label_weights = (        torch.tensor(label_frequences, device=args.device, dtype=torch.float) / len(train_dataset)    ) ** -1    criterion = nn.BCEWithLogitsLoss(pos_weight=label_weights)    global_step, tr_loss = train(args, train_dataset, model, tokenizer, criterion)    logger.info(" global_step = %s, average loss = %s", global_step, tr_loss)# Saving best-practices: if you use defaults names for the model, you can reload it using from_pretrained()if args.do_train and (args.local_rank == -1 or torch.distributed.get_rank() == 0):    logger.info("Saving model checkpoint to %s", args.output_dir)    # Save a trained model, configuration and tokenizer using `save_pretrained()`.    # They can then be reloaded using `from_pretrained()`    model_to_save = (        model.module if hasattr(model, "module") else model



  
  
    
    
  
  


The labels seem not binary to me.
label_frquencies looks like
Counter({'Drama': 8415, 'Comedy': 5107, 'Romance': 3226, 'Thriller': 3110, 'Crime': 2287, 'Action': 2154, 'Adventure': 1609, 'Horror': 1603, 'Mystery': 1231, 'Sci-Fi': 1212, 'Documentary': 1195, 'Fantasy': 1162, 'Family': 975, 'War': 804, 'Biography': 772, 'History': 663, 'Music': 632, 'Animation': 586, 'Musical': 503, 'Western': 423, 'Sport': 379, 'Short': 281, 'Film-Noir': 202})

What may I miss? Thanks.