albanD:

What is the loss function you’re using? Do you modify any Tensor or state inplace during that computation?


Thank you for your reply.
I am using cross entropy loss in both cases (nn.CrossEntropyLoss()).
I don’t modify any states of the model, I don’t update any tensors either.
My evaluation script is pretty standard. with torch.no_grad(): model.eval() …
For your reference, this is pretty much my code:

  
      github.com
  
  
    vfdev-5/FixMatch-pytorch/blob/master/main_fixmatch.py#L105 1

y_pred_cat = model(x_cat)
y_pred_cat = deinterleave(y_pred_cat, le)

idx1 = len(x)
idx2 = idx1 + len(weak_x)
y_pred = y_pred_cat[:idx1, ...]
y_weak_preds = y_pred_cat[idx1:idx2, ...]  # logits_weak
y_strong_preds = y_pred_cat[idx2:, ...]  # logits_strong

# supervised learning:
sup_loss = sup_criterion(y_pred, y)

# unsupervised learning:
y_weak_probas = torch.softmax(y_weak_preds, dim=1).detach()
y_pseudo = y_weak_probas.argmax(dim=1)
max_y_weak_probas, _ = y_weak_probas.max(dim=1)
unsup_loss_mask = (
    max_y_weak_probas >= engine.state.confidence_threshold
).float()
unsup_loss = (
    unsup_criterion(y_strong_preds, y_pseudo) * unsup_loss_mask



  
  
    
    
  
  


there are a lot of functions which are not present in that script (called from elsewhere) but you can assume all losses/criteria are xent
when I flip the sup_loss with unsup_loss, results change.