Transformers, Pytorch, Machine Learning, Deep Dives, Data Science.
for x, y in tqdm(loader, position=0, leave=True): logits = model(x, y[:, :-1]) loss = loss_fn(logits.contiguous().view(-1, model.vocab_size), y[:, 1:].contiguous().view(-1)) losses += loss.item() preds = logits.argmax(dim=-1) masked_pred = preds * (y[:, 1:]!=PAD_IDX) accuracy = (masked_pred == y[:,