Transformers, Pytorch, Machine Learning, Deep Dives, Data Science.
loss_fn = torch.nn.CrossEntropyLoss(ignore_index=PAD_IDX) optimizer = torch.optim.Adam(model.parameters(), lr=0.001, betas=(0.9, 0.98), eps=1e-9) # Save history to dictionnary history = { 'train_loss': [], 'eval_loss': [], 'train_acc': [], 'eval_acc': [] } # Main loop for epoch in range(1, 4):