Transformers, Pytorch, Machine Learning, Deep Dives, Data Science.
and targets are indeed reversed # s, t = next(iter(dataloader_train)) # print(s[:4, ...]) # print(t[:4, ...]) # print(s.size()) # Initialize model parameters for p in model.parameters(): if p.dim() > 1: nn.init.xavier_uniform_(p) # Define loss function : we ignore logits which are padding tokens