56293658742896592925:

self.pos_emb = nn.Parameter(torch.zeros(1, config.block_size, config.n_embd, device=self.device))


Background:
To autograd, .to is computation, so you didn’t assign a parameter to self.pos_emb but an intermediate tensor.