nn.LSTM uses the optimized CUDNN LSTM kernel by default, which avoids all the overhead of building a graph that contains hundreds of separate mathematical operations and the overhead of launching those operations as separate kernels. You’ll get the same slow performance by setting torch.backends.cudnn.enabled=False.