Not only pytorch, take a look at this IBM Benchmark 8 of various high performance scientific libraries on the LU factorization problem, as you would see from the results, the performance on small sized sectors can be highly misleading, you really should take a look at https://github.com/soumith/convnet-benchmarks 11 for a more comprehensive benchmark of deep learning frameworks