I’m seeing similar behavior. I ran your script from both PyTorch built from source and the Conda installation:
built from source:
Allocation  6.476427316665649
Torch Blas  4.414772272109985
Numpy Blas  2.665677547454834
Torch cuBlas  0.14812421798706055
Final 13.705262184143066

conda:
Allocation  5.521166086196899
Torch Blas  39.35049605369568
Numpy Blas  39.40145969390869
Final 84.42150139808655

It looks like something is wrong with Conda.
A minor note: Your script only measures cuBlas launch time. Not execution time. You need a torch.cuda.synchronize() call to measure execution time