Thanks for the reply.
Two things here:

One is I don’t get the speedup as you get with mkldnn. Infact it is taking more time with mkldnn.
For this :

python -m timeit --setup="import torch; from torch.utils import mkldnn as mkldnn_utils; net = torch.nn.Linear(1000, 2); net = mkldnn_utils.to_mkldnn(net); batch = torch.rand(16, 1000); batch = batch.to_mkldnn()" "net(batch)"


Secondly I get the following error whenever there I try to to use mkldnn.to_mkldnn(pretrained_model) such as resnet18 or resnet50

File "/opt/dev/envs/deep_learning/lib/python3.5/site-packages/torchvision/models/resnet.py", line 161, in forward
    x = x.view(x.size(0), -1)
RuntimeError: Currently Mkldnn tensor does not support view. Change to use reshape instead

I guess I should build PyTorch from source inorder to get speedup as you say.