Sorry It’s a bit difficult to explain. Thanks for your answer.
There are 3 models

Keras
Pytorch (using weight_decay argument in torch.optim to add regularizer)
Pytorch (creating function to calculate L2 norm of weights along with coefficient to add regularizer)

All 3 models showed the same result most of the time except when it printed
Gap of output with builtin pytorch  18.459202 (model  1 vs model 2)
Gap of output with manual pytorch  18.459202 (model  1 vs model 3)

which showed the gap between keras and pytorch.
However when the program ends, I still have those variables in the workspace (as I worked from IDE) so I printed out the value and the gap disappeared (exact same variable as above). It goes from 18.459202 to 4.091505e-06 as below.
In[410]: np.average(np.abs(keras_output-builtin_model.forward(torch.from_numpy(X_train).type(torch.FloatTensor)).data.numpy()))
Out[410]: 4.091505e-06

There’s no code in between these steps though since the program will end after it print out a gap.