Wow, solved! I checked the Efficientnet-pytorch source code and suddendly found I should turn the model to eval() mode.
Now the predictions are same!!
model = efn.EfficientNet.from_name('efficientnet-b0')
model = model.to(device)
_ = model.eval()  ## hey, look here!!

img = torch.ones((2, 3, 680, 680))*0.5
img = img.to(device)
preds1 = model(img)
preds2 = model(img)
preds3 = model(img)

print(preds1[0][0])
print(preds2[0][0])
print(preds3[0][0])