Hafizur_Rahman:

Number of images I used: Perfect # 405 and Defected # 385.


This is quite a tiny dataset, so I would expect that your model might be overfitting to the training set, as generalizing from 405 images might be quite tricky.
Besides the obvious advice to collect more data, you could try to increase the regularization and add more augmentation to the mages.