One point that I need to mention is that instead of writing codes for all sections then running model to find bugs, build small parts of code, such as only one layer or just one Dataset/DataLoader and try to interact with them for arbitrary inputs using basic function in python/PyTorch. For instance, you need to be able to extract only 1 batch of images and show them or play with lmost anything in your code. Literally, you must be able to understand the mechanism of each module you use such as model.train(), model.eval(), etc.