Increasing the depth of a model does not necessarily lead to the enhancement of accuracy. By increasing the model parameters, you are increasing the model capacity to learning more complex patterns. This often leads to a phenomenon called overfitting, which is not desirable. It is still vague how you want to increase the number of layers. Do you want to add more convolution layers? or more fully connected layers? where do you exactly want to add them?I suggest you to spend some time reading this introduction to PyTorch DEEP LEARNING WITH PYTORCH: A 60 MINUTE BLITZ.