I think a lot of boilerplate code could be reduced by using a higher-level API on top of PyTorch such as Ignite, Catalyst, or PyTorch Lightning. These libraries provide a different level of abstraction and might still fit your use case. If I’m not mistaken, you have still the full control with your training loop etc.
For the "same" padding implementation: since the padding values depend on the input shape, you could use a helper function, which calculates the padding and uses the functional API. @rwightman implemented an approach here 1 for his MedianPool2d layer.


Could you explain what the benefits in version control would be and what makes the most trouble for you using the dynamic approach in PyTorch?


If you are seeing issues in TensorBoard, feel free to create an issue on GitHub, so that we can track and fix it. Personally, I use visdom 4, which might not have the full functionality of TensorBoard, but worked better for me.