Although the model architecture looks relatively nonstandard (pure convolution, extensive use of 5x5 convolutions, no residual layers), the overfitting might be attributable to the dataset. What is the extent of the training and validation set here, and is the data distribution comparable? For example, are the road conditions similar, orientation of camera comparable, etc.?
If collecting more data is not an option, there are various data augmentations that might be applicable in this setting:
https://pytorch.org/vision/stable/transforms.html 1