First of all, you don’t have to pass all the parameters when you are using the default value.
I hope you solved it by now but I suggest try loading the pre-trained weights for a dataset they trained on and not your own dataset and see if it works. Then if it was working fine, it’s probably because of the first issue I mentioned. Usually, the dimensions of feature maps will be defined using the size of your input to prevent hard-coding for different input sizes. But I think the output of your dataset have a different dimensions from the one used in pre-training. You can check it by creating an object from your dataset class and just printing the shape of a sample.