You won’t be able to load the state_dict anymore, as it would create shape mismatches.
However, you could keep the original model (with the larger parameter set), load the state_dict, and reduce the number of parameters later.
The main question would be: how would you like to reduce the number of filters using the pre-trained model:

remove half of the filters (lower, higher, random index?)
reduce the filters somehow (mean, median, sum?)
…