You can delete key-value pairs from an OrderedDict using del as seen here:
d = OrderedDict()
d['a'] = 1
d['b'] = 2
print(d)
del d['a']
print(d)

However, @googlebot already posted a solution where you would copy some of the pretrained parameters into the state_dict.