Hi @aman_goyal ,
Here is the full code snippet about how I tried.
import torch
from collections import OrderedDict

# load the model
model = torch.load('qdtrack-frcnn_r50_fpn_12e_bdd100k-13328aed.pth')

# separate backbone layers
new_state_dict = {}
for name, param in model['state_dict'].items():
    if 'backbone' in name:
        new_state_dict[name] = param

new_state_dict = OrderedDict(new_state_dict)

# construct new model with backbone layers
new_model = {
    'meta': model['meta'],
    'state_dict': new_state_dict,
}

# save the new model which has only backbone layers
torch.save(new_model, 'backbone.pth')

If this works let me know.