Hi @ssetu,
I think the best way to achieve that would be to create a Module inheriting from FasterRCNN (minus the box_predictor) with a list of your fine-tuned box_predictors, define a custom forward with an extra “detection scenario” argument where you would switch on the right box_predictor, depending on that extra argument.