Adding to what @ptrblck said … u can set a threshold value on scores predicted by the model and subsequently filter the corresponding masks … e-g
    outputs = model(image)
    outputs = [{k: v.to(cpu_device) for k, v in t.items()} for t in outputs]
    pred_score = list(outputs[0]['scores'].numpy())
    pred_t = [pred_score.index(x) for x in pred_score if x>=threshold][-1]
    masks = (outputs[0]['masks']>0.5).squeeze().numpy()
    masks = masks[:pred_t+1]