I guess backbone parameters are being updated somehow even when it’s in eval model and torch.no_grad is used.
Because this is failing -
timm_backbone = timm.create_model('resnet50', num_classes=0).eval()

assert model.backbone(x) == timm_backbone(x)