Thank you! Restarted runtime and voila it works.
It was interesting because I could run other timm models and load from timm, but DetrForObjectDetection.from_pretrained('facebook/detr-resnet-50') would throw the error.