I managed to get it working with the following:
self.backbone = models.resnet50(pretrained=False)
first_layer = torch.nn.Conv2d(12, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
self.backbone2 = nn.Sequential()
self.backbone2.add_module(name="conv1", module=first_layer)
[self.backbone2.add_module(name, child) for name, child in self.backbone.named_children() if name != "conv1"]
self.backbone = self.backbone2

nn.Sequential() has a add_module() function that takes a name and the module as parameter which we can use to get the same names as the original architecture.
Hope that helps (after 2 years)