So I made the following changes and now it seems to work:
checkpoint = torch.load(checkpoint_path, map_location=device) 
model = Model()
model = nn.DataParallel(model, device_ids=device_ids) 
model.to(self.device) 
print(model.module.conv1.weight[0]) # shows random weights

model.load_state_dict(checkpoint['model_state'], strict=False)
print(checkpoint['model_state']['module.conv1.weight'][0]) # shows pretrained weights of checkpoint

print(model.module.conv1.weight[0]) # shows pretrained weights in model

The problem was that I had to initialise my model as a DataParallel object and additionally call .module when accessing the weights of a specific layer.