For anyone who wants to copy a model (model1) and its optimizer (optimizer1) without knowing beforehand which type of optimizer it is, I use:
model2 = copy.deepcopy(model1)
optimizer2 = type(optimizer1)(model2.parameters())
optimizer2.load_state_dict(optimizer1.state_dict())