@ptrblck - Thanks!  By lossless I mean a model can be round-tripped gpu-cpu-gpu and resume training as though the round trip did not take place - the weights are exactly the same, the optimizer state is the same, and nothing needs to be set up again.  I’m not sure what you mean by moving between devices is differentiable, but it sounds like the optimizer should continue working after a round trip.
Putting together all of your suggestions - the train/inference/train loop could look something like this in simplified code:
# different models, but with compatible weights
model = ...
inference_model = ...
optimizer = optim.AdamW(model.parameters(), ...)

while True:

    model.to(device)
    model.train()
    
    for (data, target) in loader:
        data, target = data.to(device), target.to(device)
        optimizer.zero_grad()
        logits = model(data)
        loss = loss_func(logits, target)
        loss.backward()
        optimizer.step()
        
    model.to('cpu')

    inference_model.load_state_dict(copy.deepcopy(model.state_dict())) # is deepcopy needed here?
    inference_model.half() # doesn't affect original model's weights
    inference_model.to(device)
    inference_model.eval()
    
    with torch.no_grad():
        for (data, target) in loader:
            data, target = data.to(device), target.to(device)
            logits = inference_model(data)

    inference_model.to('cpu') # maybe not necessary, just need to unload it, don't need a copy of the weights

Does that look about right?
Main things I’m not sure about
(a) Really don’t need to do anything with the optimizer, it would “just work”?   I  now realize in typical code the optimizer is created with model.parameters() before model.to(device), so it must work across moves   Are there any implications of doing a lot of moves vs just one?
(b) Copy state dict from one model to another like shown?