@ptrblck Thank you very much I added:
      wrapped_model.to(device=device)
      for encoder in wrapped_model.prompt_encoders:
            encoder.to(device=device)

Interestingly, when there was a single encoder or a list of encoders including one encoder, I didn’t need to explicitly put it on device, but for the list of encoders i must! I wonder why?! Probably because in the case of single encoder, I put it on device in the first iteration of the loop.