text: Title: The provided `peft_type` 'PROMPT_TUNING' is not compatible with the `PeftMixedModel`.
  Body: ### Feature request
  
  PROMPT_TUNING is an useful adapter and it would be great if we can combine it with LORA.
  
  ### Motivation
  
  Lots of finetunes on consumer grade hardware leverage lora. It would be great we can mix prompt tuning with lora as plug and play.
  
  ### Your contribution
  
  I would like to submit a PR if there is interest.