text: Title: Independently process LoRa adapters and base model
  Body: Hello, 
  
  As I understand, during inference, the current library merges the base model weights with the learned LoRa adapters to avoid increasing in latency, as in h=(W+BA)x. However, I'm wondering if there's a way to process the base model and the learned adapters independently, as in h = Wx+BAx. The Wx computation is easy to implement (just load the original base model and perform inference) but I'm stuck on how to perform the BAx computation. 
  
  For my particular use case, I would like to benchmark the runtime of the BAx computations relative to Wx on the Whisper model.
  
  Thank you for your help!