I’m not sure, how your model works.
Your forward method seems to use model sharding, i.e. different parts of the model are located on different devices.
However, you are never transferring the inputs to GPU1, since you are not reassigning it in this line of code:
inputs.to('cuda:1')

Also, based on your code you are only calling model.to('cuda:0'), which would mean everything runs on GPU0.
However, it seems that at least your labels are on GPU1, which should create an error in your loss calculation.
Are code parts missing or am I missing something?