You could use different optimizers for each input type and call it w.r.t to the currently used input.
The use case won’t work if you are working with a batch of images containing multiple types of inputs and you would have to define a “default” optimizer.