The dropout layer doesn’t have trainable parameters, but besides that the code looks alright.
Since your model is training with the “standard” approach, could you use your per-parameter optimizer with the same learning rate for all parameters, as you’ve used in the working approach?
Assuming your model is training fine afterwards, the currently specified learning rates might be bad for your new approach.