Solved by ptrblck in post #4 
              
              
                This approach would pass the same param_mat to all optimizers, which would update it with its gradient and I assume that’s the use case you are looking for.