Mixed Newton Method for Optimization in Complex Spaces
<|reference_start|>Mixed Newton Method for Optimization in Complex Spaces: In this paper, we modify and apply the recently introduced Mixed Newton Method, which is originally designed for minimizing real-valued functions of complex variables, to the minimization of real-valued functions of real variables by extending the functions to complex space. We show that arbitrary regularizations preserve the favorable local convergence properties of the method, and construct a special type of regularization used to prevent convergence to complex minima. We compare several variants of the method applied to training neural networks with real and complex parameters.<|reference_end|>
@article{yudin2024mixed,
    title={Mixed Newton Method for Optimization in Complex Spaces},
    author={Nikita Yudin, Roland Hildebrand, Sergey Bakhurin, Alexander Degtyarev,
  Anna Lisachenko, Ilya Kuruzov, Andrei Semenov, Mohammad Alkousa},
    journal={arXiv preprint arXiv:2407.20367},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.20367},
    primaryClass={math.OC cs.LG}
}