Journal Article

Riemannian stochastic fixed point optimization algorithm

Sakai, Hiroyuki, Iiduka, Hideaki

2022

Overview

This paper considers a stochastic optimization problem over the fixed point sets of quasinonexpansive mappings on Riemannian manifolds. The problem enables us to consider Riemannian hierarchical optimization problems over complicated sets, such as the intersection of many closed convex sets, the set of all minimizers of a nonsmooth convex function, and the intersection of sublevel sets of nonsmooth convex functions. We focus on adaptive learning rate optimization algorithms, which adapt step-sizes (referred to as learning rates in the machine learning field) to find optimal solutions quickly. We then propose a Riemannian stochastic fixed point optimization algorithm, which combines fixed point approximation methods on Riemannian manifolds with the adaptive learning rate optimization algorithms. We also give convergence analyses of the proposed algorithm for nonsmooth convex and smooth nonconvex optimization. The analysis results indicate that, with small constant step-sizes, the proposed algorithm approximates a solution to the problem. Consideration of the case in which step-size sequences are diminishing demonstrates that the proposed algorithm solves the problem with a guaranteed convergence rate. This paper also provides numerical comparisons that demonstrate the effectiveness of the proposed algorithms with formulas based on the adaptive learning rate optimization algorithms, such as Adam and AMSGrad.

• Publisher: Springer US,Springer Nature B.V
• Subject: Adaptive learning / Algebra / Algorithms / Computer Science / Convergence / Convex analysis / Convexity / Euclidean space / Fixed points (mathematics) / Intersections / Machine learning / Natural language processing / Numeric Computing / Numerical Analysis / Optimization / Optimization algorithms / Original Paper / Random variables / Riemann manifold / Theory of Computation