Paper

A Stochastic Block-coordinate Proximal Newton Method for Nonconvex Composite Minimization

Zhu, Hong, Qian, Xun

2024

Overview

We propose a stochastic block-coordinate proximal Newton method for minimizing the sum of a blockwise Lipschitz-continuously differentiable function and a separable nonsmooth convex function. In each iteration, this method randomly selects a block and approximately solves a strongly convex regularized quadratic subproblem, utilizing second-order information from the smooth component of the objective function. A backtracking line search is employed to ensure the monotonicity of the objective value. We demonstrate that under certain sampling assumption, the fundamental convergence results of our proposed stochastic method are in accordance with the corresponding results for the inexact proximal Newton method. We study the convergence of the sequence of expected objective values and the convergence of the sequence of expected residual mapping norms under various sampling assumptions. Furthermore, we introduce a method that employs the unit step size in conjunction with the Lipschitz constant of the gradient of the smooth component to formulate the strongly convex regularized quadratic subproblem. In addition to establishing the global convergence rate, we also provide a local convergence analysis for this method under certain sampling assumption and the higher-order metric subregularity of the residual mapping. To the best knowledge of the authors, this is the first stochastic second-order algorithm with a superlinear local convergence rate for addressing nonconvex composite optimization problems. Finally, we conduct numerical experiments to demonstrate the effectiveness and convergence of the proposed algorithm.

• Publisher: Cornell University Library, arXiv.org
• Subject: Algorithms / Convergence / Mapping / Newton methods / Optimization / Sampling