Explore the vast range of titles available.

The Levenberg–Marquardt method is a fundamental regularization technique for the Newton method applied to nonlinear equations, possibly constrained, and possibly with singular or even nonisolated solutions. We review the literature on the subject, in particular relating to each other various convergence frameworks and results. In this process, the analysis is performed from a unified perspective, and some new results are obtained as well. We discuss smooth and piecewise smooth equations, inexact solution of subproblems, and globalization techniques. Attention is also paid to the LP-Newton method, because of its relations to the Levenberg–Marquardt method.

We present a new theoretical analysis of local superlinear convergence of classical quasi-Newton methods from the convex Broyden class. As a result, we obtain a significant improvement in the currently known estimates of the convergence rates for these methods. In particular, we show that the corresponding rate of the Broyden–Fletcher–Goldfarb–Shanno method depends only on the product of the dimensionality of the problem and the logarithm of its condition number.

We consider a class of difference-of-convex (DC) optimization problems where the objective function is the sum of a smooth function and a possibly nonsmooth DC function. The application of proximal DC algorithms to address this problem class is well-known. In this paper, we combine a proximal DC algorithm with an inexact proximal Newton-type method to propose an inexact proximal DC Newton-type method. We demonstrate global convergence properties of the proposed method. In addition, we give a memoryless quasi-Newton matrix for scaled proximal mappings and consider a two-dimensional system of semi-smooth equations that arise in calculating scaled proximal mappings. To efficiently obtain the scaled proximal mappings, we adopt a semi-smooth Newton method to inexactly solve the system. Finally, we present some numerical experiments to investigate the efficiency of the proposed method, which show that the proposed method outperforms existing methods.

We propose a BFGS method with Wolfe line searches for unconstrained multiobjective optimization problems. The algorithm is well defined even for general nonconvex problems. Global convergence and R-linear convergence to a Pareto optimal point are established for strongly convex problems. In the local convergence analysis, if the objective functions are locally strongly convex with Lipschitz continuous Hessians, the rate of convergence is Q-superlinear. In this respect, our method exactly mimics the classical BFGS method for single-criterion optimization.

In federated learning, client models are often trained on local training sets that vary in size and distribution. Such statistical heterogeneity in training data leads to performance variations across local models. Even within a model, some parameter estimates can be more reliable than others. Most existing FL approaches (such as FedAvg), however, do not explicitly address such variations in client parameter estimates and treat all local parameters with equal importance in the model aggregation. This disregard of varying evidential credence among client models often leads to slow convergence and a sensitive global model. We address this gap by proposing an aggregation mechanism based upon the Hessian matrix. Further, by making use of the first-order information of the loss function, we can use the Hessian as a scaling matrix in a manner akin to that employed in Quasi-Newton methods. This treatment captures the impact of data quality variations across local models. Experiments show that our method is superior to the baselines of Federated Average (FedAvg), FedProx, Federated Curvature (FedCurv) and Federated Newton Learn (FedNL) for image classification on MNIST, Fashion-MNIST, and CIFAR-10 datasets when the client models are trained using statistically heterogeneous data.

The sparse group Lasso is a widely used statistical model which encourages the sparsity both on a group and within the group level. In this paper, we develop an efficient augmented Lagrangian method for large-scale non-overlapping sparse group Lasso problems with each subproblem being solved by a superlinearly convergent inexact semismooth Newton method. Theoretically, we prove that, if the penalty parameter is chosen sufficiently large, the augmented Lagrangian method converges globally at an arbitrarily fast linear rate for the primal iterative sequence, the dual infeasibility, and the duality gap of the primal and dual objective functions. Computationally, we derive explicitly the generalized Jacobian of the proximal mapping associated with the sparse group Lasso regularizer and exploit fully the underlying second order sparsity through the semismooth Newton method. The efficiency and robustness of our proposed algorithm are demonstrated by numerical experiments on both the synthetic and real data sets.

In this paper, we consider the composite optimization problems over the Stiefel manifold. A successful method to solve this class of problems is the proximal gradient method proposed by Chen et al. (SIAM J Optim 30:210–239, 2020. https://doi.org/10.1137/18M122457X ). Motivated by the proximal Newton-type techniques in the Euclidean space, we present a Riemannian proximal quasi-Newton method, named ManPQN, to solve the composite optimization problems. The global convergence of the ManPQN method is proved and iteration complexity for obtaining an ϵ -stationary point is analyzed. Under some mild conditions, we also establish the local linear convergence result of the ManPQN method. Numerical results are encouraging, which shows that the proximal quasi-Newton technique can be used to accelerate the proximal gradient method.

We investigate the problem of optimal transport in the so-called Kantorovich form, i.e. given two Radon measures on two compact sets, we seek an optimal transport plan which is another Radon measure on the product of the sets that has these two measures as marginals and minimizes a certain cost function. We consider quadratic regularization of the problem, which forces the optimal transport plan to be a square integrable function rather than a Radon measure. We derive the dual problem and show strong duality and existence of primal and dual solutions to the regularized problem. Then we derive two algorithms to solve the dual problem of the regularized problem: A Gauss–Seidel method and a semismooth quasi-Newton method and investigate both methods numerically. Our experiments show that the methods perform well even for small regularization parameters. Quadratic regularization is of interest since the resulting optimal transport plans are sparse, i.e. they have a small support (which is not the case for the often used entropic regularization where the optimal transport plan always has full measure).

This paper considers the parameter estimation problems of photovoltaic cell models. In order to overcome the complexity of the model structure, through applying the hierarchical identification principle and decomposing the photovoltaic cell model into two sub-models with a smaller number of parameters. The nonlinear identification model becomes a combination of a linear sub-model and a nonlinear sub-model. A two-stage gradient-based iterative and a two-stage Newton iterative algorithms are proposed to estimate the parameters of photovoltaic cell models by using the negative gradient search and the Newton method. The performance of the proposed algorithms is assessed by using the simulation from the experimental data, and the evaluation results test the effectiveness of the proposed algorithms. In particular, the model built by using the obtained parameter estimates can fit the I-V curve, the P-V curve and the maximum power point well.

This study explores a novel signal detection method for a stress-strain sensing system based on the principle of Brillouin backscattering. Currently, signal detection methods in this field face challenges such as high computational load, low accuracy, and poor consistency. This paper proposes the use of the Adaptive Gradient Descent Algorithm (Adam) for fitting calculations on Brillouin scattering signals, and it establishes a Brillouin strain measurement system for heterodyne coherent detection. Experimental results indicate that the strain error after applying the Adam method to fit the scattering spectrum curve is 27.60, representing only 59% of the strain error obtained using the Gauss-Newton method, while the error from the Levenberg-Marquardt method is even higher. Additionally, the signal processing time of the Adam algorithm is 18.5 ms, outperforming the other two methods. This approach holds significant practical value for enhancing the precision of data feature extraction in Brillouin scattering sensing technology.