Explore the vast range of titles available.

In this paper, we present two Dai-Yuan type iterative methods for solving large-scale systems of nonlinear monotone equations. The methods can be considered as extensions of the classical Dai-Yuan conjugate gradient method for unconstrained optimization. By employing two different approaches, the Dai-Yuan method is modified to develop two different search directions, which are combined with the hyperplane projection technique of Solodov and Svaiter. The first search direction was obtained by carrying out eigenvalue study of the search direction matrix of an adaptive DY scheme, while the second is obtained by minimizing the distance between two adaptive versions of the DY method. Global convergence of the methods are established under mild conditions and preliminary numerical results show that the proposed methods are promising and more effective compared to some existing methods in the literature.

In this paper, we analyze the global convergence of a general non-monotone line search method on Riemannian manifolds. For this end, we introduce some properties for the tangent search directions that guarantee the convergence, to a stationary point, of this family of optimization methods under appropriate assumptions. A modified version of the non-monotone line search of Zhang and Hager is the chosen globalization strategy to determine the step-size at each iteration. In addition, we develop a new globally convergent Riemannian conjugate gradient method that satisfies the direction assumptions introduced in this work. Finally, some numerical experiments are performed in order to demonstrate the effectiveness of the new procedure.

Several modifications have been proposed to speed up the alternating least squares (ALS) method of fitting the PARAFAC model. The most widely used is line search, which extrapolates from linear trends in the parameter changes over prior iterations to estimate the parameter values that would be obtained after many additional ALS iterations. We propose some extensions of this approach that incorporate a more sophisticated extrapolation, using information on nonlinear trends in the parameters and changing all the parameter sets simultaneously. The new method, called \"enhanced line search (ELS),\" can be implemented at different levels of complexity, depending on how many different extrapolation parameters (for different modes) are jointly optimized during each iteration. We report some tests of the simplest parameter version, using simulated data. The performance of this lowest-level of ELS depends on the nature of the convergence difficulty. It significantly outperforms standard LS when there is a \"convergence bottleneck,\" a situation where some modes have almost collinear factors but others do not, but is somewhat less effective in classic \"swamp\" situations where factors are highly collinear in all modes. This is illustrated by examples. To demonstrate how ELS can be adapted to different N-way decompositions, we also apply it to a four-way array to perform a blind identification of an under-determined mixture (UDM). Since analysis of this dataset happens to involve a serious convergence \"bottleneck\" (collinear factors in two of the four modes), it provides another example of a situation in which ELS dramatically outperforms standard line search.

In this paper, a new nonmonotone line search rule is proposed,which is verified to be an improved version of the nonmonotone line search technique proposed by Zhang and Hager. Unlike the Zhang and Hager’s method, our nonmonotone line search is proved to own a nice property similar to the standard Armijo line search. In virtue of such a property, global convergence is established for the developed algorithm, where the search direction is supposed to satisfy some mild conditions and the stepsize is chosen by the new line search rule. R-linear convergence of the developed algorithm is proved for strongly convex objective functions. The developed algorithm is used to solve the test problems available in the CUTEr, the numerical results demonstrate that the new line search strategy outperforms the other similar ones.

The technique of nonmonotone line search has received many successful applications and extensions in nonlinear optimization. This paper provides some basic analyses of the nonmonotone line search. Specifically, we analyze the nonmonotone line search methods for general nonconvex functions along different lines. The analyses are helpful in establishing the global convergence of a nonmonotone line search method under weaker conditions on the search direction. We explore also the relations between nonmonotone line search and R-linear convergence assuming that the objective function is uniformly convex. In addition, by taking the inexact Newton method as an example, we observe a numerical drawback of the original nonmonotone line search and suggest a standard Armijo line search when the nonmonotone line search condition is not satisfied by the prior trial steplength. The numerical results show the usefulness of such suggestion for the inexact Newton method.

The line search techniques together with the Newton method are the best methods to solve nonlinear systems of equations. These methods use the gradient directions because they required low storage. In this paper, we suggest a new line search algorithm with gradient direction to solve the nonlinear systems of equations. The purpose of this algorithm is to reduce the number of iterations and the function evaluations, and it can increase the effectiveness of the approach. The algorithms global convergence is proved. The numerical results indicates the efficiency of the new algorithm and it is promised for solving nonlinear systems of equations.

The derivative-free projection methodology is important and highly efficient method to solve large scale monotone equations of nonlinear systems. In this work, we suggested a new class of extensions projection approach employs along with a new line search to show a class of new double projection technique for solving monotone systems of nonlinear equations. Our algorithm can be applied to solve nonsmooth equations, furthermore it's suitable for large scale equations due to simplicity and limited memory. This method constricts new two appropriate hyperplanes in each point strictly separates xk from the solution set, it can obtain the next iteration xk+1 by projecting xk onto the intersection of two halfspaces and include the solution set of the problem. The global convergence of the given method is investigated with mild assumptions. The numerical experiments prove that the new approach is working well and so promising.

We present global convergence rates for a line-search method which is based on random first-order models and directions whose quality is ensured only with certain probability. We show that in terms of the order of the accuracy, the evaluation complexity of such a method is the same as its counterparts that use deterministic accurate models; the use of probabilistic models only increases the complexity by a constant, which depends on the probability of the models being good. We particularize and improve these results in the convex and strongly convex case. We also analyze a probabilistic cubic regularization variant that allows approximate probabilistic second-order models and show improved complexity bounds compared to probabilistic first-order methods; again, as a function of the accuracy, the probabilistic cubic regularization bounds are of the same (optimal) order as for the deterministic case.

Based on the classic augmented Lagrangian multiplier method, we propose, analyze and test an algorithm for solving a class of equality-constrained non-smooth optimization problems (chiefly but not necessarily convex programs) with a particular structure. The algorithm effectively combines an alternating direction technique with a nonmonotone line search to minimize the augmented Lagrangian function at each iteration. We establish convergence for this algorithm, and apply it to solving problems in image reconstruction with total variation regularization. We present numerical results showing that the resulting solver, called TVAL3, is competitive with, and often outperforms, other state-of-the-art solvers in the field.

In this paper, we seek the conjugate gradient direction closest to the direction of the scaled memoryless BFGS method and propose a family of conjugate gradient methods for unconstrained optimization. An improved Wolfe line search is also proposed, which can avoid a numerical drawback of the original Wolfe line search and guarantee the global convergence of the conjugate gradient method under mild conditions. To accelerate the algorithm, we introduce adaptive restarts along negative gradients based on the extent to which the function approximates some quadratic function during previous iterations. Numerical experiments with the CUTEr collection show that the proposed algorithm is promising. [PUBLICATION ABSTRACT]