Explore the vast range of titles available.

This paper presents a layer-wise training procedure of neural networks by minimizing a Variance-Invariance-Covariance Regularization (VICReg) loss at each layer. The procedure is beneficial when annotated data are scarce but enough unlabeled data are present. Being able to update the parameters locally at each layer also handles problems such as vanishing gradient and initialization sensitivity in backpropagation. The procedure utilizes two forward passes instead of one forward and one backward pass as done in backpropagation, where one forward pass works on original data and the other on an augmented version of the data. It is shown that this procedure can construct more compact but informative spaces progressively at each layer. The architecture of the model is selected to be pyramidal, enabling effective feature extraction. In addition, we optimize weights for variance, invariance, and covariance terms of the loss function so that the model can capture higher-level semantic information optimally. After training the model, we assess its learned representations by measuring clustering quality metrics and performance on classification tasks utilizing a few labeled data. To evaluate the proposed approach, we do several experiments with different datasets: MNIST, EMNIST, Fashion MNIST, and CIFAR-100. The experimental results show that the training procedure enhances the classification accuracy of Deep Neural Networks (DNNs) trained on MNIST, EMNIST, Fashion MNIST, and CIFAR-100 by approximately 7%, 16%, 1%, and 7% respectively compared to the baseline models of similar architectures.

This paper studies a strongly convergent inertial forward-backward-forward algorithm for the variational inequality problem in Hilbert spaces. In our convergence analysis, we do not assume the on-line rule of the inertial parameters and the iterates, which have been assumed by several authors whenever a strongly convergent algorithm with an inertial extrapolation step is proposed for a variational inequality problem. Consequently, our proof arguments are different from what is obtainable in the relevant literature. Finally, we give numerical tests to confirm the theoretical analysis and show that our proposed algorithm is superior to related ones in the literature.

We deal with monotone inclusion problems of the form 0 ∈ Ax + Dx + NC(x) in real Hilbert spaces, where A is a maximally monotone operator, D a cocoercive operator and C the nonempty set of zeros of another cocoercive operator. We propose a forward-backward penalty algorithm for solving this problem which extends the one proposed by Attouch et al. (SIAM J. Optim. 21(4): 1251-1274, 2011). The condition which guarantees the weak ergodic convergence of the sequence of iterates generated by the proposed scheme is formulated by means of the Fitzpatrick function associated to the maximally monotone operator that describes the set C. In the second part we introduce a forward-backward-forward algorithm for monotone inclusion problems having the same structure, but this time by replacing the cocoercivity hypotheses with Lipschitz continuity conditions. The latter penalty type algorithm opens the gate to handle monotone inclusion problems with more complicated structures, for instance, involving compositions of maximally monotone operators with linear continuous ones.

In this work, we propose inclusion problems based on a novel class of forward-backwardforward algorithms. Our approach incorporates multi-inertial extrapolations and utilizes a self-adaptive technique to eliminate the need for explicitly selecting Lipschitz assumptions to enhance the speed convergence of the algorithm. We establish a weak convergence theorem under suitable assumptions. Furthermore, we conduct numerical tests on image deblurring as a practical application. The experimental results demonstrate that our algorithm surpasses some existing methods in the literature, which shows its superior performance and effectiveness.

Bayesian inference for coupled hidden Markov models frequently relies on data augmentation techniques for imputation of the hidden state processes. Considerable progress has been made on developing such techniques, mainly using Markov chain Monte Carlo (MCMC) methods. However, as the dimensionality and complexity of the hidden processes increase some of these methods become inefficient, either because they produce MCMC chains with high autocorrelation or because they become computationally intractable. Motivated by this fact we developed a novel MCMC algorithm, which is a modification of the forward filtering backward sampling algorithm, that achieves a good balance between computation and mixing properties, and thus can be used to analyze models with large numbers of hidden chains. Even though our approach is developed under the assumption of a Markovian model, we show how this assumption can be relaxed leading to minor modifications in the algorithm. Our approach is particularly well suited to epidemic models, where the hidden Markov chains represent the infection status of an individual through time. The performance of our method is assessed on simulated data on epidemic models for the spread of Escherichia coli O157:H7 in cattle. Supplementary materials for this article are available online.

In this paper, we consider an iterative approximation problem of split variational inequalities in Hilbert spaces. In order to solve this split problem, we construct an iterative algorithm which combines a forward-backward-forward method and a self-adaptive rule to update the step-sizes. We prove that the constructed algorithm converges strongly to a solution of the split variational inequalities under some mild assumptions.

Recent deep learning models, including GPT-4, have achieved remarkable performance using the back-propagation (BP) algorithm. However, the mechanism of BP is fundamentally different from how the human brain processes learning. To address this discrepancy, the Forward-Forward (FF) algorithm was introduced. Although FF enables deep learning without backward passes, it suffers from instability, dependence on artificial input construction, and limited generalizability. To overcome these challenges, we propose Local Back-Propagation (LBP), a method that integrates layer-wise unsupervised learning with standard inputs and conventional loss functions. Specifically, LBP demonstrates high training stability and competitive accuracy, significantly outperforming FF-based training methods. Moreover, LBP reduces memory usage by up to 48% compared to convolutional neural networks trained with back-propagation, making it particularly suitable for resource-constrained environments such as federated learning. These results suggest that LBP is a promising biologically inspired training method for decentralized deep learning.

For solving monotone inclusion problems, we propose an inertial under-relaxed version of the relative-error hybrid proximal extragradient method. We study the asymptotic convergence of the method, as well as its nonasymptotic global convergence rates in terms of iteration complexity. We analyze the new method under more flexible assumptions than existing ones, both on the extrapolation and on the relative-error parameters. The approach is applied to two types of forward-backward type methods for solving structured monotone inclusions.

The principle underlying this paper is the basic observation that the problem of simultaneously solving a large class of composite monotone inclusions and their duals can be reduced to that of finding a zero of the sum of a maximally monotone operator and a linear skew-adjoint operator. An algorithmic framework is developed for solving this generic problem in a Hilbert space setting. New primal-dual splitting algorithms are derived from this framework for inclusions involving composite monotone operators, and convergence results are established. These algorithms draw their simplicity and efficacy from the fact that they operate in a fully decomposed fashion in the sense that the monotone operators and the linear transformations involved are activated separately at each iteration. Comparisons with existing methods are made and applications to composite variational problems are demonstrated. [PUBLICATION ABSTRACT]

A convex minimization problem in the form of the sum of two proper lower-semicontinuous convex functions has received much attention from the community of optimization due to its broad applications to many disciplines, such as machine learning, regression and classification problems, image and signal processing, compressed sensing and optimal control. Many methods have been proposed to solve such problems but most of them take advantage of Lipschitz continuous assumption on the derivative of one function from the sum of them. In this work, we introduce a new accelerated algorithm for solving the mentioned convex minimization problem by using a linesearch technique together with a viscosity inertial forward–backward algorithm (VIFBA). A strong convergence result of the proposed method is obtained under some control conditions. As applications, we apply our proposed method to solve regression and classification problems by using an extreme learning machine model. Moreover, we show that our proposed algorithm has more efficiency and better convergence behavior than some algorithms mentioned in the literature.