Catalogue Search | MBRL
Are you sure you want to remove the book from the shelf?
{{itemTitle}}
33,482
result(s) for
"kernel"
Sort by:
Kernel methods and machine learning
\"Offering a fundamental basis in kernel-based learning theory, this book covers both statistical and algebraic principles. It provides over 30 major theorems for kernel-based supervised and unsupervised learning models. The first of the theorems establishes a condition, arguably necessary and sufficient, for the kernelization of learning models. In addition, several other theorems are devoted to proving mathematical equivalence between seemingly unrelated models. With over 25 closed-form and iterative algorithms, the book provides a step-by-step guide to algorithmic procedures and analysing which factors to consider in tackling a given problem, enabling readers to improve specifically designed learning algorithms, build models for new applications and develop efficient techniques suitable for green machine learning technologies. Numerous real-world examples and over 200 problems, several of which are Matlab-based simulation exercises, make this an essential resource for graduate students and professionals in computer science, electrical and biomedical engineering. Solutions to problems are provided online for instructors\"-- Provided by publisher.
Book
A review of kernel methods for feature extraction in nonlinear process monitoring
Kernel methods are a class of learning machines for the fast recognition of nonlinear patterns in any data set. In this paper, the applications of kernel methods for feature extraction in industrial process monitoring are systematically reviewed. First, we describe the reasons for using kernel methods and contextualize them among other machine learning tools. Second, by reviewing a total of 230 papers, this work has identified 12 major issues surrounding the use of kernel methods for nonlinear feature extraction. Each issue was discussed as to why they are important and how they were addressed through the years by many researchers. We also present a breakdown of the commonly used kernel functions, parameter selection routes, and case studies. Lastly, this review provides an outlook into the future of kernel-based process monitoring, which can hopefully instigate more advanced yet practical solutions in the process industries.
Journal Article
Stability of heat kernel estimates for symmetric non-local Dirichlet forms
In this paper, we consider symmetric jump processes of mixed-type on metric measure spaces under general volume doubling condition,
and establish stability of two-sided heat kernel estimates and heat kernel upper bounds. We obtain their stable equivalent
characterizations in terms of the jumping kernels, variants of cut-off Sobolev inequalities, and the Faber-Krahn inequalities. In
particular, we establish stability of heat kernel estimates for
eBook
JUST INTERPOLATE
In the absence of explicit regularization, Kernel “Ridgeless” Regression with nonlinear kernels has the potential to fit the training data perfectly. It has been observed empirically, however, that such interpolated solutions can still generalize well on test data. We isolate a phenomenon of implicit regularization for minimum-norm interpolated solutions which is due to a combination of high dimensionality of the input data, curvature of the kernel function and favorable geometric properties of the data such as an eigenvalue decay of the empirical covariance and kernel matrices. In addition to deriving a data-dependent upper bound on the out-of-sample error, we present experimental evidence suggesting that the phenomenon occurs in the MNIST dataset.
Journal Article
LINEARIZED TWO-LAYERS NEURAL NETWORKS IN HIGH DIMENSION
We consider the problem of learning an unknown function f
* on the d-dimensional sphere with respect to the square loss, given i.i.d. samples {(yi
, xi
)}
i≤n
where xi
is a feature vector uniformly distributed on the sphere and yi
= f
*(xi
) + εi
. We study two popular classes of models that can be regarded as linearizations of two-layers neural networks around a random initialization: the random features model of Rahimi–Recht (RF); the neural tangent model of Jacot–Gabriel–Hongler (NT). Both these models can also be regarded as randomized approximations of kernel ridge regression (with respect to different kernels), and enjoy universal approximation properties when the number of neurons N diverges, for a fixed dimension d.
We consider two specific regimes: the infinite-sample finite-width regime, in which n = ∞ while d and N are large but finite, and the infinite-width finite-sample regime in which N = ∞ while d and n are large but finite. In the first regime, we prove that if d
ℓ+δ
≤ N ≤ d
ℓ+1−δ
for small δ > 0, then RF effectively fits a degree-ℓ polynomial in the raw features, and NT fits a degree-(ℓ + 1) polynomial. In the second regime, both RF and NT reduce to kernel methods with rotationally invariant kernels. We prove that, if the sample size satisfies d
ℓ+δ
≤ n ≤ d
ℓ+1−δ
, then kernel methods can fit at most a degree-ℓ polynomial in the raw features. This lower bound is achieved by kernel ridge regression, and near-optimal prediction error is achieved for vanishing ridge regularization.
Journal Article
Parameter investigation of support vector machine classifier with kernel functions
Support vector machine (SVM) is one of the well-known learning algorithms for classification and regression problems. SVM parameters such as kernel parameters and penalty parameter have a great influence on the complexity and performance of predicting models. Hence, the model selection in SVM involves the penalty parameter and kernel parameters. However, these parameters are usually selected and used as a black box, without understanding the internal details. In this paper, the behavior of the SVM classifier is analyzed when these parameters take different values. This analysis consists of illustrative examples, visualization, and mathematical and geometrical interpretations with the aim of providing the basics of kernel functions with SVM and to show how it works to serve as a comprehensive source for researchers who are interested in this field. This paper starts by highlighting the definition and underlying principles of SVM in details. Moreover, different kernel functions are introduced and the impact of each parameter in these kernel functions is explained from different perspectives.
Journal Article
A Study of Seven Asymmetric Kernels for the Estimation of Cumulative Distribution Functions
In this paper, we complement a study recently conducted in a paper of H.A. Mombeni, B. Masouri and M.R. Akhoond by introducing five new asymmetric kernel c.d.f. estimators on the half-line [0,∞), namely the Gamma, inverse Gamma, LogNormal, inverse Gaussian and reciprocal inverse Gaussian kernel c.d.f. estimators. For these five new estimators, we prove the asymptotic normality and we find asymptotic expressions for the following quantities: bias, variance, mean squared error and mean integrated squared error. A numerical study then compares the performance of the five new c.d.f. estimators against traditional methods and the Birnbaum–Saunders and Weibull kernel c.d.f. estimators from Mombeni, Masouri and Akhoond. By using the same experimental design, we show that the LogNormal and Birnbaum–Saunders kernel c.d.f. estimators perform the best overall, while the other asymmetric kernel estimators are sometimes better but always at least competitive against the boundary kernel method from C. Tenreiro.
Journal Article
Exponential concentration in quantum kernel methods
Kernel methods in Quantum Machine Learning (QML) have recently gained significant attention as a potential candidate for achieving a quantum advantage in data analysis. Among other attractive properties, when training a kernel-based model one is guaranteed to find the optimal model’s parameters due to the convexity of the training landscape. However, this is based on the assumption that the quantum kernel can be efficiently obtained from quantum hardware. In this work we study the performance of quantum kernel models from the perspective of the resources needed to accurately estimate kernel values. We show that, under certain conditions, values of quantum kernels over different input data can be exponentially concentrated (in the number of qubits) towards some fixed value. Thus on training with a polynomial number of measurements, one ends up with a trivial model where the predictions on unseen inputs are independent of the input data. We identify four sources that can lead to concentration including expressivity of data embedding, global measurements, entanglement and noise. For each source, an associated concentration bound of quantum kernels is analytically derived. Lastly, we show that when dealing with classical data, training a parametrized data embedding with a kernel alignment method is also susceptible to exponential concentration. Our results are verified through numerical simulations for several QML tasks. Altogether, we provide guidelines indicating that certain features should be avoided to ensure the efficient evaluation of quantum kernels and so the performance of quantum kernel methods.
Quantum kernel methods are usually believed to enjoy better trainability than quantum neural networks which may suffer from a well-studied barren plateau. Here, building over previous evidence, the authors show that practical implications of exponential concentration result in a trivial data-insensitive model after training, and identify commonly used features that induce the concentration.
Journal Article
Generalized Mercer Kernels and Reproducing Kernel Banach Spaces
This article studies constructions of reproducing kernel Banach spaces (RKBSs) which may be viewed as a generalization of reproducing
kernel Hilbert spaces (RKHSs). A key point is to endow Banach spaces with reproducing kernels such that machine learning in RKBSs can be
well-posed and of easy implementation. First we verify many advanced properties of the general RKBSs such as density, continuity,
separability, implicit representation, imbedding, compactness, representer theorem for learning methods, oracle inequality, and
universal approximation. Then, we develop a new concept of generalized Mercer kernels to construct
eBook
Algebras of Singular Integral Operators with Kernels Controlled by Multiple Norms
The authors study algebras of singular integral operators on \\mathbb R^n and nilpotent Lie groups that arise when considering the composition of Calderón-Zygmund operators with different homogeneities, such as operators occuring in sub-elliptic problems and those arising in elliptic problems. These algebras are characterized in a number of different but equivalent ways: in terms of kernel estimates and cancellation conditions, in terms of estimates of the symbol, and in terms of decompositions into dyadic sums of dilates of bump functions. The resulting operators are pseudo-local and bounded on L^p for 1 \\lt p \\lt \\infty . While the usual class of Calderón-Zygmund operators is invariant under a one-parameter family of dilations, the operators studied here fall outside this class, and reflect a multi-parameter structure.
eBook