Explore the vast range of titles available.

Background Supervised machine learning is increasingly being used to estimate clinical predictive models. Several supervised machine learning models involve hyper-parameters, whose values must be judiciously specified to ensure adequate predictive performance. Objective To compare several (nine) hyper-parameter optimization (HPO) methods, for tuning the hyper-parameters of an extreme gradient boosting model, with application to predicting high-need high-cost health care users. Methods Extreme gradient boosting models were estimated using a randomly sampled training dataset. Models were separately trained using nine different HPO methods: 1) random sampling, 2) simulated annealing, 3) quasi-Monte Carlo sampling, 4-5) two variations of Bayesian hyper-parameter optimization via tree-Parzen estimation, 6-7) two implementations of Bayesian hyper-parameter optimization via Gaussian processes, 8) Bayesian hyper-parameter optimization via random forests, and 9) the covariance matrix adaptation evolutionary strategy. For each HPO method, we estimated 100 extreme gradient boosting models at different hyper-parameter configurations; and evaluated model performance using an AUC metric on a randomly sampled validation dataset. Using the best model identified by each HPO method, we evaluated generalization performance in terms of discrimination and calibration metrics on a randomly sampled held-out test dataset (internal validation) and a temporally independent dataset (external validation). Results The extreme gradient boosting model estimated using default hyper-parameter settings had reasonable discrimination (AUC=0.82) but was not well calibrated. Hyper-parameter tuning using any HPO algorithm/sampler improved model discrimination (AUC=0.84), resulted in models with near perfect calibration, and consistently identified features predictive of high-need high-cost health care users. Conclusions In our study, all HPO algorithms resulted in similar gains in model performance relative to baseline models. This finding likely relates to our study dataset having a large sample size, a relatively small number of features, and a strong signal to noise ratio; and would likely apply to other datasets with similar characteristics.

Finding the best configuration of a neural network’s hyper-parameters may take too long to be feasible using an exhaustive search, especially when the cardinality of the search space has a big combinatorial number of possible solutions with various hyper-parameters. This problem is aggravated when we also need to optimize the parameters of the neural network, such as the weight of the hidden neurons and biases. Extreme learning machines (ELMs) are part of the random weights neural network family, in which parameters are randomly initialized, and the solution, unlike gradient-descent-based algorithms, can be found analytically. This ability is especially useful for metaheuristic analysis due to its reduced training times allowing a faster optimization process, but the problem of finding the best hyper-parameter configuration is still remaining. In this paper, we propose a modification of the artificial bee colony (ABC) metaheuristic to act as parameterizers for a regularized ELM, incorporating three methods: an adaptive mechanism for ABC to balance exploration (global search) and exploitation (local search), an adaptation of the opposition-based learning technique called opposition local-based learning (OLBL) to strengthen exploitation, and a record of access to the search space called forbidden redundant indexes (FRI) that allow us to avoid redundant calculations and track the explored percentage of the search space. We set ten parameterizations applying different combinations of the proposed methods, limiting them to explore up to approximately 10% of the search space, with results over 98% compared to the maximum performance obtained in the exhaustive search in binary and multiclass datasets. The results demonstrate a promising use of these parameterizations to optimize the hyper-parameters of the R-ELM in datasets with different characteristics in cases where computational efficiency is required, with the possibility of extending its use to other problems with similar characteristics with minor modifications, such as the parameterization of support vector machines, digital image filters, and other neural networks, among others.