Journal Article

Fundamental limits to learning closed-form mathematical models from data

Guimerà, Roger, Duch, Jordi, Reichardt, Ignasi, De Los Ríos, Harry R., Fajardo-Fontiveros, Oscar, Sales-Pardo, Marta

2023

Overview

Given a finite and noisy dataset generated with a closed-form mathematical model, when is it possible to learn the true generating model from the data alone? This is the question we investigate here. We show that this model-learning problem displays a transition from a low-noise phase in which the true model can be learned, to a phase in which the observation noise is too high for the true model to be learned by any method. Both in the low-noise phase and in the high-noise phase, probabilistic model selection leads to optimal generalization to unseen data. This is in contrast to standard machine learning approaches, including artificial neural networks, which in this particular problem are limited, in the low-noise phase, by their ability to interpolate. In the transition region between the learnable and unlearnable phases, generalization is hard for all approaches including probabilistic model selection. Learning analytical models from noisy data remains challenging and depends essentially on the noise level. The authors analyze the transition of the model-learning problem from a low-noise phase to a phase where noise is too high for the underlying model to be learned by any method, and estimate upper bounds for the transition noise.

• Publisher: Nature Publishing Group UK,Nature Publishing Group,Nature Portfolio
• Subject: 639/705/1042 / 639/766/530 / Approximation / Artificial neural networks / Closed form solutions / Datasets / Exact solutions / Humanities and Social Sciences / Machine learning / Mathematical analysis / Mathematical models / multidisciplinary / Neural networks / Noise levels / Probabilistic models / Science / Science (multidisciplinary) / Upper bounds