Explore the vast range of titles available.

In this survey, we begin with a concise introduction to information theory within Shannon’s framework, focusing on the key concept of Shannon entropy and its related quantities: relative entropy, joint entropy, conditional entropy, and mutual information. We then demonstrate how to apply these information-theoretic tools in quantum chemistry, adopting either classical or quantum formalisms based on the choice of information carrier involved.

Entropy is a powerful tool for quantification of the brain function and its information processing capacity. This is evident in its broad domain of applications that range from functional interactivity between the brain regions to quantification of the state of consciousness. A number of previous reviews summarized the use of entropic measures in neuroscience. However, these studies either focused on the overall use of nonlinear analytical methodologies for quantification of the brain activity or their contents pertained to a particular area of neuroscientific research. The present study aims at complementing these previous reviews in two ways. First, by covering the literature that specifically makes use of entropy for studying the brain function. Second, by highlighting the three fields of research in which the use of entropy has yielded highly promising results: the (altered) state of consciousness, the ageing brain, and the quantification of the brain networks’ information processing. In so doing, the present overview identifies that the use of entropic measures for the study of consciousness and its (altered) states led the field to substantially advance the previous findings. Moreover, it realizes that the use of these measures for the study of the ageing brain resulted in significant insights on various ways that the process of ageing may affect the dynamics and information processing capacity of the brain. It further reveals that their utilization for analysis of the brain regional interactivity formed a bridge between the previous two research areas, thereby providing further evidence in support of their results. It concludes by highlighting some potential considerations that may help future research to refine the use of entropic measures for the study of brain complexity and its function. The present study helps realize that (despite their seemingly differing lines of inquiry) the study of consciousness, the ageing brain, and the brain networks’ information processing are highly interrelated. Specifically, it identifies that the complexity, as quantified by entropy, is a fundamental property of conscious experience, which also plays a vital role in the brain’s capacity for adaptation and therefore whose loss by ageing constitutes a basis for diseases and disorders. Interestingly, these two perspectives neatly come together through the association of entropy and the brain capacity for information processing.

We present an approach to response around arbitrary out-of-equilibrium states in the form of a fluctuation–response inequality (FRI). We study the response of an observable to a perturbation of the underlying stochastic dynamics. We find that the magnitude of the response is bounded from above by the fluctuations of the observable in the unperturbed system and the Kullback–Leibler divergence between the probability densities describing the perturbed and the unperturbed system. This establishes a connection between linear response and concepts of information theory. We show that in many physical situations, the relative entropy may be expressed in terms of physical observables. As a direct consequence of this FRI, we show that for steady-state particle transport, the differential mobility is bounded by the diffusivity. For a “virtual” perturbation proportional to the local mean velocity, we recover the thermodynamic uncertainty relation (TUR) for steady-state transport processes. Finally, we use the FRI to derive a generalization of the uncertainty relation to arbitrary dynamics, which involves higher-order cumulants of the observable. We provide an explicit example, in which the TUR is violated but its generalization is satisfied with equality.

Modeling and inference are central tomost areas of science and especially to evolving and complex systems. Critically, the information we have is often uncertain and insufficient, resulting in an underdetermined inference problem; multiple inferences, models, and theories are consistent with available information. Information theory (in particular, the maximum information entropy formalism) provides a way to deal with such complexity. It has been applied to numerous problems, within and across many disciplines, over the last few decades. In this perspective, we review the historical development of this procedure, provide an overview of the many applications of maximum entropy and its extensions to complex systems, and discuss in more detail some recent advances in constructing comprehensive theory based on this inference procedure. We also discuss efforts at the frontier of information-theoretic inference: application to complex dynamic systems with time-varying constraints, such as highly disturbed ecosystems or rapidly changing economies.

An increasing number of studies across many research fields from biomedical engineering to finance are employing measures of entropy to quantify the regularity, variability or randomness of time series and image data. Entropy, as it relates to information theory and dynamical systems theory, can be estimated in many ways, with newly developed methods being continuously introduced in the scientific literature. Despite the growing interest in entropic time series and image analysis, there is a shortage of validated, open-source software tools that enable researchers to apply these methods. To date, packages for performing entropy analysis are often run using graphical user interfaces, lack the necessary supporting documentation, or do not include functions for more advanced entropy methods, such as cross-entropy, multiscale cross-entropy or bidimensional entropy. In light of this, this paper introduces EntropyHub , an open-source toolkit for performing entropic time series analysis in MATLAB, Python and Julia. EntropyHub (version 0.1) provides an extensive range of more than forty functions for estimating cross-, multiscale, multiscale cross-, and bidimensional entropy, each including a number of keyword arguments that allows the user to specify multiple parameters in the entropy calculation. Instructions for installation, descriptions of function syntax, and examples of use are fully detailed in the supporting documentation, available on the EntropyHub website– www.EntropyHub.xyz . Compatible with Windows, Mac and Linux operating systems, EntropyHub is hosted on GitHub, as well as the native package repository for MATLAB, Python and Julia, respectively. The goal of EntropyHub is to integrate the many established entropy methods into one complete resource, providing tools that make advanced entropic time series analysis straightforward and reproducible.

When monitoring the dynamics of stochastic systems, such as interacting particles agitated by thermal noise, disentangling deterministic forces from Brownian motion is challenging. Indeed, we show that there is an information-theoretic bound, the capacity of the system when viewed as a communication channel, that limits the rate at which information about the force field can be extracted from a Brownian trajectory. This capacity provides an upper bound to the system’s entropy production rate and quantifies the rate at which the trajectory becomes distinguishable from pure Brownian motion. We propose a practical and principled method, stochastic force inference, that uses this information to approximate force fields and spatially variable diffusion coefficients. It is data efficient, including in high dimensions, robust to experimental noise, and provides a self-consistent estimate of the inference error. In addition to forces, this technique readily permits the evaluation of out-of-equilibrium currents and the corresponding entropy production with a limited amount of data.

A bstract Driven quantum systems exhibit a large variety of interesting and sometimes exotic phenomena. Of particular interest are driven conformal field theories (CFTs) which describe quantum many-body systems at criticality. In this paper, we develop both a spacetime and a quantum information geometry perspective on driven 2d CFTs. We show that for a large class of driving protocols the theories admit an alternative but equivalent formulation in terms of a CFT defined on a spacetime with a time-dependent metric. We prove this equivalence both in the operator formulation as well as in the path integral description of the theory. A complementary quantum information geometric perspective for driven 2d CFTs employs the so-called Bogoliubov-Kubo-Mori (BKM) metric, which is the counterpart of the Fisher metric of classical information theory, and which is obtained from a perturbative expansion of relative entropy. We compute the BKM metric for the universal sector of Virasoro excitations of a thermal state, which captures a large class of driving protocols, and find it to be a useful tool to classify and characterize different types of driving. For Möbius driving by the SL(2 , ℝ) subgroup, the BKM metric becomes the hyperbolic metric on the disk. We show how the non-trivial dynamics of Floquet driven CFTs is encoded in the BKM geometry via Möbius transformations. This allows us to identify ergodic and non-ergodic regimes in the driving. We also explain how holographic driven CFTs are dual to driven BTZ black holes with evolving horizons. The deformation of the black hole horizon towards and away from the asymptotic boundary provides a holographic understanding of heating and cooling in Floquet CFTs.

In the past 15 years, statistical physics has been successful as a framework for modelling complex networks. On the theoretical side, this approach has unveiled a variety of physical phenomena, such as the emergence of mixed distributions and ensemble non-equivalence, that are observed in heterogeneous networks but not in homogeneous systems. At the same time, thanks to the deep connection between the principle of maximum entropy and information theory, statistical physics has led to the definition of null models for networks that reproduce features of real-world systems but that are otherwise as random as possible. We review here the statistical physics approach and the null models for complex networks, focusing in particular on analytical frameworks that reproduce local network features. We show how these models have been used to detect statistically significant structural patterns in real-world networks and to reconstruct the network structure in cases of incomplete information. We further survey the statistical physics models that reproduce more complex, semilocal network features using Markov chain Monte Carlo sampling, as well as models of generalized network structures, such as multiplex networks, interacting networks and simplicial complexes.This Review describes advances in the statistical physics of complex networks and provides a reference for the state of the art in theoretical network modelling and applications to real-world systems for pattern detection and network reconstruction.

A bstract We propose a new formula for computing holographic Renyi entropies in the presence of multiple extremal surfaces. Our proposal is based on computing the wave function in the basis of fixed-area states and assuming a diagonal approximation for the Renyi entropy. For Renyi index n ≥ 1, our proposal agrees with the existing cosmic brane proposal for holographic Renyi entropy. For n < 1, however, our proposal predicts a new phase with leading order (in Newton’s constant G ) corrections to the cosmic brane proposal, even far from entanglement phase transitions and when bulk quantum corrections are unimportant. Recast in terms of optimization over fixed-area states, the difference between the two proposals can be understood to come from the order of optimization: for n < 1, the cosmic brane proposal is a minimax prescription whereas our proposal is a maximin prescription. We demonstrate the presence of such leading order corrections using illustrative examples. In particular, our proposal reproduces existing results in the literature for the PSSY model and high-energy eigenstates, providing a universal explanation for previously found leading order corrections to the n < 1 Renyi entropies.