Exploring Complex System Behaviors via Information-Theoretic Measures: Applications to Physiological Dynamics and Machine Learning

Recent advances in data-driven modeling of complex systems have identified information-theoretic metrics as powerful tools for extracting meaningful insights by exploiting the stochastic nature of investigated phenomena. In this perspective, these metrics have been widely applied to the study of intertwined physiological dynamics within the framework of network physiology. Nevertheless, the literature still presents a fragmented characterization of the several aspects that should be taken into account when applying these metrics, that are related to both theoretical foundations and practical computation, which in turn affect their meaningful interpretation in the context of neuroscience and physiology.This thesis provides a comprehensive investigation of established information-theoretic measures designed for the analysis of the individual components of a complex system and their pairwise interactions, as well as more recent approaches aimed at capturing emergent behaviors arising from the interaction of multiple components. Particular emphasis is given to the challenges associated with the practical computation of these metrics, highlighting how the choice of an appropriate estimation strategy should account for data characteristics - such as nonlinearity, nonstationary, and dimensionality - as well as for the desired trade-offs between feasibility, robustness, and computational cost. The effectiveness of these approaches in enabling a hierarchical evaluation of interactions in complex systems is first demonstrated on simulation studies and then illustrated on different applications in the fields of network physiology and machine learning. In the former domain, the use of metrics quantifying the complexity of individual physiological systems, along with pairwise measures of coupling and causality, and high-order measures of redundant and synergistic interactions, reveals the intrinsically intricate nature of healthy physiological dynamics. Moreover, the combined use of estimation approaches capable of capturing both linear and nonlinear dynamics enables a more comprehensive characterization of well-established regulatory mechanisms, such as those governing cardiovascular and cardiorespiratory interactions among heart rate, arterial pressure, and respiratory activity, as well as less explored parameters, such as arterial compliance. In the context of machine learning, the adoption of information-theoretic measures capturing high-order feature interactions is shown to improve learning models performance through advanced feature selection and importance strategies. Applications to physiological feature sets further highlight the enhanced interpretability achieved by integrating these metrics into machine learning frameworks.Overall, the findings of this thesis demonstrate the effectiveness of information theory in providing a unified analytical framework for the study of network systems, enabling a stratified characterization and a comprehensive understanding of the underlying complex behaviors.