Fast nonlinear integration drives accurate encoding of input information in large multiscale systems

Abstract

Biological and artificial systems encode information through complex nonlinear operations across multiple timescales. A clear understanding of the interplay between this multiscale structure and the nature of nonlinearities at play is, however, missing. Here, we study a general model where the input signal is propagated to an output unit through a processing layer via nonlinear activation functions. We focus on two widely implemented paradigms: nonlinear summation, where signals are first nonlinearly transformed and then combined; nonlinear integration, where they are combined first and then transformed. We find that fast-processing capabilities systematically enhance input-output mutual information, and nonlinear integration outperforms summation in large systems. Conversely, a nontrivial interplay between the two strategies emerges in lower dimensions as a function of interaction strength, heterogeneity, and sparsity of conections between the units. Finally, we reveal a tradeoff between input and processing sizes in strong-coupling regimes. Our results shed light on relevant features of nonlinear information processing with implications for both biological and artificial systems.