Integrated information decomposition

Time-delayed mutual information is a single number. Integrated information decomposition splits it into sixteen terms of redundant, unique, and synergistic flow. DIPHINE estimates all of them for continuous data with one diffusion model.

DIPHINE estimates the sixteen terms of integrated information decomposition (ΦID) for two coupled continuous processes, from nine mutual informations given by a single diffusion network. Earlier ΦID computations were limited to Gaussian or discrete systems.

Paper (preprint, 2026)

Take two coupled processes, for example breathing and heart rhythm, observed over time. The time-delayed mutual information I(X_t; X_{t+1}) tells how much the present state of the pair says about the next one. It lumps together quite different things: each process keeping its own state, information passing from one to the other, information both carry, and information that exists only in their joint state. Integrated information decomposition separates them.

From PID to ΦID

Partial information decomposition (Williams and Beer, 2010) splits the information that two sources X_1, X_2 carry about a target Y into four parts,

I(X_1, X_2; Y) = R(X_1, X_2 \to Y) + U(X_1 \to Y) + U(X_2 \to Y) + S(X_1, X_2 \to Y).

The redundancy R is available from either source, each unique term U from one source only, and the synergy S only from both together. The four parts sit on a lattice, and given a redundancy function they follow by Möbius inversion on that lattice.

Integrated information decomposition (Mediano et al., 2025) applies the same idea to a process: the present \{X_{1,t}, X_{2,t}\} plays the role of the sources and the future \{X_{1,t+1}, X_{2,t+1}\} the role of the targets. Each of the four source types (redundant, unique to 1, unique to 2, synergistic) pairs with each of the four target types, which gives sixteen atoms \pi(\alpha, \beta) that add up to the time-delayed mutual information,

I(X_t; X_{t+1}) = \sum_{(\alpha,\beta)} \pi(\alpha, \beta).

For example, \pi(\text{Red}, \text{Syn}) is information that either variable carries now and that can only be decoded from both variables at the next step. Transfer entropy, active information storage, and integrated information are all sums of some of these atoms.

The ΦID lattice for two sources and two targets. Each node is an atom (source type → target type), from Red→Red at the bottom to Syn→Syn at the top.

The decomposition needs a redundancy function. We use the minimum mutual information, extended to the double lattice,

I^{\Phi}_\cap(\alpha, \beta) = \min_{\mathbf a \in \alpha}\ \min_{\mathbf b \in \beta}\ I(\mathbf a; \mathbf b),

which is the only choice with a closed form in the Gaussian case. A known side effect is that some atoms can come out negative, since this redundancy is not monotone on the double lattice.

For continuous, non-Gaussian data there was no general way to compute ΦID. Estimators of partial information decomposition beyond the Gaussian and discrete cases handle a single target. Using a generic mutual information estimator would mean training nine separate models, each with its own bias.

The estimator

Every redundancy value is a minimum over mutual informations I(\mathbf a; \mathbf b), where \mathbf a is a nonempty subset of the present and \mathbf b a nonempty subset of the future. That makes 3 \times 3 = 9 terms. We estimate each with the conditional form of the identity behind this series,

I(\mathbf a; \mathbf b) = \mathbb E_{\mathbf a}\Big[\operatorname{KL}\big(p_{\mathbf b \mid \mathbf a} \,\big\|\, p_{\mathbf b}\big)\Big] \simeq \int_0^T \frac{g_\tau^2}{2}\, \mathbb E\Big\lVert \nabla \log p_\tau(\tilde{\mathbf b} \mid \mathbf a) - \nabla \log p_\tau(\tilde{\mathbf b}) \Big\rVert^2 \mathrm d\tau ,

which is nonnegative by construction. One network takes the four blocks [X_{1,t}, X_{2,t}, X_{1,t+1}, X_{2,t+1}] with a mask in \{-1, 0, 1\}^4 (diffused, clean conditioning, or marginalized), and twelve masks cover every score we need. From there the steps are mechanical: estimate the nine mutual informations, take the minima to get the sixteen redundancy values, and recover the atoms bottom-up by Möbius inversion, which factorizes over sources and targets. The atoms sum to the estimated I(X_t; X_{t+1}) by construction.

The error analysis explains which atoms are hard. As long as the ordering of the mutual informations inside each minimum is right, the error of every atom is an exact linear combination of the mutual information errors, with coefficients in \{0, \pm 1\}. Red→Red inherits the error of a single term, while Syn→Syn collects the errors of four or six terms, which makes it provably the hardest atom to estimate.

Results

On Gaussian autoregressive systems, where the atoms have a closed form (coupled, coupled in one direction, and decoupled), the mutual informations are estimated to within about 0.01 nats on average. The atom errors range from about 0.001 for Red→Red to between 0.018 and 0.036 for Syn→Syn, as the analysis predicts. When each process becomes d = 3, 5, or 10 dimensional, the mutual information error grows to about 0.015, 0.03, and 0.06 to 0.08 nats.

Mean absolute error of the mutual informations (left) and of the atoms (right) for DIPHINE and four estimators trained nine times each, as the dimension of each process grows from 1 to 10.

The comparison with MINE, NWJ, InfoNCE, and KSG, each trained once per mutual information, depends on dimension. At d = 3, InfoNCE and NWJ are as accurate as DIPHINE or slightly better. At d = 10 (20 dimensions in total), DIPHINE has the lowest atom error, 0.038 to 0.044 nats, against 0.055 to 0.063 for InfoNCE and more than 0.43 for KSG, with one network where the others need nine. The estimates hold under the half-cube transformation and degrade under the normal CDF transformation.

Breathing and heart rhythm across age

We apply DIPHINE to the Fantasia database (PhysioNet): healthy subjects at rest, young (21 to 34 years) and elderly (68 to 85 years), with X_t = [\text{respiration}_t, \text{RR interval}_t].

Left: transfer entropy recovered from the atoms, for young and elderly subjects in both directions. Right: the same transfer entropy split into its atoms; the Syn→Red atom dominates in both directions.

The transfer entropy obtained from the atoms is higher in young subjects in both directions. In the young group the flow from respiration to heart rhythm dominates, and in the elderly group this asymmetry fades, in line with the known loss of parasympathetic cardiorespiratory coupling with age. The decomposition shows something transfer entropy alone cannot: in both directions most of the transferred information is synergistic in the present and redundant in the future (the Syn→Red atom). The self-storage atoms (Un→Un) are the largest and grow with age, which may reflect the two systems becoming more independent.

All sixteen atoms (nats) for young and elderly subjects in both directions.

The method is limited to ergodic systems split into two parts; with more parts the lattice grows super-exponentially. The minimum mutual information redundancy can give negative atoms, and the estimators come without formal convergence guarantees.

References

• S. P. Galeano Muñoz, M. Bounoua, G. Franzese, P. Michiardi, M. Filippone. DIPHINE: Diffusion-based ΦID Neural Estimator. Preprint, 2026. arXiv:2606.18997.
• P. A. M. Mediano, F. E. Rosas, A. I. Luppi, R. L. Carhart-Harris, D. Bor, A. K. Seth, A. B. Barrett. Toward a unified taxonomy of information dynamics via integrated information decomposition. PNAS, 2025.
• P. L. Williams, R. D. Beer. Nonnegative decomposition of multivariate information. 2010.
• A. B. Barrett. Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems. Physical Review E, 2015.