Reactive flows and unproductive cycles in irreversible Markov chains

Abstract

We present a comprehensive theory for analysis and understanding of transition events between an initial set A and a target set B for general ergodic finite-state space Markov chains or jump processes, including random walks on networks as they occur, e.g., in Markov State Modelling in molecular dynamics. The theory allows us to decompose the probability flow generated by transition events between the sets A and B into the productive part that directly flows from A to B through reaction pathways and the unproductive part that runs in loops and is supported on cycles of the underlying network. It applies to random walks on directed networks and nonreversible Markov processes and can be seen as an extension of Transition Path Theory. Information on reaction pathways and unproductive cycles results from the stochastic cycle decomposition of the underlying network which also allows to compute their corresponding weight, thus characterizing completely which structure is used how often in transition events. The new theory is illustrated by an application to a Markov State Model resulting from weakly damped Langevin dynamics where the unproductive cycles are associated with periodic orbits of the underlying Hamiltonian dynamics.