Abstract
Abstract
In recent years, various limitations of conventional supervised learning (SL) have been identified, motivating the development of reinforcement learning (RL)–and quantum RL that leverages quantum resources such as entanglement and superposition. Among the various RL methodologies, the policy gradient method is considered to have many benefits; for instance, it allows an agent to learn without explicitly knowing the crucial information of the environment such as state transition probabilities and initial state distribution. Meanwhile, from the perspective of learning, two indicators are often regarded as significant: expressivity and trainability (for gradient-based methods). While a number of attempts have been made to quantify the expressivity and trainability of Neural Network models and parameterized quantum circuits, clear efforts suitable for RL settings have so far been lacking, despite the inherent differences between conventional SL and RL. Therefore, in this study, we propose revising the notion of expressivity into a temporal expressivity suited to RL dynamics, and show that the mutual information (MI) between the action distribution and the discretized reward signal provides an upper bound for the scaled gradient norm, while yielding an information-theoretic decomposition and a residual-aware upper bound for the proposed temporal expressivity metric. Finally, under explicit concentration assumptions, we show that MI-based Temporal Expressivity and Trainability measure induces an assumption-based, one-sided prescreening criterion for initialization-time gradient fragility.