Abstract
Video-based surgical action-triplet recognition represents each surgical action as an $\langle $ instrument, verb, target $\rangle $ triplet, which is fundamental for understanding complex surgical scenes. However, this task remains challenging due to heterogeneous spatio-temporal characteristics and complex inter-component dependencies. To address these issues, we propose a Relation-Aware Class Activation-guided Mixture-of-Experts framework (REACT) for surgical triplet recognition. REACT jointly models spatial details and temporal dynamics while capturing inter-component relations via a Class Activation-guided Mixture-of-Experts (CAMoE) architecture. Class Activation Maps (CAMs) serve as weak semantic priors to guide expert routing, while Feature Modulation Experts (FMEs) adaptively align heterogeneous component features into a shared semantic space. Extensive experiments on CholecT45 and CholecTriplet show that REACT achieves state-of-the-art triplet-level performance with strong relational consistency. Additional evaluation on SIRNet dataset further demonstrates cross-procedure generalization. Code is available at https://github.com/accusemaker/REACT.