Abstract
Online federated learning (OFL) is essential for privacy-preserving collaborative online analytics over decentralized streams. Different from batch-based FL, OFL faces new challenges including longitudinal privacy leakage, and accumulated utility loss and communication costs, caused by the infinite data streams. This paper first extends the definition of traditional differential privacy (DP) to OFL, to provide window-based privacy protection with a tunable granularity for infinite streams. By analyzing baseline methods, a generic sampling-based solution framework is then proposed for designing a DP-enhanced OFL algorithm. We prove that despite the DP constraint, the sampling solution framework can achieve an asymptotic optimality when time tends to infinity. Finally, we present Sampling$^{3}$3-OFL, an adaptive triple-sampling strategy driven by deep reinforcement learning, which can dynamically determine a near-optimal sampling strategy with significant gains in both utility and efficiency. Extensive experiments on six real-world datasets demonstrate that Sampling$^{3}$3-OFL can scale to millions of streams, and achieves utility improvements of 0.74%-15.84% and communication cost reductions of 33.33%-95.24% across these datasets compared to state-of-the-art methods.