Abstract
Existing feature extraction-based image fusion methods have an initial output of noise and iteratively refine them toward the target through training. This approach results in slow convergence and unstable training, with suboptimal fusion quality. To address these limitations, we propose prior-based fusion (PBF), a novel paradigm that constructs the fused output using multi-source image priors as its structural foundation, unlike noise-initialized approaches. This strategy enables faster and more stable training, while allowing the model to focus on complementary information exploration and detail enhancement, ultimately achieving superior fusion quality. Building upon the PBF paradigm, we propose PBFNet, a general-purpose image fusion framework, with its unified version PBFNet-U. They use a manually designed fusion criterion as the switching function to achieve optimal prior information extraction with only a slight increase in model complexity. The novel dense residual dense block is used as a feature extraction block, and at the same time creates a continuous memory between convolutional blocks and convolutional layers, which has a stronger feature extraction capability than residual block, dense block, residual dense block, and aggregated residual dense block. Furthermore, a channel attention mechanism is incorporated to adaptively fuse encoded features, and a gradient-enhanced loss function is designed to improve gradient perception. Both quantitative metrics and visual assessments demonstrate that PBFNet and its unified version surpass state-of-the-art methods across four fusion tasks. The results of ablation studies and discussion and analysis demonstrate the significant advantages of the proposed paradigm and method in stabilizing training, accelerating convergence, and improving fusion performance.