BAM: Bilateral Activation Mechanism for Image Fusion
1. ABSTRACT
As the conventional activation functions such as ReLU, LeakyReLU, and PReLU, the negative parts in feature maps are simply truncated or linearized, which may result in unflexible structure and undesired information distortion. In this paper, we propose a simple but effective Bilateral Activation Mechanism (BAM) which could be applied to the activation function to offer an efficient feature extraction model. Based on BAM, the Bilateral ReLU Residual Block (BRRB) that still sufficiently keeps the nonlinear characteristic of ReLU is constructed to separate the feature maps into two parts,i.e., the positive and negative components, then adaptively represent and extract the features by two independent convolution layers.
Besides, our mechanism will not increase any extra parameters or computational burden in the network. We finally embed the BRRB into a basic ResNet architecture, called BRResNet, it is easy to obtain state-of-the-art performance in two image fusion tasks, i.e., pansharpening and hyperspectral image super-resolution (HISR).
Additionally, deeper analysis and ablation study demonstrate the effectiveness of BAM, the lightweight property of the network, etc.
2. INTRODUCTION
l Pansharpening
3. MOTIVATION
5. EXPERIMENT SETTING
l Hyperspectral Image Super-Resolution (HISR)
l CNNs-based Approaches for Pansharpening, e.g., PNN [1]
[1] G. Masi, D. Cozzolino, L. Verdoliva, and G. Scarpa, “Pansharpening by convolutional neural networks,” Remote Sensing, vol. 8, pp. 594, 2016.
Activation function, e.g., ReLU, as the important tool which plays a role in activating the nonlinear fitting ability of CNNs. Thus we
present a framework from the direction of the activation mechanism extension, expecting to explore and utilize the features that can not be activated while retaining the nonlinearity
l More development history of CNN can be found from [2]
[2] Vivone, G., et al. "A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting pansharpening with classical and emerging pansharpening methods. IEEE Geoscience and Remote Sensing Magazine.
l An Observation
p HISR ： 1) Datasets:
p CAVE dataset
p HAVARD dataset
2) Metrics ： SAM, PSNR, SSIM, ERGAS p Pansharpening ：
1) Datasets:
p 8-band data: WorldView-3 (WV3) 1) reduced-resolution examples 2) full-resolution examples
p 4-band data: Quikbird (QB), GaoFen2 (GF2) 2) Metrics ：
p Reduced-resolution: SAM, ERGAS, SCC, Q8 p Full-resolution: QNR, D
�, D
S7. CONCLUSIONS
As a mechanism, BAM provides a more efficient feature extraction mode without increase the
computational burden. Also, it has many variants and can be used as a substitution to replace any structure like “Activation + Convolution", giving us more flexibility in designing the network structure.
6. RESULTS
The current main improvement direction is to change the network structure, such as deepening of depth,
increasing width, and multi-scale convolution operations
Still existing image residuals, which can be further utilized in next step for better output!
o PReLU[3] or LeakyReLU [4]
l Limitation
Feature extraction of the negative part is strengthened by weakening
the nonlinearity
[1] K.M He, et al., Surpassing human-level performance on imagenet classification, ICCV, 2015
[2] Andrew, et al., Rectifier nonlinearities improve neural network acoustic models, ICML, 2013
