Furthermore, since PAE requires a wide separation between the ultrasonic sensor and the amplifier, it is a difficult task to make a PAE system free of EMI noise. To accelerate progress in the development of the field of PAE, we accessed the feasibility of deep learning-based methods for EMI denoising. For future work, once we have effectively de-noised the PAE images, we also expect that if we increase the training dataset, our method can be applied more widely in many areas of photoacoustic tomography to overcome EMI noise and poor SNR.
The EMI noise pattern looks very similar to the blood vessels in the Hilbert transform image. Performance test results for two in vivo data sets of rat colorectum with different EMI noise.
Introduction
In this study, we propose a deep learning-based EMI denoising algorithm for use on PA images acquired by a newly built PAE system [29]. To the best of our knowledge, no previous study has paid as much attention to the problem of EMI noise removal as our work. Based on the comparison, we propose a CNN-based denoising algorithm that best achieves our goal and apply it to in vivo data to confirm the suitability of the method for EMI denoising.
Materials and Methods
- Data source
- Data Preparation
- Classical Methods
- CNN Architectures
- CNN Training and Hyperparameter Tuning
Due to the aforementioned geometry-dependent nature of EMI noise, while we used the same experimental setup, B-scan images were acquired with different noise levels in terms of the noise amplitude over a number of experimental dates. Second, while keeping the gut region intact, we removed the noise outside the region by thresholding. That is, if a pixel value was higher than a threshold value, we assigned the minimum of the adjacent pixel values to the pixel.
The filter calculates the mean and variance of the given pixel sliding window neighborhood 3x3 and recomputes the value of the given pixel [66]. Dilation operation is to assign the value of the maximum pixel value in the neighborhood (in our case 3x3 square area). Similarly, the erosion operation is the assignment of the value of the minimum pixel value in the neighborhood [68].
We believed that denoising methods based on the idea of semantic segmentation should be able to separate locally related vertical patterns from the rest of the images and also minimize the impact of the denoising process on noisy pixels. After shrinking the image size, further dropout layers are applied to reduce the third dimension for the feature channel, and thus reduce the complexity of the neural network [73]. The shrunk input is subjected to a series of depth associations with feature maps by the ReLU convolution and activation layer encoder.
We used the L2 regression loss based on half root mean square error to train the CNN.
Results
Performance Comparison of Trained CNN Architectures
Although all four architectures belong to fully convolutional neural networks, the slight differences in detail between the architectures led to clear distinctions in the different performance scores generated. The key difference between the network structures is the information used in the recovery process. To explain it in detail: Segnet uses indices of max-pool for max-unpool between the same depths for recovery, while FCN-16s and FCN-8s use encoded information at the last two (or three) max-pool layers, which is insufficient to restore the observation target signals.
In contrast, U-Net uses the connection through switching links between the same depth levels, which compensates for the information lost in the max-pooling layers in the restoration of the signals in the decoding part. Regardless of the noise level, all trained CNNs were able to remove noise from the background region. In fact, Segnet failed to recover most of the signals of the observation target, while the FCN structures recovered ambiguous signals.
This aims to compare the four CNN-based algorithms developed in the current study with the three classical noise reduction methods, the median filter, the Wiener filter, the transverse signal gradient-based method used in [29], the adaptive median filter, the median type edge conservation filter, morphological filter and counterharmonic filter, which were chosen for comparison. The denoising performance of each architecture is shown as a pixel-wise error map, which calculates the difference between the ground truth and the average network output of all tested noise levels. Classical noise reduction algorithms, the median filter, the Wiener filter, the transverse signal gradient based method, adaptive median filter, median type edge preserving filter, morphological filter and counterharmonic filter are presented for comparison.
Although this method could remove pixels affected by EMI noise very cleanly, as shown in the figure, with a threshold set high, its main weakness was that it also removed normal capillary signals as there was quite a large overlap between the capillary signals and EMI noise in terms of pixel values and morphological features.
Performance Test for New In Vivo Data
Performance test results for two in vivo test datasets of rat colorectum with different EMI noise levels: (a) full PA-RMAP images (left) and magnified images for dashed box regions (right). Scale bars, 5 mm (horizontal only). b) B-scan (or cross-sectional) images for the positions marked in (a). Due to the unavailability of the corresponding method for quantifying the noise levels, we cannot present the corresponding values at present.
However, as shown in Figure 8a, which displays radial maximum amplitude projection (RMAP) images, the U-Net algorithm removed EMI noise from the two in vivo data sets to a reasonably satisfactory level, and thus the finest, mesh-like capillaries were removed . networks, which typically The presented RMAP images were produced using the corresponding C-scan datasets, which consisted of 3000 B-scan image slices, and the AI algorithm performed the denoising task in a B-scan image state for all involved image slices. To show the related process, we present in Figure 8b two sets of before and after B-scan images, which are taken from the lines marked with dashes and included in the RMAP images presented in Figure 8a.
To more clearly demonstrate the effect of EMI denoising from the two in vivo datasets, we plotted volume-rendered images for the four RMAP images presented in Figure 8a and presented the results in Figure 9. As shown, the vascular structures we searched for through attempts to image the colon and rectum, they are more clearly visible in the denoised (after) images, while they are barely visible in the raw images (before) because they were covered by EMI noise that looked like countless spikes around the vessels. For reference, several dark areas in the denoised RMAP images in Figure 8a appeared dark because the corresponding parts of the colon and rectal wall were imaged outside the working distance of the endoscopic probe, not because the associated data values were lost during the elimination process noises.
Three-dimensional rendering of the two in vivo rat colorectal test datasets presented in Figure 8 .
Discussion
This experience motivated us to proceed with the development of an efficient EMI noise removal algorithm. Although the current algorithm investigated in this study is not 100% perfect, we hope that the EMI noise is no longer an obstacle when working with PAE in the newly used areas. As an alternative to an AI-based denoising approach, one can consider the use of already existing well-developed classical denoising methods for EMI noise removal.
However, as we mentioned earlier in the introduction, during our literature review we came to the conclusion that no previous report has addressed the problem of removing EMI noise that appeared as the rainy striking pattern, as in our case. Therefore, in our previous research [29], we developed a special algorithm that could detect pixels affected by EMI noise based on the calculation and comparison of a signal gradient with the adjacent pixels along the transverse direction, which ultimately turned out not to be the case. satisfactory as the current AI-based solution. Again, the classical methods were not able to correctly distinguish a pixel affected by EMI noise from a normal capillary signal, as the two patterns become similar, as the transverse resolution of the developed PAE probe was at the OR level.
Our point is that while it would not be difficult to remove EMI noise in AR-PAE images using a non-AI based approach, it is not so simple when the problem is related to OR-PAE images. Of course, in the case of OR-PAM [10,14], although it has the same technical basis as OR-PAE, the EMI noise removal issue was not much of a concern because it is relatively easy to assemble a complete hardware system. unaffected by EMI noise. Although the feasibility of the deep learning-based EMI denoising strategy has been successfully demonstrated in this study, there are several limitations.
Second, while our endoscopic system acquired not only the PA images but also the US images, we did not consider the removal of the EMI noise from the US images.
Conclusions
Compensating for visibility artifacts in photoacoustic imaging with a deep learning approach providing predictive uncertainty, photoacoustics. Complete in vivo skin and blood vessel profile segmentation in deep learning photoacoustics based photoacoustic imaging. Deep neural network-based sinogram super-resolution and bandwidth enhancement for data-constrained photoacoustic tomography.
In conjunction with the 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuala Lumpur, Malaysia, 16-18 November 2011; pp. Throughout my time in his group, I learned to research and work hard and be devoted to my projects. I cannot underestimate the great influence of the group where I spend my last undergraduate years as an intern.
Thanks to my younger Yeozu Son, I could improve my Korean skills and have fun together. I would like to say special thanks to my friends who supported me throughout my undergraduate and graduate life. Thanks to my dear girl friends, Thao, who is an expert in formal writing and charlotka cooking, Madina, who is sisterly frisker and shaurma expert, Dilara, who always lifts my mood with her calls, Louise, Vietnamese food connoisseur whom I really miss, and Hana, who can easily read people's minds.
The biggest thanks go to my family who have always supported me in everything I do.
