International Journal of Engineering Advanced Research eISSN: 2710-7167 | Vol. 4 No. 3 [September 2022]
Journal website: http://myjms.mohe.gov.my/index.php/ijear
A COMPARATIVE STUDY OF VECTOR AND RASTER SPATIAL DATA FOR IMAGE SEGMENTATION FOCUSING
ON THE POSITION ESTIMATION ACCURACY OF VERTICES TO FORM POLYGONAL SHAPES
Ahmad Altaweel1*, Ahmad Almassri2, Amarbold Purev3 and Hiroaki Wagatsuma4
1 Power Group, Beirut, LEBANON
2 3 4 Graduate School of Life Science and Systems Engineering, Kyushu Institute of Technology, Kitakyushu, JAPAN
*Corresponding author: [email protected]
Article Information:
Article history:
Received date : 7 July 2022 Revised date : 25 July 2022 Accepted date : 1 September 2022 Published date : 7 September 2022
To cite this document:
Altaweel, A., Almassri, A., Purev, A., &
Wagatsuma, H. (2022). A
COMPARATIVE STUDY OF VECTOR AND RASTER SPATIAL DATA FOR IMAGE SEGMENTATION FOCUSING ON THE POSITION
ESTIMATION ACCURACY OF VERTICES TO FORM
POLYGONAL SHAPES.
International Journal of Engineering Advanced Research, 4(3), 25-34.
Abstract: In the field of image processing, raster data is frequently used for image segmentation and shape estimations for general purposes. On the other hand, in the medical field, which aims to help doctors in surgical operations, semi-automatic robotic manipulations such as the da Vinci Surgical Systemto minimize the necessary area to open for the living body. In the present study, a computational geometry-based method was proposed for vector data to treat the rearrangement of extracted points as candidates for edges of the target shape. On the other hand, the Harris-Stephens algorithm was introduced as a well-known method to detect vertices to form polygonal shapes in raster data as the counterpart in the comparative study. Typically, such a raster data method is not robust to perform in noisy conditions without appropriate filtering in the pre-processing. In the Harris-Stephens algorithm, the adding noises were detected as new vertices to form shapes in the area, which is inevitable in the condition without any semantic information about the shape. Since the proposed computational geometry-based method contains geometrical properties in its computation, it can be enhanced to be robust against noisy data. In the sense of designing a part of the system to be a surgical robot, which requires high accuracy of the position estimation of the target, a maximization of extractions of geometrical properties provides a significant benefit for accurate
1. Introduction
Automated image segmentation has been studied, and various methods have been proposed (Pal and Pal, 1993; Sharma and Aggarwal, 2010; Kuruvilla et al., 2016; da F. Costa and Cesar-Jr, 2018), which is highly important not only in medical fields (Liu et al., 2021; Manzke et al., 2010; Ruskó et al., 2011; Şener et al., 2016; Tang et al., 2020; Tong et al., 2019) but also in other engineering fields commonly. It was traditionally treated by image processing schemes, while recently, the vector-based calculation has been examined to compensate for those schemes because image processing schemes are inevitable limitations in the sense of accuracy. In the present paper, a typical problem focusing on corner detection in the image was treated to explore drawbacks in image processing schemes and find a possible way to compensate for them by using methods derived from computational geometry, which is a systematic design method with high accountability.
2. Literature Review
According to recent advances in technology, sensing data is getting to be massive data, especially in the medical field, to raise expectations in image-based analyses and applications (Liu et al., 2021; Manzke et al., 2010; Ruskó et al., 2011; Şener et al., 2016; Tang et al., 2020; Tong et al., 2019). Probabilistic models and neural network models are a recent trend. Ruskó and Bekes (2011) (Ruskó and Bekes, 2011) proposed a partitioned probabilistic model for liver segmentation for contrast-enhanced magnetic resonance imaging (MRI). Tong et al. (2019) (Tong et al., 2019) proposed a method based on the generative adversarial network (GAN) with shape constraint (SC- GAN) for automated segmentation of delineation of organs-at-risk (OARs) on Head and Neck (H&N) on computed tomography (CT) and MRI. Liu et al. (2021) (Liu et al., 2021) proposed a cross-modal image-oriented segmentation method by using a symmetric full convolutional neural network (SFCNN) with the unsupervised multi-domain adaptation (UMDA) and a spatial neural attention (SNA) structure, termed UMDA-SNA-SFCNN, having the benefits of without a specific requirement of annotations on the test data. Tang et al. (2021) (Liu et al., 2021) developed a deep feature fusion model (DFFM) by multi-sequence MRIs for postoperative glioma segmentation in CT images by utilizing a multi-channel CNN architecture, which combines two deep features together to produce the segmentation data. In principle, those so-called “model-free approaches”
require test data for learning and annotations of what it represents. In addition, if enough accuracy is obtained from the learning from given massive datasets, it works well without a high computation time. At the same time, it takes time for calculation in the learning stage or rich GPU computer system. It implies it takes a high computational cost in total (Fig. 1). Traditionally, manipulation of robotic arms. It contributes not only to surgical robots but also to the solver of 2D replacement puzzles known as tangram puzzles to be a general scheme for robots in all fields behaving in dynamic environments.
Keywords: Computational Geometry, Tangram, Vector Representation, 2D dissection problems, Image Segmentation.
feature extractions can be treated based on the geometrical properties of the image, and therefore intermediate approaches are expected to introduce to compensate for their drawbacks.
Figure 1: Basic paradigms to solve problems extraction of semantic information, classification of images, and segmentation of parts in the image
2.1 Problem Statement
According to the discussion in the previous section, the problem statement in the present paper addressed that computational geometry-based methods can be allocated to redeeming drawbacks in image processing schemes. This problem contains a part of issues such as a comparison of effectiveness in vector and raster data. In the case of the vector data, as schematically shown in Fig. 1, avoidance of a combinatorial explosion of target data is crucial because it causes an exponential increase in computation time. If it is possible, the accuracy of the position estimation, even in a noisy condition, can be designed. In this case, the utilization of relationship information (i.e., primitive semantics) of dots for understanding target properties (i.e., the meaning of data concerning the purpose) is also an important issue. In Fig. 2, cases of algorithms require a combinatorial explosion 𝑂(𝑁𝑚) and minimization of the cost by changing a linear-time algorithm 𝑂(𝑁𝑚).
(a) (b)
Figure 2: A schematic illustration of requirements to avoida combinatorial explosion ( 𝑶(𝑵𝒎) at least 𝑶(𝟐𝑵) case (a) and 𝑶(𝒎𝑵) case)
find nodes to satisfy 1st condition
find nodes to satisfy 2nd condition
Limited Steps find nodes to satisfy 1st condition
find nodes to satisfy 2nd condition
3. Method
In the present paper, corner detection was verified as a feature extraction commonly happing? in image processing. A typical algorithm was introduced in the image processing scheme with raster data, and we propose a compensation algorithm for the verification.
3.1 Image Processing Based Methods
Corner detection is a typical issue in image processing with raster data. In the next section, the most traditional method was introduced.
3.1.1 Harris Corner Detector
Harris–Stephens algorithm to return corner points is called “Harris Corner Detector,”, a detection operator widely used in computer vision algorithms to extract corners and collect features of an image. It was first proposed by Harris and Stephens (1988) (Harris and Stephens, 1988) and was developed based on the Moravec algorithm (Zou et al., 2008; Chen et al., 2009; Ye et al., 2009;
Zhang et al., 2012; Ram and Padmavathi, 2016; Wang et al., 2017). It is based on a signal point feature extraction operator (Wang et al., 2017). The idea is to consider a small window around each pixel in an image. What is required is to identify all such unique pixel windows by shifting each window for a small amount in each direction and then measuring the amount of change that occurs in the pixel. The detector may identify three sorts of regions in an image.
Figure 3: Three typical cases of either flat (a), edge (b) or corner in image
When there is no change in the intensity of the image in all directions then the window is encountered by a flat region (Fig. 3(a)). If there is a change in one direction, the window is encountered by an edge (Fig. (b)). Finally, the window is encountered by significant changes in all directions then there will be a corner shown in Fig. 3(c) (Ram and Padmavathi, 2016). Harris corner detector consists of several steps. The following is the actual algorithm.
Step 1: Finding the x and y derivatives of the image:
𝐼
𝑥⇔
∂𝐼∂𝑥
𝐼
𝑦⇔
∂𝐼∂𝑦
𝐼
𝑥𝐼
𝑦⇔
∂𝐼∂𝑥
∂𝐼
∂𝑦
(1)
Step 2: Finding the products of the derivatives of each pixel:
𝐼𝑥2 = 𝐼𝑥∗ 𝐼𝑥 𝐼𝑦2 = 𝐼𝑦 ∗ 𝐼𝑦 𝐼𝑥𝑦 = 𝐼𝑥∗ 𝐼𝑦
(2)
(a) (b) (c)
Step 3: Calculating the covariance matrix M:
𝑀 = ∑𝑥,𝑦 𝑤(𝑥, 𝑦) [ 𝐼𝑥2 𝐼𝑥𝐼𝑦
𝐼𝑥𝐼𝑦 𝐼𝑦2 ]
(3)
This is calculated by a window function 𝑤(𝑥, 𝑦).The equation to find the matrix M is:
𝐸 (𝑢, 𝑣) = ∑𝑥𝑦 𝑤(𝑥, 𝑦) [𝐼(𝑥 + 𝑢, 𝑦 + 𝑣) − 𝐼(𝑥, 𝑦)]
(4)
where: 𝐼(𝑥 + 𝑢, 𝑦 + 𝑣) is known as the shifted intensity, 𝐼(𝑥, 𝑦) is known as the intensity.
The above equation can be simplified as:
𝐸(𝑢, 𝑣) ≈ [𝑢 𝑣] 𝑀 [𝑢
𝑣]
(5)
Step 4: Calculating the Eigenvalues 𝜆: If 𝜆 is close to 0, then it is not a corner.
Step 5: Calculating the corner response function R:
𝑅 = 𝜆1 𝜆2− 𝛼 (𝜆1 + 𝜆2)2
(6)
𝑅 = 𝑑𝑒𝑡(𝑀) − 𝛼 𝑡𝑟𝑎𝑐𝑒(𝑀)2
(7)
Finally, the result can be obtained based on the index by using the eigenvectors. Geometric property, either edge, corner, or flat, is classified depending on the value of R:
• |𝑅| is small: the region is a flat region, which happens when λ1 and λ2 are small.
• 𝑅 < 0: the region is an edge, which happens when λ1>>λ2 or vice versa.
• 𝑅 > 0: the region is a corner, which happens when λ1 and λ2 are large and 𝜆1 ∼ 𝜆2 .
3.2 Computational Geometry Based Methods
In contrast, computational geometry methods are available (Bezdek et al., 1998; Sack and Urrutia, 2000; da F. Costa and Cesar-Jr, 2018) not only for the detection of edges but also for multiple geometric properties. If nodes are connected to represent a shape or structure in the form of polygons, various information can be obtained. For example, a classification of shapes such as circles, triangles, squares, and convex/non-convex can be implemented into a single method.
Normally, the learning scheme discussed in section 2 is designed for a specific function, such as image segmentation, corner detection, and so on. It implies that the same number of machines are necessary regarding the number of required functions. We tried to explore a possible and common architecture based on computational geometry.
3.2.1 Proposed Method to Connect Nodes Sequentially The proposed method was designed in steps:
1) Connect: nodes are connected if they are neighbours and form a polygon (Fig. 4)
2) Decimate: nodes connected to form the polygon are decimated, and representative nodes are selected to keep away at a certain distance
3) Utilize: estimate curvature at each portion along connected nodes to discriminate shapes such as linear connection and corner by using the curvature factor
Figure 4: Connection of nodes to form a polygon to adapt the requirement
3.3 Data Analysis
The computer experiment was designed to validate methods described in sections 3.1.1 and 3.2.1 (Fig. 5). A triangle filled with black dots was located in a random position and random angle. At various noise levels, the validation will be done on whether corners of the triangle can be detected or not.
Figure 5: Test pattern to verify target methods
4. Results and Discussion
In the first place, Harris Corner Detector (HCD) was examined. Noise levels were given by the random uniform variable of each dot to flip between black and white. The node 𝒖 = [𝑢1, 𝑢2, ⋯ , 𝑢𝑁] ∈ {0,1}𝑁 is flipped if each rand 𝑟𝑖 = [0, 1] exceeds 1 − 𝛾. The noise level 𝛾 is selected from the set of {1 × 10−6, 1 × 10−5 , 1 × 10−4 , 1 × 10−3}. As shown in Fig. 6, HCD detected not only corners of the triangle but also other isolated dots (marked with red color). This tendency gets worse if the noise level changes. It is inevitable and improved algorithms were proposed (Ye et al., 2009; Wang et al., 2017).
(a) (b)
(c) (d)
Figure 6: Results of Harris Corner Detector in noisy conditions (Noise levels are given as 𝜸 = 𝟏 × 𝟏𝟎−𝟔 in (a), 𝜸 = 𝟏 × 𝟏𝟎−𝟓 in (b), 𝜸 = 𝟏 × 𝟏𝟎−𝟒 in (c) and 𝜸 = 𝟏 × 𝟏𝟎−𝟑 in (d))
In the case of the computational geometry-based method, the function designed in section 3.2.1 finely works to detect corners of the triangle through steps of connect, decimate, and utilize functions (Fig.7). As Fig. 7 and 8 showed, the corner detection and angel calculation of the whole shape were clearly demonstrated.
Figure 7: Triangle corner detection by using the proposed method
Figure 8: The angle of the shape calculated by results of corner detections in Fig. 7
5. Conclusion
In the Harris-Stephens algorithm, the adding noises were detected as new vertices to form shapes in the area, which is inevitable in the condition without any semantic information about the shape.
In the proposed computational geometry-based method, geometrical properties were finely analysed, potentially overcoming the difficulty of multiple property detections to be robust against noisy data. In the sense of designing a part of the system to be a surgical robot, which requires high accuracy of the position estimation of the target, a maximization of extractions of geometrical properties provides a significant benefit for accurate manipulation of robotic arms. It contributes not only to surgical robots but also to the solver of 2D replacement puzzles known as tangram puzzles to be a general scheme for robots in all fields behaving in dynamic environments.
6. Acknowledgement
This work was supported in part by JSPS KAKENHI (16H01616, 17H06383), the New Energy and Industrial Technology Development Organization (NEDO) and Project on Regional Revitalization Through Advanced Robotics (Kyushu Institute of Technology/Kitakyushu city, Japan).
References
Bezdek, J. C., Chandrasekhar, R., & Attikouzel, Y. (1998). A Geometric Approach to Edge Detection. IEEE Transactions on Fuzzy Systems, 6(1), 52–75.
Chen, J., Zou, L., Zhang, J., & Dou, L. (2009). The Comparison and Application of Corner Detection Algorithms. Journal of Multimedia, 4(6), 435–441.
da F. Costa, L., & Cesar-Jr, R. M. (2018). Shape Classification and Analysis. CRC Press.
Harris, C. G. & Stephens, M. J. (1988). A Combined Corner and Edge Detector. Proceedings of Alvey Vision Conference, 147–151.
Kuruvilla, J., Sukumaran, D., Sankar, A., & Joy, S. P. (2016). A Review on Image Processing and Image Segmentation. Proceedings of 2016 International Conference on Data Mining and Advanced Computing (SAPIENCE), 198–203.
Liang, S., Tang, F., Huang, X., Yang, K., Zhong, T., Hu, R., Liu, S., Yuan, X., & Zhang, Y. (2019).
Deep-Learning-Based Detection and Segmentation of Organs at Risk in Nasopharyngeal Carcinoma Computed Tomographic Images for Radiotherapy Planning. European Radiology, 29(4), 1961–1967.
Liu, J., Liu, H., Gong, S., Tang, Z., Xie, Y., Yin, H., & Niyoyita, J. P. (2021). Automated Cardiac Segmentation of Cross-Modal Medical images Using Unsupervised Multi-Domain Adaptation and Spatial Neural Attention Structure. Medical Image Analysis, 72, 102135.
Manzke, R., Meyer, C., Ecabert, O., Peters, J., Noordhoek, N. J., Thiagalingam, A., Reddy, V. Y., Chan, R. C., Weese, J., Manzke, R., Meyer, C., Ecabert, O., Peters, J., Noordhoek, N. J., Thiagalingam, A., Reddy, V. Y., Chan, R. C., & Weese, J. (2010). Automatic Segmentation of Rotational X-Ray Images for Anatomic Intra-Procedural Surface Generation in Atrial Fibrillation Ablation Procedures. IEEE Transactions on Medical Imaging, 29(2), 260–272.
Pal, N. R., & Pal, S. K. (1993). A Review on Image Segmentation Techniques. Pattern Recognition, 26(9),1277-1294.
Ram, P., & Padmavathi, S. (2016). Analysis of Harris Corner Detection for Color Images.
Proceedings of 2016 International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES), 405–410.
Ruskó, L., & Bekes, G. (2011). Liver Segmentation for Contrast-Enhanced MR Images Using Partitioned Probabilistic Model. International Journal of Computer Assisted Radiology and Surgery, 6(1), 13–20.
Sack, J. R. & Urrutia. J. (2000). Handbook of Computational Geometry. Elsevier Science Publishers.
Şener, E., Mumcuoglu, E. U., & Hamcan, S. (2016). Bayesian Segmentation of Human Facial Tissue Using 3D MR-CT Information Fusion, Resolution Enhancement and Partial Volume Modelling. Computer Methods and Programs in Biomedicine, 124, 31–44.
Sharma, N., & Aggarwal, L. M. (2010). Automated medical image segmentation techniques.
Journal of Medical Physics, 35(1), 3–14.
Stubendek, A., & Karacs, K. (2018). Shape Recognition Based on Projected Edges and Global Statistical Features. Mathematical Problems in Engineering, 2018, 1–18.
Tang, F., Liang, S., Zhong, T., Huang, X., Deng, X., Zhang, Y., & Zhou, L. (2020). Postoperative Glioma Segmentation in CT Image Using Deep Feature Fusion Model Guided by Multi- Sequence MRIs. European Radiology, 30(2), 823–832.
Tong, N., Gou, S., Yang, S., Cao, M., & Sheng, K. (2019). Shape Constrained Fully Convolutional DenseNet with Adversarial Training for Multiorgan Segmentation on Head and Neck CT and Low‐Field MR Images. Medical Physics, 46(6), 2669–2682.
Wang, Z., Li, R., Shao, Z., Ma, M., Liang, J., Liu, W., Wang, J., & Liu, Y. (2017). Adaptive Harris Corner Detection Algorithm Based on Iterative Threshold. Modern Physics Letters B, 31(15), 1750181.
Ye, Z., Pei, Y., & Shi, J. (2009). An Improved Algorithm for Harris Corner Detection. Proceedings of 2009 2nd International Congress on Image and Signal Processing, 1–4.
Zhang, C., Zheng, Y., Guo, B., Li, C., & Liao, N. (2021). SCN: A Novel Shape Classification Algorithm Based on Convolutional Neural Network. Symmetry, 13(3), 499.
Zhang, X., & Ji, X. H. (2012). An Improved Harris Corner Detection Algorithm for Noised Images.
Advanced Materials Research, 433–440, 6151–6156.
Zou, L., Chen, J., Zhang, J., & Dou, L. (2008). The Comparison of Two Typical Corner Detection Algorithms. Proceedings of 2008 Second International Symposium on Intelligent Information Technology Application, 211–215.