3.3 Other segmentation
Figure 3-17: Results of instance segmentation[6].The first column is the original images, the second is the results obtained by the instance segmentation, and the last one is the ground truth.
4
Conclusion
In this paper, we introduced various techniques in image segmentation. Especially, the SNAKE model opened up the variational approach in image segmentation from which diverse edge-based models developed. However, the edge-based models have common difficulties in that it is not easily adaptable to topological changes and, hence, does not work well for images with multiple objects. This brought about the so called region-based approach to complement the drawbacks of the edge-based models. As opposed to edge-based models that detect edges by parametrized curves, the Chan-Vese model, a representative region-based model in image segmentation, captures edges as boundaries of particular regions using the level set method.
This enables to detect the more exact edges in the image but it still su↵ers from nonconvexity.
To resolve that, various improvements such as convexification have been devised. Consequently, the variational method can find multiple objects well in images with rather simple structures.
However, it is not yet well developed for complicated images such as a natural scene. In addition, one of the important problems in the variational approach is to determine the number of components, i.e., segmented regions. Because images usually contain multiple objects and we do not know a priori how many there are, the multiphase segmentation is important and it will be necessary to develop models that can automatically determine how many phases should a given image consist of.
This variational method is widely used in those fields such as medical imaging and is also used as pre-processing of other image analysis. It should be noted that the variational approach performs well for low-level image segmentation tasks. That being said, it is difficult to use it for high-level segmentation applications such as recognizing and distinguishing each object. On the other hand, machine learning techniques can perform better in those tasks.
For the development of image segmentation in the machine learning side, semantic segmen- tation has emerged as a result of the development of deep learning, which can not only capture objects in the image but also distinguish the types of objects. This allows for more sophisticated segmentation by classifying all pixels into same category. As we presented in the manuscript,
the result has a very high accuracy compared to the ground truth, which is the actual segmen- tation of the original image. In addition to the semantic segmentation, a more refined partition that separates each object in the same class is called instance segmentation which is obtained by applying the watershed algorithm to the results of semantic segmentation. Since it consists of two steps, it obtains better results than the semantic segmentation. For the development of instance segmentation, it is anticipated that the task of distinguishing each object can be done using another algorithms other than the watershed algorithm, for example graph cut, global thresholding, etc.
References
[1] H. Mewada, R. Patel, and S. Patnaik, “A novel structure tensor modulated chan–vese model for texture image segmentation,” The Computer Journal, vol. 58, no. 9, pp. 2044–
2060, 2014. viii, 16, 17
[2] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmen- tation,” inProceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431–3440, 2015. viii, 2, 20, 21, 22
[3] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,”arXiv preprint arXiv:1606.00915, 2016. ix, 2, 23, 24, 25, 26, 27
[4] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–
778, 2016. ix, 27
[5] L.-C. Chen, G. Papandreou, F. Schro↵, and H. Adam, “Rethinking atrous convolution for semantic image segmentation,”arXiv preprint arXiv:1706.05587, 2017. ix, 2, 27, 28, 29 [6] M. Bai and R. Urtasun, “Deep watershed transform for instance segmentation,” in Com-
puter Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on, pp. 2858–2866, IEEE, 2017. ix, 2, 30, 31
[7] C. Vachier and F. Meyer, “The viscous watershed transform,” Journal of Mathematical Imaging and Vision, vol. 22, no. 2-3, pp. 251–267, 2005. ix, 30
[8] M. Kass, A. Witkin, and D. Terzopoulos, “Snakes: Active contour models,”International journal of computer vision, vol. 1, no. 4, pp. 321–331, 1988. 1, 4
[9] T. F. Chan and L. A. Vese, “Active contours without edges,”IEEE Transactions on image processing, vol. 10, no. 2, pp. 266–277, 2001. 1, 10
[10] E. S. Brown, T. F. Chan, and X. Bresson, “Completely convex formulation of the chan- vese image segmentation model,”International journal of computer vision, vol. 98, no. 1, pp. 103–121, 2012. 2, 13, 14
REFERENCES [11] E. Bae and X.-C. Tai, “Efficient global minimization for the multiphase chan-vese model of image segmentation,” in International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 28–41, Springer, 2009. 2, 16
[12] X. Cai, R. Chan, and T. Zeng, “A two-stage image segmentation method using a convex variant of the mumford–shah model and thresholding,”SIAM Journal on Imaging Sciences, vol. 6, no. 1, pp. 368–390, 2013. 2, 16
[13] C. Xu and J. L. Prince, “Snakes, shapes, and gradient vector flow,”IEEE Transactions on image processing, vol. 7, no. 3, pp. 359–369, 1998. 6
[14] M. Zhang, Q. Li, L. Li, and P. Bai, “An improved algorithm based on the gvf-snake for e↵ective concavity edge detection,”Journal of Software Engineering and Applications, vol. 6, no. 04, p. 174, 2013. 7
[15] D. Mumford and J. Shah, “Optimal approximations by piecewise smooth functions and as- sociated variational problems,”Communications on pure and applied mathematics, vol. 42, no. 5, pp. 577–685, 1989. 9
[16] L. Ambrosio and V. M. Tortorelli, “Approximation of functional depending on jumps by elliptic functional via t-convergence,”Communications on Pure and Applied Mathematics, vol. 43, no. 8, pp. 999–1036, 1990. 9
[17] Y. Kee and J. Kim, “A convex relaxation of the ambrosio–tortorelli elliptic functionals for the mumford-shah functional,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4074–4081, 2014. 10
[18] T. F. Chan, S. Esedoglu, and M. Nikolova, “Algorithms for finding global minimizers of image segmentation and denoising models,”SIAM journal on applied mathematics, vol. 66, no. 5, pp. 1632–1648, 2006. 15
[19] V. Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep convolutional encoder- decoder architecture for image segmentation,”IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 12, pp. 2481–2495, 2017. 22
[20] P. Kr¨ahenb¨uhl and V. Koltun, “Efficient inference in fully connected crfs with gaussian edge potentials,” inAdvances in neural information processing systems, pp. 109–117, 2011.
25
[21] T. F. Chan and J. J. Shen, Image processing and analysis: variational, PDE, wavelet, and stochastic methods, vol. 94. Siam, 2005.
REFERENCES [22] L. Bar, T. F. Chan, G. Chung, M. Jung, N. Kiryati, R. Mohieddine, N. Sochen, and L. A.
Vese, “Mumford and shah model and its applications to image segmentation andimage restoration,” in Handbook of mathematical methods in imaging, pp. 1095–1157, Springer, 2011.
[23] X.-C. Tai and T. F. Chan, “A survey on multiple level set methods with applications for identifying piecewise constant functions,” Int. J. Numer. Anal. Model, vol. 1, no. 1, pp. 25–47, 2004.
ACKNOWLEDGEMENTS
I would like to thank my advisor Yunho Kim, who has been an invaluable mentor. His practical advice and enduring encouragement have been an inestimable source of support for me during my master’s course. I’d like to record here my acknowledgement to him for his guiding and directing this study. I would also like to thank Dr. Dongsun Lee. His varied perspectives, advice and feedback have helped me to strengthen my work. I would like to thank committee members: Bongsoo Jang, Pilwon Kim, for taking the time to provide helpful comments and questions.
I am so lucky to have friends who have faith in me and have always stood by me. In particular, I would like to thank to my classmates, Wanyong, Seyeon, and JongO. And I am very grateful to Dr. Hyojung Lee who always takes care of me and give me unstinting support. In addition, I am thankful to Soyeong who regards me as her sister and Jungwon and Keon Ho who have tea time with me. I also want to thank my labmate of Mathematical Imaging Lab, Joo Dong. I would like to thank Hoyeon and Dr. Kyunghoon Kim who helped me complete my thesis during my master’s course.
I would like to express my appreciation to parents, Kooyong Kim and Ogsun Kim, whose love and guidance are with me in whatever I pursue. Most importantly I want to thank my brother Daehwan Kim, who provide unending inspiration and trust. My family all kept me going, and this thesis would not have been possible without them. Thank you for always being there for me.