Fast Ellipse Fitting Implementation on USG Mobile Telehealth Application
Made Wira Dhanar Santika Faculty of Computer Science
Universitas Indonesia Depok, Indonesia [email protected]
Alhadi Bustamam Faculty of Mathematics and Natural
Sciences Universitas Indonesia
Depok, Indonesia I Made Agus Dwi Suarjaya Faculty of Information Technology
Universitas Udayana Bali, Indonesia
Muhammad Anwar Ma’sum Faculty of Computer Science
Universitas Indonesia Depok, Indonesia [email protected]
Adila Alfa Krisnadhi Faculty of Computer Science
Universitas Indonesia Depok, Indonesia
Adi Nurhadiyatna
Faculty of Engineering and Computing University of Zagreb
Zagreb, Croatia
Aria Kekalih Faculty of Medicine Universitas Indonesia
Depok, Indonesia
Noor Akhmad Setiawan Faculty of Engineering Universitas Gadjah Mada
Yogyakarta, Indonesia
Wisnu Jatmiko Faculty of Computer Science
Universitas Indonesia Depok, Indonesia [email protected]
Abstract— Fetal head circumference (HC) is one of the most important biometrics in assessing fetal growth during prenatal ultrasound examinations. However, measuring the fetal head is not an easy task. This study aims to create an automatic fetal head measurement system. This system is expected to run on mobile devices as part of telehealth system. HC measurement can be done with object detection method, followed by edge detection, then using every edge pixel, fetal head can be approximated using ellipse fitting. Evaluations are carried out using hit rates and error rates for ellipse fitting. From each method that was tested, the evaluation result showed that the Adaptive Boosting and Fast Ellipse Fitting (ElliFit) method had the best performance. This method also had a relatively fast execution time for a mobile device, which is 3-5 seconds.
Keywords— USG, Fetal Head Circumference, Adaptive Boosting, ElliFit.
I. INTRODUCTION
One indicator of the achievement of efforts to improve children's health is by reducing the mortality rate for children, infants and toddlers. Based on data presented by the Ministry of Health in 2018, the infant mortality rate (IMR) reached 24 deaths for every 1,000 live births (KH). The results of the inter-census population survey (SUPAS), the maternal mortality rate (MMR) reaches 305 for every 100,000 KH. One of the SDG goals is to reduce the maternal mortality rate to 102 per 100,000 KH [1].
Maternal mortality is strongly associated with neonatal mortality. Pregnancy complications tend to be the cause of stillbirths and infant mortality. One of the factors that cause the high mortality rate during pregnancy is Intrauterine Growth Restriction (IUGR). This disorder is a condition in which fetal growth is stunted. IUGR can actually be prevented if detected early and treated properly.
In order to prevent maternal and neonatal mortality, one way is to provide good health care facilities to trained health workers, such as midwives and obstetricians. Mothers who are at high risk and diagnosed with complications should have access to these health facilities. So that health workers can detect early if there are abnormalities in pregnancy.
Provision of antenatal care (ANC) expands opportunities to educate and intervene mothers to give birth in health facilities. WHO makes guidelines related to nutritional recommendations, examination of mother and fetus, and improvement of ANC. The guidelines recommend that pregnant women should be in contact with health workers at least eight times [2].
Early detection of pregnancy complications can be done using Ultrasonography (USG). This tool is able to diagnose precisely and accurately. Ultrasound is used as screening and diagnosis in antenatal care for pregnant women. The results of the study [2] prove that the use of ultrasound can detect pregnancy complications with sensitivity and specificity values above 90 percent. In addition, midwives who use ultrasound devices have twice the ability to detect abnormalities when compared to midwives who do not use ultrasound devices.
However, the number of USG instruments in Indonesia is very limited. This is because the price is very expensive and has a large enough size. So that the distribution of this tool is still evenly distributed in urban areas. In addition, the job of using an ultrasound device is not a simple job. Although this tool has high precision and accuracy, the process is still time consuming [3]. Pregnant women in rural areas do not necessarily have access to control using ultrasound devices.
One way to solve this problem is to create a tele-ultrasound system. This system can help medical personnel in remote areas consult more skilled medical personnel in more developed areas. The ultrasound instrument is also implanted with a fetal biometric measurement algorithm to speed up the measurement process. This algorithm can help medical personnel get the results of an analysis of the state of the fetus.
This system is expected to help pregnant women in rural areas get access to fetal condition control easily.
This research focuses on developing an automatic fetal head measurement algorithm. This research is expected to be used in the development of the tele-ultrasound system. To create a telehealth system, the system must be portable. This research will conduct performance testing on mobile devices.
This device certainly does not have great computing power.
So looking for a fast and accurate algorithm is the focus of this research.
II. METHODOLOGY
This section explains research methodology of the paper.
This section explains smart ultrasonography (USG) telehealth system using Fast Ellipse Fitting running on Android smartphone.
A. Tele-USG
Telehealth is the distribution of health-related service and information via electronic information and tele- communication technology. It allows long-distant patient and clinician to contact, care, reminder, etc. One way to solve the problem defined in introduction is to create a Tele-USG system. This system can help patient or medical personnel in remote areas to consult to more skilled medical personnel in more developed areas.
The ultrasound instrument is also implanted with a fetal biometric measurement algorithm to speed up the measurement process. This algorithm can help medical personnel get the results of an analysis of the state of the fetus.
This system is expected to help pregnant women in rural areas get access to control the condition of the fetal easily.
B. Fast Ellipse Fitting
Fast Ellipse Fitting (ElliFit) is an ellipse fitting method that uses geometric distances. Unlike the Hough Transform which looks for five ellipse parameters using an iterative method to obtain optimal results. The iteration process is very time consuming, so it is computationally expensive. ElliFit separates non-linear problems with ellipse fitting in such a way that they are non-iterative, numerically stable, and computationally cheap [3,6]. In general, ElliFit is represented in the model as follows:
ڭ ڭ ڭ
ݔଶ ʹݔݕ െʹݔ
ڭ ڭ ڭ
ڭ ڭ
െʹݕ െͳ
ڭ ڭ
൩ ۏ ێ ێ ێ ۍԄଵ
Ԅଶ Ԅଷ Ԅସ Ԅହے
ۑ ۑ ۑ ې
ൌ ڭ
െݕଶ ڭ
൩ (1)
where ݔ and ݕ are the edge points of the pixel and Ԅଵ, Ԅଶ, Ԅଷ, Ԅସ, and Ԅହ are intermediate variables. With an intermediate variable we can find the minor radius of a and major b, the center of the ݔ and ݕ ellipses, and the orientation of the ߠ with the following formula:
ݔൌ ሺ߶ଷെ ߶ସ߶ଶሻȀሺ߶ଵെ ߶ଶଶሻ (2) ݕൌ ሺ߶ଵ߶ସെ ߶ଷ߶ଶሻȀሺ߶ଵെ ߶ଶଶሻ (3) ߠൌ ͲǤͷ כ ܽݎܿݐܽ݊ሺʹ߶ଶሻȀሺ߶ଵെ ͳሻ (4)
ܽൌ ඨ ʹሺ߶ହ ݕଶ ݔଶ߶ଵ ʹ߶ଶሻ
ሺͳ ߶ଵሻ െ ඥሺͳ െ ߶ଵሻଶ Ͷ߶ଶଶ (5)
ܾൌ ඨ ʹሺ߶ହ ݕଶ ݔଶ߶ଵ ʹ߶ଶሻ
ሺͳ ߶ଵሻ ඥሺͳ െ ߶ଵሻଶ Ͷ߶ଶଶ (6)
C. Proposed Method
Diagram of the proposed method is shown in figure 2. The implementation of the system starts by taking the input image.
The input of the proposed method is 2-dimensional ultrasound image consisting fetal head.
The proposed method has 2 ways of pre-processing the fetal head image. If the implementation does not use object detection, the input image will be processed immediately at the edge detection stage. In the case of implementation with object detection, this process will be carried out with existing classifiers and features.
Later, one classifier and one feature will be selected per experimental scenario. Classifiers being used in this research are Random Forest [11] and Adaptive Boosting [5,12], while using either Non-Redundant Binary Local Pattern (NRLBP) [10] or Haar-like feature [5]. After obtaining the region of interest (ROI) from the image, then edge detection can be done with the Canny Edge Detection algorithm [13]. This object detection process is expected to reduce noise in the image by taking an important part of the image.
From the edge image that has been obtained, the implementation of ellipse fitting can be done to get the right ellipse. Ellipse fitting can be done with Hough Transform [9]
or ElliFit [3]. Finally, given the attributes possessed by the ellipse, the fetal head size can be measured.
III. EXPERIMENT AND ANALYSIS
This section explains dataset, experiment setup, experiment result and analysis.
Fig. 1. Smart Tele-USG System.
Fig. 2. Proposed Method.
A. Dataset
The dataset used in this paper obtained from the HC Grand Challenge [8]. The dataset contains 2-dimensional ultrasound images containing the fetal head. The images obtained are 1334 images. The channel of the image obtained is grayscale.
Each image has a size of 800 x 540 pixels. The image obtained has a .png extension.
The images that have been collected will be divided into two, the train image and the test image. There are 999 train images and 335 test images. For each train image, a ground truth image is provided which is an elliptical annotation on the train image. Also the dataset provided a file with .csv extension that stores pixel size information to be converted into millimeters. Annotation of the fetal head dataset on the HC Grand Challenge website was performed by sonographers [7].
A total of 680 images from the train image were used for this study. The image is divided into two, a positive image and a negative image. A positive image is an image of the fetal head, while a negative image is an image that is not the head of the fetus. Negative images may contain 20% to 40% of the fetal head, but it cannot be said fetal head as a whole. Among 680 images, 250 were augmented to add to the dataset for object detection training process. So that we get 930 positive images and 668 negative images.
B. Experiment Setup
The implementation of this research was carried out using a laptop with the following specifications: Intel Core i7- 7700HQ, 16 GB RAM, and Ubuntu 20.04 operating system;
and emulation of mobile devices with the following
specifications: Qualcomm SDM670 Snapdragon 670, 4 GB RAM, and the Android Pie 9.0 operating system. Any work like data pre-processing and classifier training being done with laptop. Later, performance measurement will be done using both devices.
This research is using OpenCV library for image processing and machine learning. The experiment setup for Object detection method is shown in table below. The parameter not set in the table is using default value from OpenCV.
C. Experiment Result and Analysis
Evaluations are measured using hit rate and error rate.
Ellipse fitting are being hit if the label being compared to ground truth is more than 0.8. In other words, we evaluate its intersection over union (IoU). Each IoU value will be used to measure mean of IoU (mIoU) from each ellipse fitting method.
In the hit rate evaluation, it can be seen in the table that the ElliFit method is better than Hough Transform. This is because at the edge detection stage, the edge image that is obtained often loses a lot of information. With the given edge image, the ellipse fitting method is expected to provide an approximation of the ellipse shape that is closer to ground truth. In the case of Hough Transform, there is not enough information available, so when iterating for each pixel, the results given do not approach ground truth.
Figure 4 and 5 shows visual comparison between object detection method and Hough Transform and ElliFit method, respectively. We can see from the figure that each method has it flaws. The error rate evaluation will be measured by comparing the difference of label size with ground truth. The size will be converted in centimeter. This will be done by converting pixel size to milimeter by using information from .csv file before converting it into centimeter.
For error rate evaluation, the Hough Transform method without object detection has the best results. This is because
Fig. 3. Ultrasound Dataset.
TABLE I. CONFIGURATION FO OPENCVRTREES
Configuration Value
Maximum Iteration 50
Epsilon 1
Termination Criteria TERM CRITERIA MAX ITER
Depth Tree 10
TABLE II. CONFIGURATION FO OPENCVBOOST
Configuration Value
Types REAL
Weak Counts 15
Depth Tree 10
TABLE III. HIT RATE TEST RESULTS Ellipse
Fitting Object
Detection Hit Rate (%) mIoU (%)
ElliFit - 0.26 0.54
NRLBP +
RTrees 0.55 0.66
NRLBP +
Boost 0.53 0.67
Haar +
RTrees 0.51 0.61
Haar + Boost 0.56 0.68
Hough
Transform - 0.12 0.63
NRLBP + RTrees
0.08 0.48
NRLBP + Boost
0.06 0.4
Haar +
RTrees 0.05 0.42
Haar + Boost 0.06 0.43
Hough Transform can check whether a pixel is part of the fetal head's ellipse. In contrast to ElliFit, which calculates the noise that has the potential to damage the ellipse's approximation.
Keep in mind that ElliFit method with Adaptive Boosting and the Haar-like feature has the highest hit rate. When viewed from the error rate, it is not that big and can still be developed.
This method is preferred in the development of an automatic fetal head measurement system.
Performance measurement is done by measuring the program execution time in each device. For performance measurement, we will ignore object detection method. This due to optimization from OpenCV, each method does not have significance difference for execution time.
Based on table 5, ElliFit method has a faster execution time compared to the Hough Transform. This is because Hough Transform is an iterative method. In order to find the best ellipse, Hough Transform needs to examine each pixel of the edge. This of course takes computation time. In contrast to ElliFit which already has an ellipse fitting model. This method can do elliptical approximation using the least square method.
IV. CONCLUSION
In this research, we proposed ElliFit for automatic fetal head approximation system running on Android smartphone.
Based on the experiment, the proposed method had about 3-5 seconds execution time, which is fast enough for mobile device.
ACKNOWLEDGMENT
Author would like to express gratitude for the grant KRUPT RISTEK/BRIN entitled "Pengembangan Integrated Smart Portable Telehealth USG System untuk Meningkatkan Indeks Kesehatan Ibu dan Bayi di Indonesia" year 2020 Number:NKB-2/UN2.RST/HKP.05.00/2020
REFERENCES
[1] Kemenkes, Profil Kesehatan Indonesia Tahun 2018. Kementrian Kesehatan Republik Indonesia, 2018.
[2] I. Tarigan, T. Rosita, T. Afifah, and S. Kosen, “Peningkatan kunjungan ibu hamil di puskesmas sebagai efek digunakannya ultrasound pada pemeriksaan kehamilan,” Jurnal Kesehatan Reproduksi, vol. 10, no. 2, pp. 141–152, 2019.
[3] J. Li, Y. Wang, B. Lei, J. Cheng, J. Qin, T. Wang, S. Li, and D. Ni,
“Automatic fetal head circumference measurement in ultrasound using random forest and fast ellipse fitting,” IEEE Journal of Biomedical and Health Informatics, vol. 22, pp. 215–223, Jan 2018.
[4] M. A. Ma’sum, W. Jatmiko, M. I. Tawakal, and F. A. Afif, “Automatic fetal organs detection and approximation in ultrasound image using boosting classifier and hough transform,” in 2014 International Conference on Advanced Computer Science and Information System, pp. 460–467, Oct 2014.
[5] P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, pp. I–I, Dec 2001.
[6] D. Prasad, K. Leung, and C. Quek, “Ellifit: An unconstrained, non- iterative, least squares based geometric ellipse fitting method,” Pattern Recognition, vol. 46, 01 2012.
[7] T. L. A. van den Heuvel, D. de Bruijn, C. L. de Korte, and B. van Ginneken, “Automated measurement of fetal head circumference using 2D ultrasound images,” July 2018.
[8] T. L. A. van den Heuvel, D. de Bruijn, C. L. de Korte, and B. van Ginneken, “Automated measurement of fetal head circumference using 2D ultrasound images,” July 2018. [Dataset]. Available : http://doi.org/10.5281/zenodo.1322001
[9] Yonghong Xie and Qiang Ji, “A new efficient ellipse detection method,” in Object recognition supported by user interaction for service robots, vol. 2, pp. 957–960 vol.2, 2002.
Fig. 4. Visual comparison for each object detection method.
Fig. 5. Visual comparison between HT and ElliFit after Edge Detection.
TABLE IV. ERROR RATE FOR HCAPPROXIMATION Ellipse
Fitting Object
Detection Mean
(cm) StDev
(cm) RMSE
(cm)
ElliFit - 10.4 19.7 22.3
NRLBP +
RTrees 4.1 6.5 7.7
NRLBP +
Boost 4.2 7.0 8.2
Haar +
RTrees 5.5 8.4 10.1
Haar + Boost
3.8 6.0 7.1
Hough Transform
- 2.2 1.5 2.7
NRLBP +
RTrees 4.2 2.4 4.9
NRLBP +
Boost 4.4 2.4 5.0
Haar + RTrees
4.7 2.7 5.5
Haar + Boost
4.3 2.4 5.0
TABLE V. PERFORMANCE MEASUREMENT Ellipse Fitting Laptop (seconds) Mobile (seconds)
ElliFit 1 - 2 3 – 5
Hough Transform 2 - 7 10 – 30
[10] D. T. Nguyen, Z. Zong, P. Ogunbona, and W. Li, “Object detection using non-redundant local binary patterns,” in 2010 IEEE International Conference on Image Processing, pp. 4609–4612, Sep. 2010.
[11] L. Breiman, “Random forests,” Machine Learning, vol. 45, pp. 5–32, Oct 2001.
[12] T. Chengsheng, L. Huacheng, and X. Bing, “Adaboost typical algorithm and its application research,” MATEC Web of Conferences, vol. 139, p. 00222, 01 2017.
[13] J. Canny, “A computational approach to edge detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.
PAMI-8, no. 6, pp. 679–698, 1986.
