Intelligent Medical Imaging for Breast Cancer Detection and Diabetic Retinopathy

(1)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Intelligent Medical Imaging for

Breast Cancer Detection and

Diabetic Retinopathy

Hanung Adi Nugroho

Inovasi E-Health and Biomedika untuk Indonesia

Research and Development of Intelligent Medical Imaging

Profile

Dr. Ir. Hanung Adi Nugroho

Department of Electrical Engineering and Information Technology Faculty of Engineering, UNIVERSITAS GADJAH MADA

Jl. Grafika 2, Kampus UGM, Yogyakarta 55281, Indonesia Telp./ fax. +62-274-552305

Email: adinugroho@ugm.ac.id; adinugroho@ieee.org

Research areas: Biomedical signal and image processing and analysis; computer vision; medical instrumentation; pattern recognition; data mining; statistical data analysis.

Bachelor of Engineering (S.T.) – Teknik Elektro, Universitas Gadjah Mada, Yogyakarta, Indonesia (2001)

Master of Engineering (M.E.) – School of Information Technology and Electrical Engineering, The University of Queensland, St Lucia, Brisbane, Australia (2005)

(2)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Intelligent Medical Imaging Research

Medical imaging - Overview

Currently medical imaging is limited to the acquisition of images of the human organs/ body

Medical imaging refers to the techniques and processes used to create images of the human body for clinical purposes (medical procedures seeking to reveal, diagnose or examine disease).

Medical imaging can be seen as the solution of mathematical inverse problems. This means that cause (the properties of living tissue) is inferred from

effect (the observed signal) Analysis of the images obtained _{is performed clinically by experts}

Intelligent Medical Imaging Research

Medical imaging - Technology

Gamma ray : positron emission tomography (PET)

X ray : computed tomography (CT)

a short-lived isotope, such as 18F, is incorporated into a substance used by the body such as glucose which is absorbed by the tumour of interest

(3)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Magnetic resonance imaging (MRI)

uses powerful magnets to polarise and excite hydrogen nuclei (single proton) in water molecules in human tissue, producing a detectable signal which is spatially encoded resulting in images of the body

excellent soft-tissue contrast

no known long term effects of exposure to strong static fields

health risks associated with tissue heating from exposure to the RF field and the presence of implanted devices in the body, such as pace makers

Intelligent Medical Imaging Research

Medical imaging - Technology

Ultrasound : ultrasonography

Fundus camera

Retinal image

(4)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Issues, challenge and approach

Issues

• Harmful (radiation, contrast agent) • Specialized device – difficult to use

-highly trained operator needed • Expensive (Initial cost, Maintenance) • Image Acquisition only, little or no

analysis for diagnostic purposes, subjective

Issues

• Harmful (radiation, contrast agent) • Specialized device – difficult to use

-highly trained operator needed • Expensive (Initial cost, Maintenance) • Image Acquisition only, little or no

analysis for diagnostic purposes, subjective

Approach

From medical imaging (image acquisition with enhancement) to medical image analysis (feature extraction, classification, pattern recognition, measurements) resulting in intelligent imaging (decision support systems)

Approach

From medical imaging (image acquisition with enhancement) to medical image analysis (feature extraction, classification, pattern recognition, measurements) resulting in intelligent imaging (decision support systems)

Challenge

To develop intelligent medical imaging system which is objective in analysis that is safe to the patients.

Challenge

To develop intelligent medical imaging system which is objective in analysis that is safe to the patients.

Current research in intelligent medical

imaging system at DTETI-UGM

Ophthalmology (Diabetic retinopathy) Radiology (Breast cancer)

(5)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Intelligent Medical Imaging Research in Breast Cancer

Intelligent Medical Imaging Research in Breast Cancer

Breast Cancer

(6)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Breast Cancer Detection

 No Radiation

 More Detail  Expensive

 Limited availability

 Low cost

 Short acquisition time

 No radiations

 High availability

 Convenient

 more sensitive  Depend on operator

 Radiologist s experience

 Inconsistency of interpretation

Breast compression

 Low-dose X-ray  Just for particular patient

 Limited availability

Breast Self

Exam Mammograms

USG MRI

• Elaborate the radiology knowledge into image processing and analysis technology

• Assist radiologists to diagnose nodule

CAD

CAD : Computer Aided Diagnosis

Research Objective

(7)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Diagnosis of Breast Cancer using Ultrasound

A breast ultrasound is a scan that uses penetrating sound waves that

do not affect or damage the tissue and cannot be heard by humans.

Normal

Abnormal

Intelligent Medical Imaging Research in Breast Cancer

(8)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Image

Acquisitions

Image Processing

Analysis

Display

Image

Radiologists

Computer Aided System

Diagnosis

USG

Image

Scheme of CAD System

Intelligent Medical Imaging Research in Breast Cancer

USG

Images RoI (1)

Noise and Marker

Reduction(3) Segmentation

(4)

Feature Selection(6)

Malignant / Benign

Preprocessing

GrayScale Conversion (2)

Feature Extraction (5)

• Moment based

features

• Geometry Feature

• Texture Feature Birads based

Classification (7)

Diagnosis (8)

(9)

Nodule

Background

Segmented Area

Posterior Characteristic Margin Characteristic Echo Pattern Characteristic

Texture Features

Texture Analysis

Intelligent Medical Imaging Research in Breast Cancer

Geometry Features

Geometric feature is constructed by a set of geometrical elements such as

points, lines, curves or surfaces

=

1

∑

. exp ( 2

)

=

(10)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Geometry and Moment Based Features

Nodule

Background

Shape characteristics

Margin characteristics

Moment Based Analysis

Geometry Analysis

Intelligent Medical Imaging Research in Breast Cancer

Research Roadmap

2014

2015

Shape and

Boundary

Echo

Pattern

Prototype 1

2016

Margin and

Posterior Features

Prototype 2

Clinical

Trial

Integrated

Modules

(11)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Results

Intelligent Medical Imaging Research in Breast Cancer

Unmarked Hypoechoic or Hypoechoic

• Round - Oval

• Irregular

• Circumscribed

• Circumscribed • Not Circumscribed

• Not Circumscribed

Benign Malignant

(12)

Image

Capturing

ROI and Filtering

Segmentation and

Feature Extraction

Diagnosis

i-Brids Prototype.1

Intelligent Medical Imaging Research in Breast Cancer

0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00%

Accuracy Sensitivity Specificity PPV NPV Shape 96.20% 94.70% 97.90% 94.73% 97.91% Margin 80.90% 79.50% 82.50% 78.50% 82.50%

(13)

No

Statistical Analysis

Malignant

Diagnosis

Benign

1

Number of Features Agreement

19

13

2

Number of Features due to Chance

12.5

6.5

3

Total Number of Subjects

38

4

Total Number of Agreement

32

5

Number of Agreement due to chance

19

6

Kappa

0.68

Statistical Analysis

Kappa statistics are commonly used to indicate the degree of agreement of nominal assessments made by multiple appraisers.

A Kappa0.68is in the“substantial” agreementrange between radiologists and CAD system.

Intelligent Medical Imaging Research in Breast Cancer

Unmarked Hypoechoic or Hypoechoic

• Circumscribed

• Not Circumscribed

• No Posterior Feature • Enhancement

Benign/Malignant Benign

• Shadowing Malignant

• No Posterior Feature • Enhancement

Malignant Malignant

• Shadowing Malignant

(14)

Image

Capturing

ROI and Filtering

Feature Extraction

Segmentation and

Diagnosis

i-Brids Prototype 2

Intelligent Medical Imaging Research in Breast Cancer

Accuracy of CAD System

76.00% 78.00% 80.00% 82.00% 84.00% 86.00% 88.00% 90.00% 92.00% 94.00% 96.00% 98.00%

(15)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

92% 93% 94% 95% 96% 97% 98% 99% 100%

Sensitivity Specificity PPV NPV Radiologist 1 100% 94.74% 95% 100% Radiologist 2 100% 94.74% 95% 100%

Performance Analysis of CAD System

Intelligent Medical Imaging Research in Breast Cancer

No Statistical Analysis Margin Posterior Diagnosis

Circumscribed Indistinct Enhancement _{Posterior Shadow Malignant Benign}No

1 Number of Features Agreement 22 16 21 3 9 19 19

2 Number of Features due to Chance 12.74 6.74 11.6 0.79 3.47 9.5 9.5

3 Total Number of Subjects 38 38 38

4 Total Number of Agreement 38 33 38

5 Number of Agreement due to chance 19.47 19 15.87

6 Cohen's Kappa 1 0.774 1

Statistical Analysis

A Kappa

1

is in the

“perfect” agreement

range between two radiologist

(16)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Video of i-Brids

Intelligent Medical Imaging Research in Breast Cancer

Potential Market

Health Hospitals/Clinics Puskesmas 1380 1599

3451

(17)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Recognition

[1] H. A. Nugroho, N. Faisal, I. Soesanti, and L. Choridah, “Analysis of Computer Aided Diagnosis on Digital Mammogram Images,” in2014 International Conference on Computer, Control, Informatics and Its Applications Analysis, 2014, pp. 25–29.

[2] A. Nugroho, H. A. Nugroho, and L. Choridah, “Active Contour Bilateral Filter for Breast Lesions Segmentation on Ultrasound Images,” in2015 International Conference on Science in Information Technology (ICSITech) Active, 2015, pp. 36–40.

[3] H. A. Nugroho, Y. Triyani, M. Rahmawaty, , I. Ardiyanto ,and L. Choridah, “Performance Analysis of Filtering Techniques for Speckle Reduction on Breast Ultrasound Images,” in2016 International Electronics Symposium (IES), 2016, pp. 454–458.

[4] M. Rahmawaty, H. A. Nugroho, Y. Triyani, I. Ardiyanto, and I. Soesanti, “Classification of Breast Ultrasound Images based on Texture Analysis,” in

iBioMed 2016, 2016, pp. 84–89.

[5] Y. Triyani, H. A. Nugroho, M. Rahmawaty, I. Ardiyanto, and L. Choridah, “Performance Analysis of Image Segmentation for Breast Ultrasound Images,” inICITEE 2016, 2016, no. October, pp. 415–420.

[6] H. K. N. Yusufiyah, H. A. Nugroho, T. B. Adji, and A. Nugroho, “Feature Extraction for Classifying Lesion ’ s Shape of Breast Ultrasound Images,”2nd Int. Conf. Inf. Technol. Comput. Electr. Eng., pp. 105–109, 2015.

[7] H. A. Nugroho, H. Khuzaimah, N. Yusufiyah, T. B. Adji, and A. Nugroho, “Zernike Moment Feature Extraction for Classifying Lesion ’ s Shape of Breast Ultrasound Images,” in7th International Conference on Information Technology and Electrical Engineering (ICITEE), 2015, pp. 458–463.

[8] H. A. Nugroho, N. Faisal, I. Soesanti, and L. Choridah, “Identification of Malignant Masses on Digital Mammogram Images based on Texture Feature and Correlation based Feature Selection Hanung,” in6th International Conference on Information Technology and Electrical Engineering (ICITEE), 2014.

[9] H.R. Fajrin, H. A. Nugroho, and I. Soesanti“Ekstraksi Ciri Berbasis Wavelet Dan Glcm Untuk Deteksi Dini Kanker Payudara Pada Citra Mammogram,” inSNST, 2015, pp. 47–52.

[10] M. Sahar, H. A. Nugroho, Tanur, I. Ardiyanto, and L. Choridah “Automated Detection of Breast Cancer Lesions Using Adaptive Thresholding and Morphological Operation,” inInternational Conference on Information Technology Systems and Innovation (ICITSI), 2016.

[11] Tianur, H. A. Nugroho, M. Sahar, R. Indrastuti, and L. Choridah, “Classification of Breast Ultrasound Images based on Posterior Feature,” in

International Conference on Information Technology Systems and Innovation (ICITSI), 2016.

Breast Cancer :

Intelligent Medical Imaging Research in Breast Cancer

Team Members and Collaborators

Department Electrical Engineering and Information Technology Faculty of Engineering Universitas Gadjah Mada

Department of Radiology Sardjito Hospital, Yogyakarta

• H A Nugroho

• I Ardiyanto

• M Rahmawaty

• Y Triyani

• M Sahar

• L Choridah

• R. Indrastuti

• A. Mardhiah

• Tianur

• A Nugroho

• D A Husna

• H Khuzaimah

(18)

Intelligent Medical Imaging Research in Diabetic Retinopathy

What is diabetic retinopathy

TYPE 1 DIABETES: when the pancreas

doesn’t produce insulin

TYPE 1 DIABETES: when the pancreas

doesn’t produce enough insulin (or the insulin cannot be processed)

GESTATIONAL DIABETES: when the

insulin is less effective during pregnancy

your body needs insulin to transform glucose into energy

(19)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

What is diabetic retinopathy

Diabetic Nephropathy Diabetic Neuropathy

Diabetic Cardiomyopathy Diabetic Retinopathy

(DR)

DR is retinopathy (damage to the retina) caused by complications of diabetes mellitus, which could

eventually lead to blindness. Fact :

- Nearly all patients of type-1 diabetes and 60% of patients of type-2 diabetes indicate retinopathy. - DR is the leading cause of the blindness in

developing countries among adults aged 20-74 years.

Normal vision DR vision

Intelligent Medical Imaging Research in Diabetic Retinopathy

Diabetes : fact and figures

“Worldwide”

(20)

The pathologies of DR

new blood vessels

Haemorrhages

exudates

Micro aneurysms

Moderate NPDR

Mild NPDR

No DR

Proliferative DR Severe NPDR

Issues, challenges and approaches

Issues

Diabetes mellitus affect ~10% population (DR is a real concern -epidemic stage?)

Needs access to ophthalmologist with fundus camera equipment

Low contrast Fundus images requiring Fluorescein angiography -an invasive procedure

Challenges

1. Can we develop a screening & grading system to be made accessible to all diabetes patients?

2. Can we detect DR early even before patient have visual problems?

3. Can we make non-invasive procedure as effective?

(21)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Haemorrhages detection

Enhancement

Enhancement Haemorrhages candidatesHaemorrhages candidates Detected HaemorrhagesDetected Haemorrhages

Pre-processing

Green and V band

extraction Histogram matching Opening operation Contrast

enhancement _Haemorrhages

candidate detection Post-processing

Retinal vessels

detection Retinal vessels elimination Double length

filtering operationMasking Two-dimensional

matched filtering

Fundus image

Intelligent Medical Imaging Research in Diabetic Retinopathy

Hard exudates detection

Fundus image Filtered image

Hard ExudatesDetected Hard Exudates

Green channel

extraction Complement operation Matched filter

Optic disc (OD) detection[1]

Removal OD and Morphological

(22)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

2015

2016

2017

2018

DR pathologies

detection DR screening system DR monitoring and grading system Micro aneurysms detection

Optic disc detectionMacula detection

Analysis of DR pathologies

Haemorrhages and hard exudates detection Clinical study System evaluation FAZ detection Classification

Research roadmap

General: to develop a system to assist the ophthalmologists in monitoring and diagnosing diabetic retinopathy disease.

First year: to develop algorithms in each module to detect structures and pathologies in DR retinal image.

Second year: to integrate the modules and develop an algorithm for screening DR system.

Third year: to test the system based on clinical study for monitoring and grading system.

Intelligent Medical Imaging Research in Diabetic Retinopathy

Recognition



International Conferences

[1] H. A. Nugroho, D. A. Dharmawan, I. Hidayah, and L. Listyalina, "Automated microaneurysms (MAs) detection in digital colour fundus images using matched filter," in Computer, Control, Informatics and its Applications (IC3INA), 2015 International Conference on, 2015, pp. 104-108.

[2] H. A. Nugroho, L. Listyalina, N. A. Setiawan, S. Wibirama, and D. A. Dharmawan, "Automated segmentation of optic disc area using mathematical morphology and active contour," in Computer, Control, Informatics and its Applications (IC3INA), 2015 International Conference on, 2015, pp. 18-22.

[3] H. A. Nugroho, D. Purnamasari, I. Soesanti, K. W. Oktoeberza, and D. A. Dharmawan, "Detection of foveal avascular zone in colour retinal fundus images," in 2015 International Conference on Science in Information Technology (ICSITech), 2015, pp. 225-230.

[4] H. A. Nugroho, K. W. Oktoeberza, T. B. Adji, and M. B. Sasongko, "Segmentation of exudates based on high pass filtering in retinal fundus images," in 2015 7th International Conference on Information Technology and Electrical Engineering (ICITEE), 2015, pp. 436-441.

(23)

adinugroho@ugm.ac.id adinugroho@ugm.ac.id

Recognition



Journals

[1] H. A. Nugroho, K. W. Oktoeberza, T. B. Adji, and F. Najamuddin, "Detection of Exudates on Color Fundus Images using Texture Based Feature Extraction," International Journal of Technology, vol. 6, p. 04, 2015.

[2] H.A. Nugroho, D.A. Dharmawan, and L. Listyalina, "Automated Segmentation of Foveal Avascular Zone (FAZ) in Digital Colour Retinal Fundus Images," International journal of biomedical engineering and technology, 2016.

Intelligent Medical Imaging Research in Diabetic Retinopathy

Team members and Collaborator

Rapid Assessment Diabetic Retinopathy and Intelligent System Research Groups Department of Electrical Engineering and Information Technology, Faculty of Engineering Universitas Gadjah Mada, Indonesia

(Hanung Adi Nugroho, Noor Akhmad Setiawan, Teguh Bharata Adji, Indriana Hidayah, Igi Ardiyanto, Ratna Lestari Budiani Buana, Dhimas Arief Dharmawan, Latifah Listyalina, Dewi Purnamasari, Widhia Oktoeberza KZ)

Department of Ophthalmology, Sardjito Hospital, Yogyakarta, Indonesia

(24)