Intelligent Medical Imaging

(1)

Intelligent Medical Imaging

Prof. Dr. Ir. Ahmad Fadzil M. H.

Intelligent Signal & Imaging Research Cluster Department of Electrical & Electronics Engineering Universiti Teknologi PETRONAS

Thursday, 20 June 2008

© 2008 INSTITUTE OF TECHNOLOGY PETRONAS SDN BHD All rights reserved. No part of this document may be reproduced, stored in a retrieval system or transmitted in any form or by any means (electronic, mechanical, photocopying, recording or otherwise) without the permission of the copyright owner.

Universiti Teknologi PETRONAS

www.utp.edu.my

(2)

UTP

Outline

Intelligent Signal and Image Processing Research Cluster 1

Intelligent Medical Imaging Research

Intelligent Medical Imaging Research in Vitiligo 2

3

Intelligent Medical Imaging Research in Psoriasis 4

Intelligent Medical Imaging Research in Diabetic Retinopathy 5

(3)

The Intelligent Signal and Imaging Research Group

The Intelligent Signal and Imaging Research Group is one of the leading intelligent signal and imaging research cluster in Malaysia.

Digital elevation model analysis (Remote sensing)

Research

Seismic data analysis (Geoscience)

Medical imaging l i Medical signal

anal sis

areas

^analysis

(Skin and eye-related diseases) analysis

(Heart diseases,VEP)

Approaches

Colour space analysis Independent

Component Analysis Principal

Component Analysis Morpholog

ical filter Fractal

analysis Computer Vision, Signal and Image Processing Computer Vision, Signal and Image Processing

(4)

Outline

Intelligent Medical Imaging Research 2

1

Intelligent Signal and Image Processing Research Cluster

Medical Imaging - Overview

Medical imaging refers to the techniques and processes used to Currently medical imaging is limited to the acquisition of images of the human organs/ body

create images of the human body for clinical purposes (medical procedures seeking to reveal, diagnose or examine disease).

Medical imaging can be seen as the solution of mathematical inverse problems. This means that cause (the properties of living tissue) is inferred from effect (the observed signal)

Analysis of the images obtained is performed clinically by experts

(5)

Medical Imaging - Technology

Gamma ray : positron emission tomography (PET) a short-lived isotope, such as 18F, is incorporated into a substance used by the body such as glucose

X ray : computed tomography (CT)

which is absorbed by the tumor of interest

Expose to x-ray radiation, repeated scans must be limited to avoid health effects

Magnetic Resonance Imaging (MRI)

Medical Imaging - Technology

uses powerful magnets to polarise and excite hydrogen nuclei (single proton) in y g ( g p ) water molecules in human tissue, producing a detectable signal which is spatially encoded resulting in images of the body

excellent soft-tissue contrast

no known long term effects of exposure to strong static fields

health risks associated with tissue heating from exposure to the RF field and the presence of implanted devices in the body, such as pace makers

(6)

Medical Imaging - Technology

Ultrasound : ultrasonography H-F sound, 2-10MHz, safe, 2D moving images

Visible light : camera

Issues, Challenge and Approach

Issues

• Harmful (radiation, contrast agent) Challenges

T d l i t lli t di l

• Specialized device – difficult to use - highly trained operator needed

• Expensive (Initial cost, Maintenance)

• Image Acquisition only, little or no analysis for diagnostic purposes, subjective

To develop intelligent medical imaging system which is objective in analysis that is safe to the patients.

g p p , j

Approach

From medical imaging (image acquisition with enhancement) to medical image analysis (feature extraction, classification, pattern recognition, measurements) resulting in intelligent imaging (decision support systems)

(7)

Ophthalmology (Eye)

Current Research in Intelligent Medical Imaging at UTP

Dermatology (Skin)

Vitiligo

Diabetic

R i h

Psoriasis

Retinopathy

Outline

1

(8)

Vitiligo - Introduction

Vitiligo is a skin disease where irregular white spots appear in the skin in various size and location. The spots occur when pigment cells (melanocytes) are damaged and the pigment melanin can no longer be produced

Melanin pigmentscan no longer be produced

Melanocytes damaged

are damaged and the pigment melanin can no longer be produced.

The effect of vitiligo can be very considerable on the patients’

psychological condition and patients with vitiligo have an increased risk of developing autoimmune diseases.

Vitiligo – Issues and Challenge

Issues

• Efficacy assessment largely dependent on the human eye and judgment to produce the scorings.

• Dermatologists find it visually hard to determine the areas of skin

repigmentation due to the slow progress and as a result the observations are made over a longer time frame

Repigmentation 3 months

treatment (ointment, phototherapy -

UV & laser) Patient 3

Challenge:

How to we develop an image analysis scheme that determines the non-melanin skin areas (corresponding to vitiligo skin areas) and repigmentation areas objectively over a shorter time frame?

(9)

Vitiligo - Approach

We use mathematical techniques to extract information of the skin histology from the digital images

95% of the incident light (250–700nm) penetrates into skin and follows a complex path until it exits back out of the skin or gets attenuated by skin choromophores.

Using principal component analysis followed by independent component analysis, we can decompose RGB skin image into melanin and haemoglobin spatially.

Principal Component Analysis (PCA)

RGB Digital Images

Vitiligo - Approach

Using the specific skin histology obtained from PCA/ICA, we can determine vitiligo areas objectively.

(PCA)

Independent Component Analysis (ICA)

Thresholding based on Euclidean distance Extracting skin

histology Original RGB image

Area Measurement

Repigmentation Measurement

Therapeutic Response Due to Treatment Melanin

component

Haemoglobin component

(10)

Vitiligo – PCA/ICA

RGB PCA ICA

To transform observed image data (R,G,B) into (maximally independent) components of skin (melanin, haemoglobin)

xR,yG,zB

‐log (b)

‐log (g)

RGB PCA ICA

xC1,yC2

‐log (r)

‐log (b)

‐log (g)

xM,yH

skin colour distribution

haemoglobin

‐log (b)

‐log (g) melanin

‐log (r)

* *

log (r)

‐log (r)

Vitiligo - Benefits

The system is currently being used in Hospital Kuala Lumpur for clinical trial in efficacy assessment of therapeutic treatment .

• The digital image analysis system has been able to determine the therapeutic response of the vitiligo treatment (repigmentation areas) objectively in a shorter time, hence the efficacy assessment of the treatment . This will enable dermatologists to accordingly perform accurate procedures in a shorter time period.

Demo

(11)

1. Hermawan Nugroho, Ahmad Fadzil M.H., “Skin Colour Segmentation using Principal Component Anaylsis”, Proceeding 3rd International Colloquium on Signal Processing and its Application, March 9 – 11, 2007, Melaka, Malaysia

Vitiligo - Reference

2. Hermawan Nugroho, Ahmad Fadzil M.H., S. Norashikin, H.H. Suraiya, “Computer Aided Vitiligo Monitoring using Independent Component Analysis” Proceeding International Conference on Biotechnology Engineering (ICBioE 2007),May 8 – 10, 2007, Kuala Lumpur, Malaysia

3. Hermawan Nugroho, Ahmad Fadzil M.H., S. Norashikin, H.H. Suraiya, “Determination of Skin Repigmentation Progression”, Proceeding 29th Annual International Conference of the IEEE (Institute of Electrical and Electronics Engineering) Engineering in Medicine and

the IEEE (Institute of Electrical and Electronics Engineering), Engineering in Medicine and Biology Society (EMBS) 2007, August 23-26, 2007, Lyon, France

4. M. H. Ahmad Fadzil, Hermawan Nugroho, S. Norashikin, H. H. Suraiya, “Assessment of Therapeutic Response in Skin Pigment Disorder Treatment”, 3rd International Symposium on Information Technology 2008 (ITSiM08),Kuala Lumpur.

Outline

1

(12)

Psoriasis - Introduction

Psoriasis is a chronic inflammatory, non- contagious skin disorder which typically consists of red plaques covered by silvery-white

Although psoriasis is an incurable disease, there are many available treatments to control the symptoms of psoriasis

consists of red plaques covered by silvery white scales

Plaque psoriasis is the most common form of psoriasis (80%)

the symptoms of psoriasis

However, there is no single treatment that works for every case.

Dermatologist should monitor the extent of psoriasis continuously to assess the treatment efficacy

PASI (Psoriasis Area & Severity Image)

It assesses four body region : head trunk PASI is the gold standard method to assess the extent of psoriasis and in treatment efficacy.

It assesses four body region : head, trunk, upper extremities, and lower extremities.

For each body region, the surface area involved, erythema(redness), thickness,

and scaliness of the lesion are determined. 2 2

1 1 3

3 4 3

1

( )

⁽ ⁾ 6

PASI=0.1 Rh+Th+Sh Ah+0.2 Ru+Tu+Su Au

5

3

4 4 4

3 4 3

24

6 37

( )

⁽ ⁾

( )

0.3 0.4

u u u u

h h h h

Rt Tt St At Rl Tl Sl Al

+ + + + + +

A = area (0 – 6), R = redness (0 – 4), T = thickness (0 – 4), S = scaliness (0 – 4), h = head, u = upper extremities, t = trunk, l = lower extremities.

The treatment is considered effective when the initial PASI score is reduced by 75 %

(13)

Psoriasis - Issues

The PASI score varies between dermatologists (Inter-observer variation) and during repeat visits by patients, inconsistent PASI scoring by the same dermatologist can occur (Intra-observer variation)

Calculating area of lesion

Low accuracy Tedious

Measuring thickness & scaliness Thickness & scaliness are determined

by tactile inspection

Inconvenience and subjective

Determining degree of redness

Degree of redness

is affected by patient’s skin colour

Objective Assessment of Psoriasis Area

The appearance of psoriasis lesion vary between each patient it is affected vary between each patient, it is affected by their normal skin colour

However, human visual system is able to identify the lesion based on colour difference with the

surrounding normal skin Challenge

How to quantify and emulate the ability of human visual system in differentiating the colour of psoriasis lesion from normal skin?

(14)

Objective Assessment of Psoriasis Area - Approach

We convert from RGB to CIELAB colour space which is linear to calculate colour difference between healthy skin and psoriasis lesion

Each colour is represented by L*, a*, and b*

L* = degree of lightness

a* = degree of greenness to redness b* = degree of blueness to yellowness

Objective Assessment of Psoriasis Area - Approach

Calculating colour difference (Hue difference (∆h_ab), chroma difference (∆C_ab) and

Lightness difference (∆L*)) between healthy skin and psoriasis lesion in the CIELAB colour

space L* = 100L* = 100L 100

+a*

+b*

-a*

P₁ P₂

L*1

L*₂

C_ab1 C_ab2 hab2 h_ab1

L 100

+a*

+b*

-a*

P₁ P₂

L*1

L*₂

C_ab1 C_ab2 hab2 h_ab1

-b*

L* = 0 -b*

L* = 0

* * *

1 2

* * *

1 2

ab ab ab

L L L

h h h

C C C

∆ = −

(15)

Objective Assessment of Psoriasis Area - Approach

Change RGB image to CIELAB image CIELAB image

Calculate colour difference Between healthy normal skin and lesion

Select samples of normal skin and lesion

Normal healthy skin

Lesion

Between healthy normal skin and lesion

Pixel classification into lesion and normal skin

Healed lesion

Objective Assessment of Psoriasis Erythema

Konica Minolta Chromameter CR-400 is an For accurate measurement of skin and lesion colour, we use the chromameter

instrument to measure colour by modeling characteristic of light source, spectral reflectance of the object, and colour vision of human eye.

Hue = dominant wavelength of a colour

Chroma = saturation of a colour

( )

1 * *

tan /

hab= ⁻ b a

* *2 *2

Cab= a +b

(16)

Objective Assessment of Psoriasis Erythema - Approach

f

Normal skin

Erythema (redness) is dependent on the patient’s normal skin tone

Patients are classified into three skin tones (dark, brown, fair) based on their L* values

35 40 45 50 55 60 65

Skin group

L*

Fair skin Brown skin Dark skin

Fair Brown Dark

Objective Assessment of Psoriasis Erythema - Approach

The human visual system perceive colour differences based on the combination of differences in the hue, saturation (chroma) and lightness. We investigate the effects of these parameters .

Fair skin

6 8 10 12 14

* score1

Erythema score linearly correlates with the hue difference (∆h_ab) for all skin tones

-4 -2 0 2 4 6

-5 0 5 10 15 20 25 30 35

∆hue

∆L*

score2 score3

(17)

Objective Assessment of Psoriasis Thickness & Scaliness

Measuring Method: Triangulation light block method

Data Acquisition :

Konica Minolta VIVID 910 N t t 3D Di iti

In order to assess thickness and scaliness , we aquire 3D images of the lesion

method

Scan Range: 0.6 to 1.0 m (In Standard mode) 0.5 to 2.5 m (In Extended mode)

Laser Scan Method: Galvanometer-driven rotating mirror

Accuracy : ±0.05 mm Non-contact 3D Digitizer

y

Precision : 0.008 mm Output Format

3D data: Konica Minolta format, & (STL,DXF, OBJ, ASCII points, VRML)

Color data: RGB 24-bit raster scan data

Objective Assessment of Psoriasis Thickness & Scaliness

Data Acquisition

3D image Psoriasis lesion

(18)

Objective Assessment of Psoriasis Thickness & Scaliness

Approach

Surfaces of the psoriasis lesions are digitized using 3D laser scanner Surfaces of the psoriasis lesions are digitized using 3D laser scanner .

Depth information of the 3D psoriasis lesion are converted into 2D grayscale image.

Achievements

The lesion segmentation method was applied on images of 8 patients with various skin colour. Our segmentation method achieves accuracies higher than 90%. The error occurs mainly at the border of the lesion due to colour gradation between normal skin occurs mainly at the border of the lesion due to colour gradation between normal skin and psoriasis lesion.

The erythema score of a lesion can be accurately determined by the hue difference (∆h_ab) within a particular skin type group.

The proposed method has the potential to assess erythema objectively and consistently without being influenced by other characteristic of the lesion such as area, pattern, and boundary.

The objective assessments of thickness and scaliness are currently being investigated

(19)

Research Collaboration (UTP-Dermatology Dept, Hospital Kuala Lumpur

Psoriasis - Reference

1. M. H. Ahmad Fadzil, Dani Ihtatho, M. A. Azura, H. H. Suraiya, “Objective Assessment of Psoriasis Erythema for PASI Scoring”, 30th Annual International Conference of the IEEE EMBS 2008, Vancouver, Canada, 2008.

2. M. H. Ahmad Fadzil, Dani Ihtatho, “Modeling Psoriasis Lesion Colour for PASI Erythema Scoring”, 3rd International Symposium on Information Technology 2008 (ITSiM08),Kuala Lumpur, Malaysia.

3. Hermawan Nugroho, Naz-e-Batool, M. H. Ahmad Fadzil, P. A. Venkatachalam, “Surface Analysis of Psoriasis for PASI Scaliness Assessment”, Proceedings International on Intelligent and Advanced Systems (ICIAS2007),Kuala Lumpur, Malaysia.

4. Dani Ihtatho, M. H. Ahmad Fadzil, M. A. Azura, H. H. Suraiya, “Automatic PASI Area Scoring”, Proceedings International on Intelligent and Advanced Systems (ICIAS2007), Kuala Lumpur, Malaysia.

5. Dani Ihtatho, M. H. Ahmad Fadzil, M. A. Azura, H. H. Suraiya, “Area Assessment of Psoriasis Lesion for PASI Scoring”, Proceedings of the 29th Annual International Conference of the IEEE EMBS 2007, Lyon, France.

(20)

Outline

1

What is Diabetic Retinopathy?

Diabetic retinopathy is retinopathy (damage to the retina) caused by

complications of diabetes mellitus, which could eventually lead to blindness.

Courtesy NIH National Eye Institute

Affects up to 80% of all diabetics who have had diabetes for 10 years or more.

After 20 years of diabetes, nearly all patients with type

At least 90% of new cases could be reduced if there was proper and vigilant treatment and monitoring

The same view with diabetic retinopathy

nearly all patients with type 1 diabetes (juvenile onset) and >60% of patients with type 2 diabetes (adult onset) have some degree of retinopathy.

of the eyes.

(21)

DR Pathologies

hemorrhages hemorrhages

Small blood vessels in the eye are especially vulnerable to poor blood sugar control.

An overaccumulation of

ModPDR (dataset2 - NPDR2)

An overaccumulation of glucose damages the tiny blood vessels in the retina.

During the initial stage, called nonproliferative diabetic retinopathy (NPDR), most people do not notice any changes in their vision.

DR Pathologies

exudates hemorrhages

Macula region

Some people develop a condition called macular edema.

It occurs when the damaged bl d l l k fl id d

Proliferative DR (dataset1 - PDR2)

blood vessels leak fluid and lipids onto the macula, the part of the retina that lets us see detail. The fluid makes the macula swell, which blurs vision.

Without timely treatment, these new blood vessels can bleed, cloud vision, and destroy the retina – PDR stage

(22)

Diabetic Retinopathy

Analysing fundus image can show severity of DR

Disease of the retina as a complication of diabetes mellitus.

Characterized by the progressive microvascular complications.

Normal

What do the doctors do?

Eye examination Visual acuity test

Ophthalmoscopy

Ocular Coherence Tomography

(1) leaking blood vessels,

(2) retinal swelling, such as macular edema

(3) pale, fatty deposits on the retina (exudates) – signs of leaking blood vessels

g p y

The ophthalmologist will look at the retina for early

signs of the disease of leaking blood vessels

(4) damaged nerve tissue (neuropathy) (5) any changes in the blood vessels.

signs of the disease

To allow the doctor to find the leaking blood vessels a test called fluorescein angiographyis performed. In this test, a special dye is injected into the arm.

(23)

Issues, Challenges, Approaches

Issues

Diabetes mellitus affect ~10%

Challenges

1 C d l i &

Diabetes mellitus affect ~10%

population (DR is a real concern - epidemic stage?)

Needs access to ophthalmologist with fundus camera equipment Grading severity of DR

1. Can we develop a screening &

grading system to be made accessible to all diabetes patients?

2. Can we detect DR early even before patient have visual problems?

3. Can we make non-invasive

Low contrast Fundus images requiring Fluorescein angiography - an invasive procedure

procedure as effective?

Fundus camera technology +

Image Processing & Computer Vision

1

Severity of DR related to retinal capillary

2

Very low contrast of retinal blood vessels in fundus

3

Fundus fluourescein angiography to Issues

Imaging Issues in Diabetic Retinopathy

occlusion image enhance the

contrast

Challenge

Original image (source: DRIVE) Fundus FA (Source: Selayang hospital)

How to enhance the contrast of blood vessels using non-invasive technique?

How to develop an intelligent system to support monitoring and grading of DR?

(24)

Computerized Diabetic Retinopathy Monitoring and Grading Systems

An intelligent system for effective screening of DR and in assisting ophthalmologist make accurate diagnosis without resorting to invasive methods.

Grading of DR severity Kowa nonmyd 7

Kowa VX 10i

Capabilities of Computerized DR System

Challenges

1.Enable effective screening

Identification of those individuals who may be Can we develop a screening

& grading system to be made accessible to all diabetes patients?

Can we detect DR early even before patient have visual

y at risk of developing DR and in assisting ophthalmologist make accurate diagnosis (identification of DR conditions by means of its pathologies and symptoms) without resorting to invasive methods.

2. Grading the severity and progression of DR before patient have visual

problems?

Can we make non-invasive procedure as effective?

Grading Normal

Mild Non Proliferative DR (NPDR) Moderate NPDR

Severe NPDR PDR

(25)

Segmentation of Retinal Blood Vessels - Approach

Fundus image Enhancement Extraction

Colour fundus image is processed and analyzed to monitor the severity of DR.

Fundus image Enhancement Extraction

Contrast limited adaptive histogram equalization (CLAHE) and mathematical morphology are used to enhance the contrast and extract retinal blood vessels.

Research & Development

To avoid invasive FA, we developed novel imaging solution to the low and varying contrast Fundus imagesy g g

a Fundus image (Green band) b H l bi l t d t a Fundus image (Green band) b H l bi l t d t

a. Fundus image after homomorphic filtering

b. First component a. Fundus image after

homomorphic filtering

b. First component

To detect DR early and monitor DR, we developed an alternative approach to the grading of DR

a. Fundus image (Green band) b. Haemoglobin-related component Figure 5. Contrast enhancement of retinal blood vessels a. Fundus image (Green band) b. Haemoglobin-related component

Figure 5. Contrast enhancement of retinal blood vessels

c. Second component d. Third component c. Second component d. Third component

(26)

Current research on contrast enhancement

Digital fundus images

Using independent component analysis, we decompose RGB fundus image into macular pigment, melanin and haemoglobin resulting in contrast enhancement of retinal blood vessels.

Digital fundus images

Red channel Green channel Blue channel

Independent component analysis (ICA)

Digital fundus image

Macular

Independent component analysis (ICA)

Macular pigment Haemoglobin Melanin

Retinal blood vessels

region

Contrast enhancement of retinal blood vessels

Achievements

Contrast enhancement of retinal blood vessels reduces the need of applying contrasting agent on patients.

System provides the map of retinal vasculature – able to detect free capillary zone and presence of new vessels growth.

Improve efficiency of DR screening and monitoring process.

The proposed method successfully enhances retinal vasculature with contrast enhancement factor of 3.15 and 3.2 using CLAHE and ICA, respectively, for digital retinal images.

(27)

R&D Roadmap for Computerised DR Monitoring &

Grading System

2007 2008

2008-2009 Clinical trial Market surveys

2010

System available in market

2005

Research started 2007

Patent Search

MOSTI Technofund grant RM1m 2007-2008

Prototype system

(R&D Collaboration -UTP, VITROX, Hospital Selayang) Patent filing

Diabetic Retinopathy - References

1. Ahmad Fadzil M H, Lila Iznita I, P A Venkatachalam, T V N Karunakar:

Extraction and reconstruction of retinal vasculature Journal of Medical Extraction and reconstruction of retinal vasculature. Journal of Medical Engineering Technology 2007, 31(6):435-442.

2. Ahmad Fadzil M H, Hanung Adi Nugroho, P A Venkatachalam, Hermawan Nugroho, Lila Iznita I: Determination of Retinal Pigments from Fundus Images using Independent Component Analysis. Proceedings Biomed 2008 Conference 4th Kuala Lumpur International Conference on Biomedical

Engineering.Kuala Lumpur: Springer; 2008.

3. Ahmad Fadzil M H, Hanung Adi Nugroho, P A Venkatachalam, Hermawan Nugroho, Lila Iznita I: Contrast Enhancement of Retinal Blood Vessels in

g

Digital Fundus Imageto be presented In: IASTED 8^thConference on VIIP 2008.Mallorca, Spain: IASTED; 2008.

(28)

Researchers

(29)

TERIMA KASIH

THANK YOU

© 2008 INSTITUTE OF TECHNOLOGY PETRONAS SDN BHD All rights reserved. No part of this document may be reproduced, stored in a retrieval system or transmitted in any form or by any means (electronic, mechanical, photocopying, recording or otherwise) without the permission of the copyright owner.