Thesis Organization

This thesis ontains ve hapters, inludingthe present one.

Chapter 2 desribes the proposed sheme for extrating disriminative features between

skin and non-skin regions of an image. For this, a novel disriminative analysis method also

termed as adaptive disriminative analysis (ADA) is proposed, whih gives a disriminative

feature spae. Subsequently, a disriminative spae map (DSM) is derived by using the dis-

riminative feature spae. A novel region growing method is also proposed, whih grows skin

regionsofanimagebasedonhromatipropertiesofanimage. Thisresultsadditionalredution

in the detetion errors.

InChapter 3,imagepixeldistributioninformationisinorporatedintoaglobalskindete-

tion model for aloalmodeladaptation. To obtain aloalskin detetion modelfor animage,

a loal skin distribution model (LSDM) and a loal bakground distribution model (LBDM)

are needed, whihan bederived by measuring similarity between animage pixeldistribution

model and a referene skin olour distribution model. The referene skin pixel distribution

model an be derived from a set of skin samples of faial region of a person in animage. Fi-

nally,anovelskinmapisobtainedbyperformingafusionofskinmapsobtainedfromtheglobal

and the loal skin detetion models. The proposed skin map is termed as fusion-based skin

probability map (FSPM), and nally the proposed dynami region growing (DRG) algorithm

is appliedfor additionaldetetion error redution.

Chapter 4 desribes a skin detetion method for videos when skin regions undergo hro-

matiappearanehangesduetonon-uniformillumination. Non-uniformilluminationantake

plae due to loal shading eets, whih are assoiated with the motion of skin regions. The

proposed method has two modules a stati skin detetion model, and a dynami skin de-

tetion model. The stati model onsists of a faial skin distribution model (FSDM) and a

video spei bakground model. On the other hand, the dynami model orresponds to a

moving skin-pixel distribution model(MSDM). The MSDM is dynamially updated based on

assoiated hromatihanges inskin olour.

Finally,we draw ouronlusion inChapter 5 by highlightingthe strengths and shortom-

ings of our shemes and outliningpossible extensions.

2

Detetion of Skin using Image Spei

Disriminative Feature Extration

Aurayof olour-basedskinsegmentation methods are signiantlyaetedby dierent issues

suh as presene of skin-like olours in sene bakground and unontrolled illumination ondi-

tions. Standard skin probability map (SPM) an not perfetly disriminate skin and non-skin

regions in these onditions. To overome limitations of SPM, a new probability map termed

as disriminative spae map (DSM) is proposed by extrating most disriminative features be-

tween skin and non-skin regions. A novel adaptive disriminative analysis (ADA) is proposed

to extrat most disriminantfeatures between skinand non-skinregionsfrom an image itself in

an unsupervised manner. Subsequently, a dynami region growing (DRG) method is employed

to allow skin regions to grow dynamially. The DRG ontrols false detetion by restriting

the region growing proess. Experimental results for standard databasesshow that the proposed

method an eiently segment out skin pixels in presene of skin-like bakground olours and

unontrolled illuminationonditions.

2.1 Introdution

In-spite ofhavingseveraladvantagesofolour-basedskinsegmentation,auratesegmenta-

tion ofhand orany otherbodyparts isstillabighallenge. Theauray ofolour-basedskin

detetion methodsare severely aetedby the presene of skin-likeolours inthe bakground.

Majorityofskindetetionalgorithmsuseskinolourasaprimaryfeature. However, useofsome

other features liketexture information[103℄ or depth information[104,105℄ along with olour

feature generally improvesegmentation auray. Zhiwei et al. [83℄ used Gabor wavelet-based

texture lter, whereas Dumitresu and Dumitrahe [106℄ used Gray Level Co-ourrene Ma-

trix (GLCM) texture features for skin detetion. In allthese methods, it is assumed that skin

regionsare smootherthanbakgroundregions. But,the bakground mayhavesimilartextural

smoothness asthat of skin regions. In these spei ases, hromati parameters,suh ashue,

saturation may be the disriminativefeatures between skin and non-skin regions. Kawulok et

al.[9℄usedlineardisriminativeanalysis(LDA)toderivemostdisriminativefeaturesbetween

skin and non-skin regions. In this method, LDA projetion matrix is derived using olour

and loal texture features from a set of labelled images. The LDA projetion matrix depends

on training data. Therefore, LDA gives a projetion matrix whih ensures the best possible

inter-lassdisrimination. However, naturalimagesdonothaveanyxedtexturalorhromati

pattern. Therefore, it is highly likely that the test image harateristis may be signiantly

dierent from the training images. In these spei ases, derived disriminativefeature may

not bethe most disriminativefor thetest images. So,animage speidisriminativefeature

extrationmethodisessential,sothat the imagespeifeatures an beextrated for disrim-

inant analysis for skin segmentation. Kawulok et. al. [15℄ also proposed an adaptive sheme

to selet seeds to further improve the detetion results. Hettiarahhi and Peters [16℄ showed

that the aurayofskindetetion an besignianlyimproved usingmulti-manifoldlearning.

However, multi-manifoldlearning-basedmethod uses some pre-seleted true skin and skin-like

non-skinsamplesfortraining. But,disriminationbetweentrueskinandskin-likenon-skinpix-

elsentirelydependsonseneharateristis. The methodsproposedin[10,21,23℄extrat faial

pixelto generate loallyadapted skin model. However, afae must bedeteted in animagein

order to apply these methods. Most of these methods follow unsupervised or semi-supervised

learning approahes. However, reent trend in omputer vision is Convolutional Neural Net-

work(CNN)-based arhiteturesforpixel-wisepredition,suhassemantisegmentationusing

fullyonvolutionalnetworks(FCN)[107℄. However, performaneofCNNsinomplexomputer

visionproblems, suh aspixel-wiselassiationislimited. The onvolutionallterswith large

reeptive elds in traditional CNNs result in oarse outputs for pixel-wise lassiation [107℄.

TheresultbeomesoarserduetothepreseneofmaxpoolinglayersinCNNs[108℄. Thisresults

in smooth boundaries and blob-like strutures in segmented images. Also,lak of smoothness

onstraint inCNNs may result in spurious regionswith poor objet outlinesin the segmented

images [109111℄. Zheng et al. [112℄ showed that inorporation of onditional random elds

(CRFs) at the nal layer of CNN an rene oarse preditions. However, skin detetion is a

very subjetive problem whih depends on image harateristis. A olour (e.g, brown) may

be the skin olour for Afrianpeople or it may be a bakground olour for Cauasian people.

Thus, lak ofimage speiadaptivityinexisting semantisegmentation algorithmsmay limit

their appliabilityin skin detetion.

As mentioned by Kawulok [9℄, LDA an be used to extrat most disriminative features

for skin and non-skin pixels. However, LDA needs a set of labelled training dataset. In skin

olour segmentation problem, some of the trainingsamples belonging to the non-skin regions,

like bakground regions, look very similar to the atual skin regions, and vie-versa. As the

trainingsamples for two lasses (skinand non-skin) are obtained from animage itself,and so,

a few samples orresponding to bakground and other regions take the identity of samples of

theatualskinolourregions,and vie-versa. So,therewillbemarginalinter-lassoverlapping

of samples extrated from an unknown image. Therefore, a feature derived using LDA may

not be most disriminative for an unknown image. To address this issue, a novel adaptive

disriminative analysis (ADA) for skin segmentation is proposed. It is adaptive in a sense

that the ADA adapts to inter-lass similarity or overlapping samples. Hene, the extrated

feature would be image spei, not generi for all the images. In our approah, two sets of

pixel loations

S 1

^and

S 2

^{are used for deriving ADA projetion vetor. The set}

S 1

^ontains

pixel loations that are very likely to be skin, whih implies that some of them may look like

skin, but they atually belong to non-skin regions. The other set

S 2

^{ontains pixel loations}

that are mostly non-skin, but a fewof them belonging toatual skinregions. Therefore, there

is no labelled training dataset. The proposed ADA assigns lass-memberships to eah of the

training samples in

S 1

^and

S 2

^. Subsequently, a disriminativefeature spae is extrated from the given image itselfwith the help of the proposed ADA. Finally,a disriminativespae map

(DSM) isderived by using the disriminativefeature spae. In ontrary tostandard SPM, the

proposed DSM providesbetter disriminationbetween skin and non-skin regions of animage.

The skin regionsan bedeteted eitherbydiretly thresholdingthe skinmap orDSM [13℄.

However, diret thresholding of DSM may inrease false detetion rates when skin olour is

very muh similar to bakground sene olour. Seeded region growing (SRG) of skin regions

is an alternative to diret thresolding of SPMs [113℄. The use of SRG algorithm for image

segmentationhasbeenwellinvestigated inreentyears. AdamsandBishof[114℄rstproposed

the onept of SRG.The majorlimitationof SRG algorithmis the seletionof anappropriate

threshold tojudge asimilarityondition, whih limitsitsappliabilityinskin segmentation. If

a very high threshold is seleted, then many true skin pixels ould be rejeted. On the other

hand, false detetion will be more for a small threshold. Propagation-based region growing

methodasproposedbyKawulok[14℄approximatesanimageasagraphwiththepixelsasnodes

within it. Kawulok's method is based on Dijkstra's minimum path-ost algorithm [88℄. The

regiongrowingmethodproposed byKawulok(2013)isnotadaptive. Theentireregiongrowing

proess depends on manually seleted parameters. The region growing should be dependent

onsene harateristisof an image. Region growing shouldbemore if the bakground olour

similarity with the skin olour is less, and vie-versa. To handle these issues, we proposed a

dynami region growing (DRG) methodby employing only fewmanually seleted parameters.

The proposed method isdisussed indetail in the Setionto follow.