Visualization and Perception Laboratory - CSE-IITM

(1)

Visualization and Perception Laboratory Visualization and Perception Laboratory

Department of Computer Science and Engg., IIT Madras, Chennai – 600036, India.

//www.cs.iitm.ernet.in/~sdas/vp_lab/home.htm

//vplab.cs.iitm.ernet.in

(2)

Images, scenes, pictures

Model parameters, Object/Scene

representation Vision

Visualization

VISUAL VISUAL

PERCEPTION PERCEPTION

Flowchart illustrating the concept of visual perception.

(3)

Goal: Design of new computational frameworks, models and algorithms based on human perception and visualization to solve real-life complex problems in object/scene understanding and visual realism/visualization.

Two main categories of Areas/Applications

Object/Scene understanding

• BIOMETRY:

- multi-modal

- 3-D face and video

• Defense surveillance & GIS

• Generic 3-D Object Recognition

• Shape & Structure from 3D texture

• Foreground object detection

• CSS/MCC for MPEG-VII shape descriptors

Visualization

• Super-resolution texture synthesis

• Video based rendering

• Augmented Reality

• Modeling nature

(4)

Feature Extractor Probe

(Testing sample)

Identity or Accept/

Reject

Face Recognition Problem Face Recognition Problem

Face recognition can be defined as the identification of individuals

from images of their faces by using a stored database of faces labeled with people’s identities.

Matching 1:N or 1:1

Gallery (Training samples)

(5)

1-D Processing Input

Image

Edginess Image

Eigen Analysis

1-D Processing Input

Image

Edginess Image

Eigen Analysis Transformation

Function Eigenedginess:

Transformed edginess:

Face Recognition using PCA

(6)

Grey level image

Edge image

Edginess image

Results of EPCA

Artificial Neural Networks Lab IIT Madras

(7)

R e p r e s e n ta tio n P e r fo r m a n c e

E ig e n fa c e 1 4

E ig e n e d g e 2 4

E ig e n h ill 2 1

E ig e n e d g in e s s 5 6

Face Recognition performance

(Out of 80 faces)

(8)

Recognition performance due to variations in facial expression

Category % Eigenface 94 Eigenedginess 93 Eigenhill 77 Eigenedge 47

(9)

Modular Eigen-analysis or MPCA

VP Lab, CSE, IITM

(10)

Modular PCA

ea eb ec

+ E1

E2

E3

E4

w1 w2

w3

Weighted

(11)

Results of WMPCA

(12)

Results of WMPCA

(13)

Experimental Results of

VPLAB-FACE-RECOGNIZER

We have compared the performance of our face recognition method (patent application being filed), using four linear subspace methods (PCA, LDA – 1D & 2D) on three (3) Face Databases.

Near-frontal (pose variations)

10 40

ORL

Frontal, only illumination variation

42 60

PIE

Frontal, illumination and expression variation

11 15

YALE

Description No. of

samples /class No. of

subjects

Database

(14)

PEAK RECOGNITION ACCURACY (PRA) AND EQUAL ERROR RATE (EER) OF SUBBAND FACE REPRESENTATION INTEGRATED WITH PCA, LDA,

2D-PCA, 2D-LDA AND DCV WITH SUBJECT-SPECIFIC SUBBANDS OBTAINED, USING FOUR CRITERIA, ON YALE DATABASE

Experimental Results

Training images

RESULTS ON Yale DATABASE

(15)

RESULTS ON PIE DATABASE

Training images

(16)

Results on ORL DATABASE

Training images

(17)

Dual space Face Recognition using Feature Fusion

We propose a new face recognition technique by combining information from null and range space of within-class scatter of face space.

A new space named as dual space is constructed by merging two different set of discriminatory directions obtained from null and range space separately.

1) Covariance Sum and

2) Gramm-Schmidt Orthonormalization

We employ forward and backward selection techniques to

select the best set of discriminatory features from dual space.

(18)

PERFORMANCE OF DUAL SPACE FACE RECOGNITION ON YALE, ORL AND PIE DATABASES.

PERFORMANCE OF NULL and RANGE SPACE FACE RECOGNITION ON YALE, ORL AND PIE DATABASES.

87.35 87.28

88.40 88.40

PIE

85.99 85.83

85.93 85.83

ORL

85.00 85.00

Yale

FS BS

Databases

Gram-Schmidt Covariance Sum

73.14 86.35

PIE

71.67 79.17

ORL

75.00 81.67

Yale

Range Space Null Space

Methods

(19)

Dual space Face Recognition using Decision Fusion

Two classifiers are constructed based on optimal discriminatory directions obtained from null and range spaces separately.

Then these two classifiers are combined using Sum, Product and our proposed method of decision fusion.

Given a sample x, the output response from null and range space based classifiers are:

)]

( ),...,

( ),..,

( [

)

( x d 1 x d x d x

D Null = Null Null i Null C

)]

( ),...,

( ),..,

( [

)

( x d 1 x d x d x

D Range = Range Range i Range C

These vectors, called as response vectors, are the soft class labels provided by a classifier

Our aim here is to learn each classifier separately using

training information at decision level.

(20)

Proposed method of decision fusion

We use the training information (response vectors on a validation set) as feature vectors to construct LDA and

nonparametric LDA based eigenmodel.

Now new test response vectors are calculated in the eigenmodel and are used to replace the old ones.

Replace DP(x) by,

Responses of the classifier on a validation set of the database (which is disjoint from training and testing set) can be used as training information.

Sum Rule: Computes the soft class label vectors using,

Product Rule: Computes the soft class label vectors using,

(21)

PERFORMANCE OF DUAL SPACE BASED FACE RECOGNITION ON YALE, ORL AND PIE DATABASES.

PERFORMANCE OF NULL AND RANGE SPACE FACE RECOGNITION ON YALE, ORL AND PIE DATABASES.

73.14 86.35

PIE

71.67 79.17

ORL

75.00 81.67

Yale

Range Space Null Space

Methods

100.00 100.00

83.85 83.72

PIE

97.50 96.67

81.67 81.67

ORL

86.67 86.67

Yale

P NLDA P LDA

Product Sum

Methods

(22)

Results of

Multi-modal Biometry

(23)

Decision Fusion of Face and Fingerprint

We have chosen face and fingerprint for their

universality and uniqueness properties respectively.

Feature level fusion is not possible in our case because of incompatible feature sets (e.g., fingerprint minutiae

and eigen-face coefficients).

The philosophy behind Decision Fusion:

¾ One of the classifiers can be best in terms of performance

¾ but the sets of patterns misclassified by different classifiers would not necessarily overlap.

This suggests that different classifiers offers

complementary information about the pattern to be

classified, for the same pattern classification problem .

(24)

Assuming that face and fingerprint are statistically independent for an individual, we have associated an

individual from face database with an individual from fingerprint database to create a virtual subject.

Face Databases Used: PIE and ORL.

Fingerprint Databases Used: FP A and FP B (from NIST).

So get four possible combinations: (i) PIE & FP A , (ii) PIE

& FP B , (iii) ORL & FP A and (iv) ORL & FP B .

For face Databases we have taken two test cases and for

fingerprint four test cases.

(25)

Base classifier's performance (in Percentage Accuracy) for four face and fingerprint databases (two each)

Base classifier's performance (in Percentage Accuracy) using

LDA for the face and fingerprint databases (two each)

Base classifier's performance (in Percentage Accuracy)

using nonparametric LDA for the face and fingerprint

databases (two each)

(26)

Combined performance with PIE and FP A databases for

different Decision Fusion Strategies

(27)

Combined performance with PIE and FP B databases for

different Decision Fusion Strategies

(28)

Combined performance with ORL and FP A databases

for different Decision Fusion Strategies

(29)

Combined performance with ORL and FP B databases

for different Decision Fusion Strategies

(30)

Overview of the approach for Road Overview of the approach for Road

detection from satellite images detection from satellite images

Satellite image is processed in four stages:

• First stage: Edge map is obtained from satellite image using 1-D processing

• Second stage: Edge map is post-processed using a set of binary morphological operations

• Third stage: Linear and curvilinear segments are

extracted using a GMM based extractor from the edge map

• Final stage: A FFNN is used to detect road segments from GMM based extractor output

(31)

Satellite image of a part of Chennai,

Source: NRSA, 1998. Same from Google-Earth, Nov-2005.

(32)

Satellite image of a part of Chennai (i) Post-processed edge map (ii) GMM output

Manually plotted road network using a GUI tool (iii) ANN output

Scanned map of same part

(33)

Satellite Image of a part of Chennai city.

Res: 5.8 m

Extracted road segments

using a hybrid approach

SATELLITE IMAGE – ROAD DETECTION

(34)

A satellite image

ANN output Road boundaries from ANN output

Output using GMM Manually plotted road network

(35)

Multi-spectral Satellite image, Res: 1m

Extracted road segments using

a hybrid approach

(36)

2 2 - - D TEXTURE SEGMENTATION D TEXTURE SEGMENTATION

(37)

Flow chart of the method of texture

classification

Filtering

Non-linearity

Smoothing

Normalizing non-linearity

Classifier Filter responses

Local energy function

Local energy estimates

Feature vectors

Input image

Segmented map

(38)

Image with five Texture regions

Segmentation result using:

Gabor + DWT, FCM

(39)

Results (Cont.)

I1 I2 I3 I4 I5

Segmented maps (DWT+DCT)

Input images

(40)

Results (Cont.)

I6 I7 I8 I9 I10

Input images

Segmented maps (DWT+DCT)

(41)

Results on SAR images

SAR Image

Fine segmented image

Coarse segmented image

Coarse segmented

image

(42)

Filtering

Smoothing using 2-D asymmetric Gaussian filter

Self-Organizing feature Map (SOM)

Smoothing using 2-D symmetric Gaussian filter

Edge detection using Canny operator

Stages of processing for texture edge detection.

Edge map

Edge Linking

Input image Input image Edge map

Experimental results of texture edge detection.

Texture Edge Detection

(43)

Results

Input image Edge map Input image Edge map Input image Edge map

(44)

Integrating Region and Edge

Information for Texture Segmentation

We have used a modified constraint satisfaction neural networks termed as Constraint Satisfaction Neural Network for Complementary Information Integration (CSNN-CII), which integrates the region and edge based approaches.

+

(45)

Input Image Segmented map before integration (Ref: [Rao 2004]) Edge map before

integration

(Ref: [Gupta 2006]) Segmented map

and Edge map after integration

Results

(46)

Input Image

Segmented map before integration

Edge map before integration

Segmented map and Edge map after integration

Results

(47)

Results

Input Image

Segmented map before integration

Edge map before integration

Segmented map

and Edge map

after integration

(48)

Results (Cont.)

Input Image

Segmented map before integration (Ref: [Shivani 04])

Segmented map and Edge map after integration Segmented map

after integration as

given in [Munoz 02]

(49)

Proposed Hierarchical Landform Classification

Input Image

Coastal Fluvial

DP I

DP II Dunes

Fuse Landform Segments Parabolic

Longitudinal Barchanoids Sandy plain

Inselberg Salt Flats

Desertic

Fuse Landform Segments

Active Channel

FP I

Flood Plains

Bars Ox-Bow

FP II FP III

Fuse Landform Segments

Class-Labeled Output Image

Alluvial Plains CP I

Creek Coastal Bars

Swamp

Forested Swamp

Sea Sandy

Beach

Alluvial plain

CP II CP III

Class-Labeled Output Image

(50)

Integrated Landform Processing

(51)

Landform Processing

(52)

Integrated Results for Coastal landform

Coastal Image (size: 1060 x 1062)

Final Output

(53)

Fused Output Image

Input Image

(54)

Scene Classification

Test image Segmentation using DWT features

Feature detection from segments

Modified PNN Output

class

Training samples

Classification of indoor vs. outdoor scenes using texture, color and shape features and probabilistic neural networks

Indoor image

Outdoor

image

(55)

Examples of correctly classified

indoor images

Examples of correctly classified

outdoor images

Examples of indoor images misclassified

as outdoor images

Examples of outdoor images misclassified

as indoor images

(56)

Results

83.1 89.6

Color + Texture + Shape

90.8 94.0

Color + Texture

83.8 90.4

Shape + Texture

71.5 89.2

Shape + Color

86.9 94.0

Texture

53.5 94.0

Color

66.5 63.5

Shape

Outdoor Indoor

Feature set

INDOOR VS. OUTDOOR

CLASSIFICATION ACCURACY (%) ON IITM-SCID WITHOUT INTEGRATING REGION AND EDGE INFROMATION

83.8 89.7

Color + Texture + Shape

92.3 94.3

Color + Texture

84.8 90.5

Shape + Texture

- -

-

- -

-

- -

-

- -

-

Outdoor Indoor

Feature set

INDOOR VS. OUTDOOR

CLASSIFICATION ACCURACY (%) ON IITM-SCID BY INTEGRATING REGION AND EDGE INFROMATION

Ref: [lalit06b]

(57)

3D Texture Analysis 3D Texture Analysis

using DWT:

Orientation estimation Orientation estimation

and 3D texture segmentation

(58)

system ordinate

- co Surface )

, , (

system ordinate

- co Image

) , (

system ordinate

- co World )

, , (

−

s s

s

y z

x v u

z y x

) Elevation(

,

Azimuth ( φ ) θ

Polar – azimuth representation

Advantages:

1. Separable Analysis along horizontal and vertical directions 2. Representation is complete

2 2 2

)]

tan(

) (

[

] ) (

[ ) sec

( α

α

i

i r

i

t F u

u F

u f

f −

= +

The Spatial Frequency The Spatial Frequency Distribution (SFD)

Distribution (SFD)

(59)

Algorithm DWT

Dominant scale selection Scale to frequency

Select points

Voting

Scalogram Input image

Scalemap

SFD

Angle NFM

Regression

(60)

Results of Simulated Image set

Dyadic level-3 decomposition

(61)

0 10 20 30 40 45

Input images

Energy distribution Scalogram

Spectrum

High frequency image, Origin at center

(62)

Curve fitting using Least squares

Normalized Frequency Map (NFM)

Selection of Points on the NFM

Experimentally obtained SFD

Comparison of Experimentally obtained SFD and theoretical SFD

0 10 20 30 40 45

Experimentally obtained SFD Simulated SFD

(63)

computed errors

F=520 parabola exp sfd parabola exp sfd 0 -0.59695 4.678939 4.089852 4.505786 4.678939 4.089852 10 11.15202 13.93284 11.96059 2.746978 3.932839 2.108769 20 24.73764 22.60145 20.53271 4.737642 2.601446 0.532714 30 37.29204 33.17194 30.02636 7.292045 3.171942 0.481817 40 45.62462 42.56704 38.9246 5.624617 2.567039 1.075396

abs errors

0 1 2 3 4 5 6 7 8

1 2 3 4 5

angles

errors parabola

exp sfd

RESULTS Using dyadic Daubeschies

WAVELET FILTERS

(64)

4-channel

computed errors

parabola exp sfd parabola exp sfd

0 8.25 6.02 2.61 8.25 6.02 2.61

10 17.08 12.31 9.07 7.08 2.31 0.93

20 26.34 26.07 23.62 6.34 6.07 3.62

30 38.07 37.40 31.30 8.07 7.40 1.30

40 48.55 47.44 38.98 8.55 7.44 1.02

0 1 2 3 4 5 6 7 8 9

0 10 20 30 40

true angle

abs error

parabola exp sfd

RESULTS Using 4-CHANNEL

WAVELET FILTERS

(65)

Real world images (Super and Bovik)

Stadium Bleachers steps Steps rotated Venetian Blind Brick wall

Aluminum wall

-30 -20 -10 0

Grill vertical Grill horizontal Ventilator 0

^o

Ventilator -20

^o

(66)

Comparison of errors

0 4 8 12 16 20

br ick 55 _0 gr ill 30_

0 gr ill 0_1

0

sta di um _0 _7 0

ste ps _0 _7 0 Ve ne tia n B lind_

...

Ve nt ilo ato r_0 _20 Ven

tilo ato r_ 0_ 0

wa ll 0

wa ll 1 0

wa ll 2 0

wa ll 3 0

Input image

A b s o lu te E rr o r

Super& Bovik Dyadic

M-channel

(67)

Azimuth = 22.07 Elevation = 20 Azimuth=26 Elevation=-37

2.80 -22.80

Azimuth = -20

-0.98 22.98

Polar = 22

Error Computed

True angle

1.04 -38.04

Azimuth = -37

4.73 30.73

Polar = 26

True angle

Azimuth =30 Elevation=-54

-54.14 0.14 Azimuth = -54

1.66 28.34

Polar = 30

True angle

Real world images (Rebeiro & Hancock), in PAMI’01

(68)

Our lab. images

in 2006:

(69)

3-D Real world Texture image

Fuzzy segmentation using wavelet features

The problem of 3-D texture segmentation

M -c ha nn e l W a ve le t f ilt er ban k

1 2

M

Local VAR.

Compn.

Local POWER Compn.

Fe a tu re S et I Fea tu re Se t I I

Wavelet

Coefficients Post-processing

3-D Texture processing using

m-channel wavelet filterbank

(70)

3-D Real world Texture image

3-channel DWT

Fuzzy segmentation using wavelet features

Segmentation using 3-D (gradient) texture features.

Desired number of classes (input to FCM):

2 3

(71)

Image of

WATER_MIRROR

Segmentation results, with:

3-channel bi-orthogonal

filter

4-channel orthogonal

filter

(72)

Curved Texture surface

Directional wavelet-based

(4-channel) 3D texture features:

Future Work: Segmentation of 3D textures having a

combination of planar and curved objects.

(73)

GENERIC OBJECT GENERIC OBJECT

RECOGNITION RECOGNITION

(GOR)

(74)

What is Generic Object Recognition?

z Generic Class Recognition:

Recognition not restricted to a single class of objects, say only faces or cars.

Spans multiple classes having a wide variety of distinguishing features.

Aims at specifying to which subclass a previously unseen object belongs to, rather than an exact identification of the object.

Requires use of multimodal features, selecting and decision fusion of different classifiers.

Object Recognition can be generic in two ways:

z Object Recognition : Distinguishing between multiple objects within a class eg. face recognition, fingerprint recognition, vehicle recognition.

z Generic Object (multiclass) Recognition with Pose:

Pose-invariant object recognition for various class of objects.

Requires feature fusion and/or decision fusion to deal with multiple classes.

Use of 3D information (model for structure

representation) for pose detection and

visualization.

(75)

Proposed Framework

• Hierarchical framework with two stages of processing

Distance Transform based Feature

Extraction and Matching Select a set of rank-ordered

samples

…

Selected Samples

Average, Select sample with minimum distance Minimum distance Report Recognized

Object

Stop Test

Image

Linear Subspace Analysis-based Intelligent Choice

Image Gallery

A

D.T. based

…

Correlation Distances Eigendistances

…

Compute Eigenspace using 2D-PCA

Project

Training Samples onto eigenspace

Training Samples

Projected Samples

…

A

(76)

Samples from COIL Object Database Parametric Object Eigenspace using 1D PCA

z Manifold is constructed for object eigenspace by projecting object images onto object eigenspace and using cubic spline interpolation

[ ] ( ( , ) ( ) )

) ( )

( 2 )

( 1 )

(

. l p p , p ,..., k p T r p l p

r e e e x c

f = −

z Three most prominent eigenvectors shown. Dots correspond to projections of learning samples.

z Constant illumination. Thus, appearance is given by a curve with single parameter, rather than a surface.

Object Eigenspace Construction using 1D PCA

(77)

Generic Classifier for Intelligent Choice : 2D PCA

z Used to reduce search space to a small number of samples

z Eigenspace represents a compact model of object’s appearance

Parametric Eigenspaces for two objects

z Three most prominent eigenvectors shown. Dots correspond to projections of learning samples.

z Constant illumination. Thus, appearance is given by a

curve with single parameter, rather than a surface.

(78)

2D PCA (Cont.)

Eigenspace of two objects with distinct appearances

(79)

2D PCA (Cont.)

Eigenspace of two objects with similar appearances and

different shapes

(80)

Results for 2D PCA with Shape Matching

(81)

Results (First phase) of GOR

over 100 objects of COIL database

Percentage Accuracy of 2D PCA Percentage Accuracy of 2D PCA with Shape Matching

Comparison of percentage accuracies of 2D PCA and 2D PCA with

shape-based Matching for eigendimension =3

(82)

2D PCA Vs ICA

Comparisoin of Recognition Accuracies of 2D PCA and ICA on COIL-100 Database

88 90 92 94 96 98

Pose Increment for training

% Acccuracy

2D PCA 96.375 93.625 93.25 89.5 88.75

ICA 97 94.5 93.75 91.25 91

10 15 20 25 30

2D PCA EigenDimension = 10; No. of ICs = 110

(83)

Decision Fusion of 2D PCA and ICA

• Both 2D PCA and ICA contribute complementary information to the classification task.

• Classifier Combination

Test Image

…

Training Samples 2D PCA

ICA

Matching Score

Normalized Scores

Report Closest

Object

∑

Ref: [Kittler04]

(84)

Decision Fusion of 2D PCA and ICA

Performance of Combined Classifier using SUM Rule for Fusion on COIL-100 Database

80 85 90 95 100

Pose Increment for training

% Accuracy

2D PCA 96.375 93.625 93.25 89.5 88.75 84.125 82

ICA 97 94.5 93.75 91.25 91 86 84

2D PCA & ICA Fusion 97.375 94.625 93.75 91.375 91.375 86.875 84.25

10 15 20 25 30 35 40

2D PCA EigenDimension = 10; No. of ICs = 110

(85)

Shape Matching on Fused Generic Classifier

Comparision of effect of Shape Matching on 2D PCA, ICA and Combined Classifier

89 91 93 95 97 99

Pose Increment for training

% A c cu ra cy

2D PCA+Shape 97.5 93.875 93.75 91.375 90

ICA + Shape 97.25 95.5 93.75 90.625 90.5

Fusion + Shape 98.25 95.875 93.75 91.5 91.375

10 15 20 25 30

(86)

Comparison of Proposed Framework with present state of art techniques

*Results reported in literature

$

Our Implementation

96.03 1152

32

^*

SVM

97 400

96.639 3600 100

ICA

100 720 20

^*

2D PCA

97.694 98.25

95.468 96.375

% Accuracy 100

3600 400

Test 720

Samples

100 100

^$

20

^*

No. of Objects

Proposed Fusion + Shape

Matching 1D PCA

Technique

(87)

Results:

Cluttered Scenes Extracted Object Using GrabCut

Recognized Object

from the database

Application: Recognition in Cluttered background

(88)

Cluttered Scenes Extracted Object Using GrabCut

Recognized Object

from the database

Application: Recognition in Cluttered background

(89)

Known Background Region

Known Foreground Region Unknown Region

(90)

Original Videos Segmented Moving object

Retrieved Model &

Rendered Image

Application: Recognition from Video

(91)

Application: Content-based Video Retrieval ( CBVR ):

Develop a system to retrieve the video similar to that of a given query.

Sample Videos in the database

Query Images

Retrieved videos from the database

Video no: 25 Video no: 1

1

10

25

4

33 Video

no

(92)

Use of Active Contour for texture object detection

Contour Initialization Detected Texture Boundary

(93)

Modeling of Texture Force

Texture Feature Extraction Scalogram Estimation

Texture Feature Estimation

Segmentation Texture Image

Texture Feature Image

(94)

Contd…..

Texture Feature Extraction Scalogram Estimation

Texture Feature Estimation

Segmentation Texture Image

Input Texture Image

Scalogram Estimation

Texture Force Modeling

Texture Feature Estimation

Texture Feature Image

(95)

Segmentation Output

Texture Feature

Image

Input Image

(96)

Segmentation output

Texture Feature

Image

Input Image

Experimental Results Contd…..

(97)

Our proposed approach for segmentation of object with

hole, using the combination of Active Contour and GrabCut

SnakeCut

Integration of Active Contour and GrabCut

(98)

SnakeCut

Central rectangle is not part of the object

User selects object to be

segmented

Desired output

Sna keC ut

Problem in this Output:

Upper part of the object has been thrown off

Gr ab Cu t

Problem in this Output:

Hole is detected as part of the object

Active Contour

(99)

Real world Result-1

Here, objective is to crop the wheel from the input image

Cropped image should not contain any part of the

background

(100)

Snake Output

GrabCut Output

SnakeCut

Output

(101)

Real world Result-2

Here, objective is to crop the soldier from the input image

Cropped image should not contain any part of the

background

(102)

GrabCut Output

SnakeCut

Output

Snake Output

(103)

TEMPLATE MATCHING

(104)

The Problem Definition

IMRN

IMT

Given a bitmap template (IMT) and a noisy bitmap image IMRN which contains IMT (believe me):

FIND OUT the location of IMT in IMRN !

Go to the next page for more:

(105)

Problem explanation for pessimists.

• IMRN (in previous page) is obtained by adding a large level of “Salt and Pepper” noise onto IMR bitmap image.

• IMT is also obtained from IMR as shown above.

IMT

IMR

(106)

The RESULT beats the

human EYE – comprehensively

IMR IMRN

IMT

(107)

Target Scene

Target

Scene

(108)

Face Detection

(109)

Face Detection

(110)

Super-resolution texture synthesis using Image Quilting

Minimum-error boundary cut.

Blocks constrained by overlap.

Random placement

of blocks.

(111)

Image Quilting

Input Image

Output for Patch Size of 32 Pixels and Overlap Size of 6 Pixels.

Input Image

(112)

Image Quilting

Input Image

Input Image

(113)

Image Quilting

Input Image

Input Image

(114)

Image Quilting

Input Image

Input Image

(115)

Future Work (planned and unplanned) Future Work (planned and unplanned)

• Large scale visual Surveillance and monitoring

• 3D scene analysis

• Multi-sensor data fusion and super-resolution analysis

• Modeling of soft objects

• Mobile Reality

• Artificial life

• Visual Area Networking (VAN)

(116)

To be To be

Continued Continued

&

Thank You Thank You

June, 2007

(117)

Visualization and Perception Laboratory - CSE-IITM

Visualization and Perception Laboratory Visualization and Perception Laboratory

Flowchart illustrating the concept of visual perception.

- multi-modal

• Augmented Reality

1-D Processing Input

R e p r e s e n ta tio n P e r fo r m a n c e

Modular PCA

Frontal, illumination and expression variation

Training images

 A new space named as dual space is constructed by merging two different set of discriminatory directions obtained from null and range space separately.

PERFORMANCE OF NULL and RANGE SPACE FACE RECOGNITION ON YALE, ORL AND PIE DATABASES.

 Then these two classifiers are combined using Sum, Product and our proposed method of decision fusion.

training information at decision level.

 Replace DP(x) by,

PERFORMANCE OF NULL AND RANGE SPACE FACE RECOGNITION ON YALE, ORL AND PIE DATABASES.

universality and uniqueness properties respectively.

 This suggests that different classifiers offers

 Face Databases Used: PIE and ORL.

LDA for the face and fingerprint databases (two each)

different Decision Fusion Strategies

detection from satellite images detection from satellite images

extracted using a GMM based extractor from the edge map

Manually plotted road network using a GUI tool (iii) ANN output

Satellite Image of a part of Chennai city.

A satellite image

Multi-spectral Satellite image, Res: 1m

Input image

SAR Image

Stages of processing for texture edge detection.

Input Image Segmented map before integration (Ref: [Rao 2004]) Edge map before

Edge map before integration

Integrated Landform Processing

class

as outdoor images

Color + Texture + Shape

and 3D texture segmentation

Algorithm DWT

Curve fitting using Least squares

angles

RESULTS Using dyadic Daubeschies

4-channel

RESULTS Using 4-CHANNEL

0

tilo ato r_ 0_ 0

Our lab. images

3-D Real world Texture image

The problem of 3-D texture segmentation

3-D Real world Texture image

3-channel DWT

Fuzzy segmentation using wavelet features

Image of

RECOGNITION RECOGNITION

 Requires use of multimodal features, selecting and decision fusion of different classifiers.

Use of 3D information (model for structure

Distance Transform based Feature

Correlation Distances Eigendistances

Samples from COIL Object Database Parametric Object Eigenspace using 1D PCA

Object Eigenspace Construction using 1D PCA

z Constant illumination. Thus, appearance is given by a

over 100 objects of COIL database

Pose Increment for training

Decision Fusion of 2D PCA and ICA

Performance of Combined Classifier using SUM Rule for Fusion on COIL-100 Database

Pose Increment for training

Shape Matching on Fused Generic Classifier

% Accuracy 100

Recognized Object

Known Background Region

Original Videos Segmented Moving object

Use of Active Contour for texture object detection

Texture Feature Estimation

Segmentation output

User selects object to be

Cropped image should not contain any part of the

Snake Output

• IMT is also obtained from IMR as shown above.

Random placement

Input Image

Image Quilting

A new space named as dual space is constructed by merging two different set of discriminatory directions obtained from null and range space separately.

Then these two classifiers are combined using Sum, Product and our proposed method of decision fusion.

Replace DP(x) by,

This suggests that different classifiers offers

Face Databases Used: PIE and ORL.

Requires use of multimodal features, selecting and decision fusion of different classifiers.

Use of 3D information (model for structure