Lecture 4
Image Restoration
• Image restoration: recover an image that has been
degraded by using a prior knowledge of the degradation phenomenon.
A Model of Image Degradation/Restoration Process
• Degradation
– Degradation function H
A Model of Image Degradation/Restoration Process
If is a process, then
the degraded image is given in the spatial domain by
( , ) ( , ) ( , ) ( , )
H
linear, position-invariant
A Model of Image Degradation/Restoration Process
The model of the degraded image is given in
the frequency domain by
Noise Sources
• The principal sources of noise in digital images arise during image acquisition and/or transmission
Image acquisition
e.g., light levels, sensor temperature, etc.
Transmission
Noise Models (1)
• White noise
– The Fourier spectrum of noise is constant
• With the exception of spatially periodic noise, we assume – Noise is independent of spatial coordinates
Noise Models (2)
Gaussian noise
Electronic circuit noise, sensor noise due to poor illumination and/or high temperature
Range Imaging (1)
Short for High Dynamic Range Imaging. HDRI is an imaging
technique that allows for a greater dynamic range of exposure than would be obtained through any normal imaging process.
It is now popularly used to refer to the process of tone
mapping* together with bracketed** exposures of normal
digital images, giving the end result a high, often exaggerated dynamic range
* Tone mapping is a technique used in image processing and computer graphics to map a set of colours to another; often to approximate the appearance of HDRI in media with a more limited dynamic range
Range Imaging (2)
The intention of HDRI is to accurately represent the wide range of intensity levels found in real scenes ranging from direct sunlight to shadows
HDRI, also called HDR (High Dynamic Range) is a feature
Range Imaging: Examples (1)
Tower Bridge in Sacramento,
Range Imaging: Examples (2)
http://en.wikipedia.org/wiki/High_dynamic_range_imaging
Noise Models (3)
Erlang (gamma) noise: Laser imaging
Exponential noise: Laser imaging
Uniform noise: Least descriptive; Basis for numerous random number generators
Gaussian Noise (1)
2 2
( ) / 2
The PDF of Gaussian random variable, z, is given by
1 ( )
2
z z
p z e
where, represents intensity
is the mean (average) value of z is the standard deviation
z z
Gaussian Noise (2)
2 2
( ) / 2
The PDF of Gaussian random variable, z, is given by
1 ( )
2
z z
p z e
– 70% of its values will be in the range
– 95% of its values will be in the range
(
),(
)
Rayleigh Noise
2
( ) /
The PDF of Rayleigh noise is given by
2
( ) for
( )
0 for
z a b
z a e z a
p z b
z a 2
The mean and variance of this density are given by / 4
(4 )
4
z a b
Erlang (Gamma) Noise
1
The PDF of Erlang noise is given by
for 0 ( ) ( 1)!
0 for
b b
az a z
e z
p z b
z a 2 2
The mean and variance of this density are given by /
/ z b a
b a
Exponential Noise
The PDF of exponential noise is given by
for 0 ( )
0 for
az ae z p z z a 2 2
Uniform Noise
The PDF of uniform noise is given by
1
for a ( )
0 otherwise
z b p z b a
2 2
The mean and variance of this density are given by ( ) / 2
( ) / 12
z a b
b a
Impulse (Salt-and-Pepper) Noise
The PDF of (bipolar) impulse noise is given by
for
( ) for
0 otherwise
a b
P z a
p z P z b
If either or is zero, the impulse noise is calledPa Pb unipolar
if , gray-level will appear as a light dot,
while level will appear like a dark dot.
b a b
a
Periodic Noise
• Periodic noise in an image arises typically from electrical or electromechanical interference during image acquisition.
• It is a type of spatially dependent noise
Estimation of Noise Parameters (1)
Estimation of Noise Parameters (2)
Consider a subimage denoted by , and let ( ), 0, 1, ..., -1,
denote the probability estimates of the intensities of the pixels in .
The mean and variance of the pixels in :
s i
S p z i L
S S 1 0 1 2 2 0 ( )
and ( ) ( )
L
i s i i
L
i s i
i
z z p z
z z p z
Restoration in the Presence of Noise Only
̶ Spatial Filtering
Noise model without degradation
( , ) ( , ) ( , )
and
( , ) ( , ) ( , )
g x y f x y x y
G u v F u v N u v
Spatial Filtering: Mean Filters (1)
Let represent the set of coordinates in a rectangle subimage window of size , centered at ( , ).
xy
S
m n x y
Arithmetic mean filter
Spatial Filtering: Mean Filters (2)
Generally, a geometric mean filter achieves smoothing comparable to the arithmetic mean filter, but it tends to lose less image detail in the process
mn S t s xy t s g y x f 1 ) , ( ) , ( ) , ( ˆ
Spatial Filtering: Order-Statistic Filters (1)
( , )
max ) , ( ˆ ) ,( g s t
Spatial Filtering: Order-Statistic Filters (2)
( , ) min ( , )
max 2 1 ) , ( ˆ ) , ( ) , ( t s g t s g y x f xy
xy s t S
S t s
Spatial Filtering: Order-Statistic Filters (3)
We delete the / 2 lowest and the / 2 highest intensity values of ( , ) in the neighborhood . Let ( , ) represent the remaining
- pixels.
xy r
d d
g s t S g s t
mn d
xy S t sr
s
t
g
d
mn
y
x
f
) , ()
,
(
1
)
,
(
ˆ
Spatial Filtering: Adaptive Filters
Adaptive filters
The behavior changes based on statistical characteristics of the image inside the filter region defined by the mхn rectangular window.
Adaptive Filters:
Adaptive, Local Noise Reduction Filters (1)
2
: local region
The response of the filter at the center point (x,y) of is based on four quantities:
(a) ( , ), the value of the noisy image at ( , ); (b) , the variance of the noise corrupti
xy
xy
S
S
g x y x y
2
ng ( , ) to form ( , );
(c) L, the local mean of the pixels in xy;
f x y g x y
m S
S
Adaptive Filters:
Adaptive, Local Noise Reduction Filters (2)
2
2
The behavior of the filter:
(a) if is zero, the filter should return simply the value
of ( , ).
(b) if the local variance is high relative to , the filter
should return a value cl
g x y
ose to ( , );
(c) if the two variances are equal, the filter returns the
arithmetic mean value of the pixels in xy.
g x y
Adaptive Filters:
Adaptive, Local Noise Reduction Filters (3)
L
L
m
y
x
g
y
x
g
y
x
f
ˆ
(
,
)
(
,
)
(
,
)
2 2
Adaptive Filters:
Adaptive Median Filters (1)
min
max
med
The notation:
minimum intensity value in
maximum intensity value in
median intensity value in
intensity value at coordinates ( , )
maximum all xy xy xy xy z S z S z S
z x y
S
Adaptive Filters:
Adaptive Median Filters (2)
med min med max
max
The adaptive median-filtering works in two stages: Stage A:
A1 = ; A2 =
if A1>0 and A2<0, go to stage B Else increase the window size
if window size , re
z z z z
S
med
min max
med
peat stage A; Else output Stage B:
B1 = ; B2 =
if B1>0 and B2<0, output ; Else output
xy xy
xy
z
z z z z
z z
Adaptive Filters:
Adaptive Median Filters (2)
med min med max
max
The adaptive median-filtering works in two stages: Stage A:
A1 = ; A2 =
if A1>0 and A2<0, go to stage B Else increase the window size
if window size , re
z z z z
S
med
min max
peat stage A; Else output Stage B:
B1 = xy ; B2 = xy
z
z z z z
The median filter output is an impulse
or not
Example:
Periodic Noise Reduction by Frequency Domain Filtering
The basic idea
Periodic noise appears as concentrated bursts of energy in the Fourier transform, at locations corresponding to the frequencies of the periodic interference
Approach
Several interference components are present, the methods discussed in the preceding sections are not always acceptable because they remove much image information
Linear, Position-Invariant Degradations
( , )
( , )
( , )
Linear, Position-Invariant Degradations
An operator having the input-output relationship
( , ) ( , ) is said to be position invariant
if
( , ) ( , )
for any ( , ) and any and .
g x y H f x y
H f x y g x y
f x y
1 2
1
2
1 2
is linear
( , ) ( , ) ( , ) ( , )
and are any two input images.
H
H af x y bf x y aH f x y bH f x y
f f
Linear, Position-Invariant Degradations
In the presence of additive noise,
if is a linear operator and position invariant,
( , ) ( , ) ( , ) ( , )
( , ) ( , ) ( , )
( , ) ( , ) ( , ) ( , )
H
g x y f h x y d d x y
h x y f x y x y
G u v H u v F u v N u v
Estimating the Degradation Function
• Three principal ways to estimate the degradation function
1. Observation
2. Experimentation
Mathematical Modeling (1)
• Environmental conditions cause degradation
A model about atmospheric turbulence
2 2 5/6
( )
( , )
: a constant that depends on
the nature of the turbulence
k u v
H u v
e
k
Mathematical Modeling (2)
• Derive a mathematical model from basic principles
Mathematical Modeling (3)
0 0
Suppose that an image ( , ) undergoes planar motion,
( ) and ( ) are the time-varying components of motion
in the - and -directions, respectively.
The optical imaging process is perfect. T is th
f x y
x t y t
x y
0 0
0
e duration
of the exposure. The blurred image ( , )
( , ) T ( ), ( )
g x y
Mathematical Modeling (4)
0 0
0 0
2 ( ) ( )
0
Suppose that the image undergoes uniform linear motion
in the -direction and -direction, at a rate given by
( ) / and ( ) /
( , )
T j ux t vy t
x y
x t at T y t bt T
H u v e dt
e
2 [ ] /
0
( )
sin ( )
T
j ua vb t T
j ua vb
dt
T
ua vb e
Inverse Filtering
)
,
(
/
)
,
(
)
,
(
)
,
(
)
,
(
)
,
(
*
)
,
(
)
,
(
ˆ
)
,
(
)
,
(
)
,
(
ˆ
v
u
H
v
u
N
v
u
F
v
u
H
v
u
N
v
u
H
v
u
F
v
u
F
v
u
H
v
u
G
v
u
F
Inverse Filtering
1. We can't exactly recover the undegraded image because ( , ) is not known.
2. If the degradation function has zero or very
small values, then the ratio ( , ) / ( , ) could easily dominate th
N u v
Inverse Filtering
2
/ 2 (
EXAMPLE
The image in Fig. 5.25(b) was inverse filtered using the
exact inverse of the degradation function that generated
that image. That is, the degradation function is
( , )H u v ek u M v
5/6 2
/ 2)
, 0.0025
N
k
A Butterworth lowpass
function of order 10
Minimum Mean Square Error (Wiener) Filtering
N. Wiener (1942)
Objective
Find an estimate of the uncorrupted image such that the mean square error between them is minimized
2
2
)
ˆ
(
f
f
E
Minimum Mean Square Error (Wiener) Filtering ) , ( ) , ( / ) , ( ) , ( ) , ( ) , ( 1 ) , ( ˆ 2 2 v u G v u S v u S v u H v u H v u H v u F f H(u,v): degradation function
H*(u,v): complex conjugate of H(u,v)
|H(u,v)|2 = H(u,v)H*(u,v)
Sn(u,v)= |N(u,v)|2: power spectrum of the noise
Sf(u,v)= |F(u,v)|2 : power spectrum of the undegraded
Minimum Mean Square Error (Wiener) Filtering ) , ( ) , ( ) , ( ) , ( 1 ) , ( ˆ 2 2 v u G K v u H v u H v u H v u F
Left:
degradated image
Middle: inverse filtering
Constrained Least Squares Filtering
• In Wiener filter, the power spectra of the undegraded image and noise must be known. Although a constant estimate is sometimes useful, it is not always suitable.
Constrained Least Squares Filtering
2 2
* ( , )
( , ) ( , )
| ( , ) | | ( , ) |
( , ) is the Fourier transform of the function
0 1 0
( , ) 1 4 1
0 1 0
is a parameter
H u v
F u v G u v
H u v P u v
P u v
p x y
