Filters
Another 3 3 smoothing filter
5.3 Formal Properties of Linear Filters
of Linear Filters
1 9·
⎡
⎢⎢
⎢⎣
0 0 0 0 0 0 1 1 1 0
0 1 1 1 0
0 1 1 1 0 0 0 0 0 0
⎤
⎥⎥
⎥⎦ 571 ·
⎡
⎢⎢
⎢⎣
0 1 2 1 0 1 3 5 3 1
2 5 9 5 2
1 3 5 3 1 0 1 2 1 0
⎤
⎥⎥
⎥⎦ 161 ·
⎡
⎢⎢
⎢⎣
0 0−1 0 0 0−1−2−1 0
−1−2 16−2−1 0−1−2−1 0 0 0−1 0 0
⎤
⎥⎥
⎥⎦
(a) (b) (c)
Fig. 5.8
Typical examples of linear fil- ters, illustrated as 3D plots (top), profiles (center), and approximations by discrete filter matrices (bottom). The
“box” filter (a) and the Gauss filter (b) are bothsmoothing filterswith all-positive coef- ficients. The “Laplacian” or
“Mexican hat” filter (c) is a difference filter. It computes the weighted difference be- tween the center pixel and the surrounding pixels and thus reacts most strongly to local intensity peaks.
center. The Gaussian filter is isotropic if the discrete filter matrix is large enough for a sufficient approximation (at least 5×5). As a low-pass filter, the Gaussian is “well-behaved” in frequency space and thus clearly superior to the box filter. The 2D Gaussian filter is separable into a pair of 1D filters (see Sec. 5.3.3), which facilitates its efficient implementation.2
Difference filters
If some of the filter coefficients are negative, the filter calculation can be interpreted as the difference of two sums: the weighted sum of all pixels with associated positive coefficients minus the weighted sum of pixels with negative coefficients in the filter regionRH, that is,
I(u, v) =
(i,j)∈R+
I(u+i, v+j)· |H(i, j)| −
(i,j)∈R−
I(u+i, v+j)· |H(i, j)|, (5.15) where RH+ and RH− denote the partitions of the filter with positive coefficientsH(i, j)>0 and negative coefficients H(i, j)<0, respec- tively. For example, the 5×5 Laplace filter inFig. 5.8(c)computes the difference between the center pixel (with weight 16) and the weighted sum of 12 surrounding pixels (with weights−1 or−2). The remain- ing 12 pixels have associated zero coefficients and are thus ignored in the computation.
While local intensity variations aresmoothedby averaging, we can expect the exact contrary to happen when differences are taken: local intensity changes areenhanced. Important applications of difference filters thus include edge detection (Sec. 6.2) and image sharpening (Sec. 6.6).
5.3 Formal Properties of Linear Filters
In the previous sections, we have approached the concept of filters in a rather casual manner to quickly get a grasp of how filters are defined and used. While such a level of treatment may be sufficient for most practical purposes, the power of linear filters may not really
2 See also Sec. E in the Appendix.
99
5Filters be apparent yet considering the limited range of (simple) applications seen so far.
The real importance of linear filters (and perhaps their formal elegance) only becomes visible when taking a closer look at some of the underlying theoretical details. At this point, it may be surprising to the experienced reader that we have not mentioned the term “con- volution” in this context yet. We make up for this in the remaining parts of this section.
5.3.1 Linear Convolution
The operation associated with a linear filter, as described in the pre- vious section, is not an invention of digital image processing but has been known in mathematics for a long time. It is calledlinear con- volution3 and in general combines two functions of the same dimen- sionality, either continuous or discrete. For discrete, 2D functions I andH, the convolution operation is defined as
I(u, v) = ∞ i=−∞
∞ j=−∞
I(u−i, v−j)·H(i, j), (5.16)
or, expressed with the designated convolution operator (∗) in the form
I=I∗H. (5.17)
This almost looks the same as Eqn. (5.5), with two differences: the range of the variablesi, jin the summation and the negative signs in the coordinates ofI(u−i, v−j). The first point is easy to explain:
because the coefficients outside the filter matrixH(i, j), also referred to as a filterkernel, are assumed to be zero, the positions outside the matrix are irrelevant in the summation. To resolve the coordinate issue, we modify Eqn. (5.16) by replacing the summation variables i, j to
I(u, v) =
(i,j)∈RH
I(u−i, v−j)·H(i, j) (5.18)
=
(i,j)∈RH
I(u+i, v+j)·H(−i,−j) (5.19)
=
(i,j)∈RH
I(u+i, v+j)·H∗(i, j). (5.20)
The result is identical to the linear filter in Eqn. (5.5), with the H∗(i, j) =H(−i,−j) being the horizontally and verticallyreflected (i.e., rotated by 180◦) kernel H. To be precise, the operation in Eqn. (5.5) actually defines the linearcorrelation, which is merely a convolution with a reflected filter matrix.4
3 Oddly enough the simple concept of convolution is often (though un- justly) feared as an intractable mystery.
4 Of course this is the same in the 1D case. Linear correlation is typically used for comparing images or subpatterns (see Sec. 23.1 for details).
100
5.3Formal Properties of Linear Filters
I I
H
Fig. 5.9
Convolution as a “black box”
operation. The original im- ageIis subjected to a linear convolution (∗) with the convo- lution kernelH, producing the output imageI.
Thus the mathematical concept underlying all linear filters is the convolution operation (∗) and its results are completely and suffi- ciently specified by the convolution matrix (or kernel)H. To illus- trate this relationship, the convolution is often pictured as a “black box” operation, as shown inFig. 5.9.
5.3.2 Formal Properties of Linear Convolution
The importance of linear convolution is based on its simple math- ematical properties as well as its multitude of manifestations and applications. Linear convolution is a suitable model for many types of natural phenomena, including mechanical, acoustic, and optical systems. In particular (as shown in Ch. 18), there are strong formal links to the Fourier representation of signals in the frequency domain that are extremely valuable for understanding complex phenomena, such as sampling and aliasing. In the following, however, we first look at some important properties of linear convolution in the accustomed
“signal” or image space.
Commutativity
Linear convolution iscommutative; that is, for any imageIand filter kernelH,
I∗H =H∗I. (5.21)
Thus the result is the same if the image and filter kernel are inter- changed, and it makes no difference if we convolve the imageI with the kernelH or the other way around. The two functionsI andH are interchangeable and may assume either role.
Linearity
Linear filters are so called because of the linearity properties of the convolution operation, which manifests itself in various aspects. For example, if an image is multiplied by a scalar constants∈R, then the result of the convolution multiplies by the same factor, that is,
(s·I)∗H = I∗(s·H) = s·(I∗H) . (5.22) Similarly, if we add two imagesI1, I2 pixel by pixel and convolve the resulting image with some kernelH, the same outcome is obtained
101
5Filters by convolving each image individually and adding the two results afterward, that is,
(I1+I2)∗H = (I1∗H) + (I2∗H). (5.23) It may be surprising, however, that simplyaddinga constant (scalar) valuebto the image doesnotadd to the convolved result by the same amount,
(b+I)∗H = b+ (I∗H), (5.24) and is thus not part of the linearity property. While linearity is an important theoretical property, one should note that in practice
“linear” filters are often only partially linear because of rounding errors or a limited range of output values.
Associativity
Linear convolution is associative, meaning that the order of successive filter operations is irrelevant, that is,
(I∗H1)∗H2=I∗(H1∗H2). (5.25) Thus multiple successive filters can be applied in any order, and multiple filters can be arbitrarily combined into new filters.
5.3.3 Separability of Linear Filters
A direct consequence of associativity is the separability of linear fil- ters. If a convolution kernelH can be expressed as the convolution of multiple kernelsHi in the form
H=H1∗H2∗. . .∗Hn, (5.26) then (as a consequence of Eqn. (5.25)) the filter operationI∗H may be performed as a sequence of convolutions with the constituting kernelsHi,
I∗H =I∗(H1∗H2∗. . .∗Hn)
= (. . .((I∗H1)∗H2)∗. . .∗Hn). (5.27) Depending upon the type of decomposition, this may result in signif- icant computational savings.
x/y separability
The possibility of separating a 2D kernel H into a pair of 1D ker- nels hx, hy is of particular relevance and is used in many practical applications. Let us assume, as a simple example, that the filter is composed of the 1D kernelshxandhy, with
hx=
1 1 1 1 1
and hy=
⎡
⎣1 1 1
⎤
⎦, (5.28)
respectively. If these filters are applied sequentially to the imageI, I= (I∗hx)∗hy, (5.29) 102
5.3Formal Properties of Linear Filters then (according to Eqn. (5.27)) this is equivalent to applying the
composite filter
H=hx∗hy=
⎡
⎣1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
⎤
⎦. (5.30)
Thus the 2D 5×3 “box” filterH can be constructed from two 1D filters of lengths 5 and 3, respectively (which is obviously true for box filters of any size). But what is the advantage of this? In the aforementioned case, the required amount of processing is 5·3 = 15 steps per image pixel for the 2D filterH as compared with 5 + 3 = 8 steps for the two separate 1D filters, a reduction of almost 50 %. In general, the number of operations for a 2D filter grows quadratically with the filter size (side length) but onlylinearly if the filter isx/y-separable. Clearly, separability is an eminent bonus for the implementation of large linear filters (see also Sec. 5.5.1).
Separable Gaussian filters
In general, a 2D filter isx/y-separable if (as in the earlier example) the filter functionH(i, j) can be expressed as the outer product (⊗) of two 1D functions,
H(i, j) =hx(i)·hy(j), (5.31) because in this case the resulting function also corresponds to the convolution product H = Hx∗Hy. A prominent example is the widely employed 2D Gaussian functionGσ(x, y) (Eqn. (5.14)), which can be expressed as the product
Gσ(x, y) =e−x2+y
2
2σ2 (5.32)
= exp(−2σx22)·exp(−2σy22) =gσ(x)·gσ(y). (5.33) Thus a 2D Gaussian filterHσG can be implemented by a pair of 1D Gaussian filtershGx,σ andhGy,σ as
I∗HσG=I∗hGx,σ∗hGy,σ. (5.34) The ordering of the two 1D filters is not relevant in this case. With differentσ-values along thexandyaxes, elliptical 2D Gaussians can be realized as separable filters in the same fashion.
The Gaussian function decays relatively slowly with increasing distance from the center. To avoid visible truncation errors, discrete approximations of the Gaussian should have a sufficiently large extent of about±2.5σto±3.5σsamples. For example, a discrete 2D Gaus- sian with “radius”σ= 10 requires a minimum filter size of 51×51 pixels, in which case the x/y-separable version can be expected to run about 50 times faster than the full 2D filter. The Java method makeGaussKernel1d()in Prog. 5.4 shows how to dynamically create a 1D Gaussian filter kernel with an extent of±3σ(i.e., a vector of odd length 6σ+ 1). As an example, this method is used for imple- menting “unsharp masking” filters where relatively large Gaussian kernels may be required (see Prog. 6.1 in Sec. 6.6.2).
103
5Filters
Prog. 5.4 Dynamic creation of 1D Gaussian filter kernels. For a givenσ, the Java method makeGaussKernel1d()returns a discrete 1D Gaussian filter ker- nel (floatarray) large enough to avoid truncation effects.
1 float[] makeGaussKernel1d(double sigma) {
2 // create the 1D kernelh:
3 int center = (int) (3.0 * sigma);
4 float[] h = new float[2 * center + 1]; // odd size 5 // fill the 1D kernelh:
6 double sigma2 = sigma * sigma; //σ2
7 for (int i = 0; i < h.length; i++) {
8 double r = center - i;
9 h[i] = (float) Math.exp(-0.5 * (r * r) / sigma2);
10 }
11 return h;
12 }
5.3.4 Impulse Response of a Filter
Linear convolution is a binary operation involving two functions as its operands; it also has a “neutral element”, which of course is a function, too. The impulse or Dirac function δ() is neutral under convolution, that is,
I∗δ=I. (5.35)
In the 2D, discrete case, the impulse function is defined as δ(u, v) = 1 foru=v= 0,
0 otherwise. (5.36)
Interpreted as an image, this function is merely a single bright pixel (with value 1) at the coordinate origin contained in a dark (zero value) plane of infinite extent (Fig. 5.10).
When the Dirac function is used as the filter kernel in a linear convolution as in Eqn. (5.35), the result is identical to the original image (Fig. 5.11). The reverse situation is more interesting, however, where some filterH is applied to the impulseδas the input function.
What happens? Since convolution is commutative (Eqn. (5.21)) it is evident that
H∗δ=δ∗H =H (5.37)
and thus the result of this filter operation is identical to the filter H itself (Fig. 5.12)! While sending an impulse into a linear filter to obtain its filter function may seem paradoxical at first, it makes sense if the properties (coefficients) of the filterH are unknown. Assuming that the filter is actually linear, complete information about this filter is obtained by injecting only a single impulse and measuring the result, which is called the “impulse response” of the filter. Among
Fig. 5.10 Discrete 2Dimpulseor Diracfunctionδ(u, v).
u v
u= 0
v= 0 δ(u, v)
104