(a) (b) (c)
Figure 1.8: Illustration of variations in the details of the hand posture image with respect to illumination changes. (a) Poor illumination - dark image; (b) Normal (average) illumination - average contrast and (c) High illumination - high contrast.
0 50 100 150 200 250
0 200 400 600
Intensity values (I.V.)
Probability of I.V.
(a)
0 50 100 150 200 250
0 200 400 600
Intensity values (I.V.)
Probability of I.V.
(b)
0 50 100 150 200 250
0 200 400 600
Intensity values (I.V.)
Probability of I.V.
(c)
Figure 1.9: Histograms of (a) the dark image; (b) the average contrast image and (c) the high contrast image shown in Figure 1.8.
(a) (b)
Figure 1.10: Examples of hand posture images taken in varying background: (a) hand posture acquired in a uniform background and (b) hand posture images acquired in complex backgrounds. The hand posture images are taken from the Jochen Triesch static hand posture database [2].
under poor illumination with the corresponding histogram shown in Figure 1.9(a). The hand posture captured under normal illumination and the corresponding plot of image histogram are shown in Figure 1.8(b) and Figure 1.9(b) respectively. Similarly, Figure 1.8(c) is an example of the hand posture image captured under relatively high illumination and the corresponding plot of histogram is shown in Figure 1.9(c). Under poor illumination, the dynamic range of the intensity values is low and hence, the resultant image is dark and has a poor contrast.
In the case of normal illumination, the dynamic range of the intensity values has increased and distribution of the intensity values within the range is almost uniform. Hence, the resultant image is relatively bright and has
1.5 Issues in vision based hand posture recognition
good contrast. Similarly, under high illumination the dynamic range of the intensity values is relatively more and the resultant image has higher contrast than the poor and the normal illumination images.
Additionally, segmentation errors also occur while segmenting the hand postures from a complex or clut- tered background that contains several other objects with almost similar colour or geometrical characteristics as the hand region. The proper segmentation of hand postures is also affected if the colour of the user’s clothing coincides the skin colour. Some examples of the hand posture captured under different backgrounds are shown in Figure 1.10.
1.5.2 Geometrical distortions
The other major issue involved in accurate recognition of the hand postures is the geometrical distortions that occur due to geometrical transformations, variations in the hand posture parameters and variations due to changes in the angle of view.
1.5.2.1 Geometrical transformations
The geometrical transformations affecting the performance of the recognition unit includes the scale, the rotational and the translational changes induced during gesture acquisition as described below.
• The scale represents the spatial resolution of the acquired hand posture. The resolution will differ with respect to the variations in the hand geometry of the users and the distance between the gesturer and the camera.
• Rotation changes refer to the variation in the orientation of the hand posture that occurs either when the user rotates the hand while gesturing or when the camera is rotated along its plane within the field-of-view (FOV).
• Translational changes represent the variation in the spatial location of the hand posture that occurs due to the user’s movement of the hand.
1.5.2.2 Variations in the hand posture parameter
As explained in Section 1.2, the parameters that characterise the hand shape are the angles caused by the flexion/extension and the abduction/adduction movements of the finger joints. Among these, the flexion and the adduction movements are positive joint excursions and the extension and the abduction movements are negative joint excursions. The joint angle between two adjacent bone segments is measured by considering one
Little finger
Carpals Metacarpophalangeal joints (MP) DIP
PIP MP
Proximal interphalangeal joints (PIP)
Distal interphalangeal joints (DIP) Interphalangeal joints (IP)
Middle finger Ring finger
Thumb
Index finger
Middle finger
IP
(a)
Figure 1.11: Illustration of hand posture parameters using the hand skeleton. The joint angles represent the hand posture parameters.
20 20
(a)
[ 30 , 35 ]
MP! " # " #
0
M P ! #
90
M P! #
(b)
0 70
0
(c)
Figure 1.12: Illustration of (a) finger abduction; (b) MP joint range of motion, flexion-extension and (c) Palmar abduction and adduction of the thumb at the MP joint. The negative angle in (b) refers to the extension movement.
of the bone segments at close distance to the carpals as the reference axis. The procedure for measuring the hand posture parameters at the finger joints is illustrated using a hand skeleton in Figure 1.11(a). Similarly, a few examples illustrating the angular positions of the bone segments with respect to the abduction and the flexion movements of the metacarpal joints (MP) are shown in Figure 1.12. The maximum value of the motion parameters with respect to each finger joints are given in Table 1.2.
Based on these movement parameters, the hand postures can be considered as simple postures and complex postures. With simple postures, every individual finger is either extended or flexed to the maximum range.
Complex postures are those in which the fingers can be bent at any angle within the maximum range of motion in order to constitute a hand posture. In the case of complex postures, the joint angles defining a hand posture are only approximations that lie within a defined range of angular values. The structural variations with respect to a hand posture occur due to the changes in the flexibility of the user’s hand joints within the defined range.
Similarly, the hand posture parameters vary due to the variations in the hand geometry. An experimental study on the effects of the hand length and the flexibility of the joint angles in [68] states that the joint flexibility
1.5 Issues in vision based hand posture recognition
(a)
(b)
Figure 1.13: Examples of a hand posture taken at various angles of view. The figure illustrates the structural deviations or deviations in the appearance of the hand posture. Similarly, occlusion of certain parts of the hand can be observed at each angle of view. The hand posture images are taken from the Massey hand posture database for the American sign language [3].
Table 1.2: Maximum range of motion parameters defining the movements with respect to the thumb and the finger joints [7].
Extension Flexion Abduction Adduction
Fingers θMP θPIP θDIP θMP θPIP θDIP
20◦ 0◦
[−30◦,−35◦] 0◦ −20◦ 90◦ [100◦,120◦] [80◦,90◦]
Thumb θMP θIP θMP θIP
70◦ 0◦
0◦ −20◦ 50◦ 90◦
of the fingers increase with the increase in the hand length. Therefore, the variations in the flexibility of the user’s hand and the hand geometry result in the deviation of hand posture parameters due to which there is diversity in the appearance of a hand posture.
1.5.2.3 Variations due to the angle of view
In the field of imaging, the angle of view is known as the view-angle. The viewpoint refers to the position of the camera with respect to the object of focus [69]. The optimal choice of the viewing angle or the viewpoint is determined by the amount of perspective distortion. Perspective distortion is a phenomenon in which, the part of the object present at a larger distance from the camera appears to be smaller than the closer part of the same object and vice versa [69, 70]. As a result, the perceived shape of the object is distorted / altered. The distortion is caused if the focal plane is not parallel to the objects surface and/or not in level with the centre of the object.It means that the camera is not at equidistance from all the parts of the object [70]. The variations in the viewpoint result in structural deviations and self-occlusion of the fingers. A few examples illustrating the
structural variations and the occlusion errors in a hand posture due to variations in the view-angle are shown in Figure 1.13.