Step 6: Identification)
4. Experiments on different databases
In this section, the performance of the designed method is tested on different palmprint databases. More details and further updates can be found at [56]. The error rates of ROI localization are enumerated in Table 2. For IITD and COEP, the numbers in hard samples stand for PalmID_SampleID; for PolyU, the numbers stand for PalmID. After each experiment, the error cases are analyzed in detail.
4.1 ROI extraction for IITD Touchless Palmprint Database
Figure 4 shows the ROIs localized by the proposed method. Although the palm images in IITD Touchless Palmprint Database are captured in a black box, it is still
Figure 3.
Abnormal ROIs caused by complex background objects, difficult hand poses, and bad illuminations.
difficult to segment the palm region. The strong light source and the small enclosure lead to light reflections and ray occlusions, which generate many bright regions in the background and dark regions on the palm surface. Hence, the brightness infor- mation is not sufficient for segmenting the palm region. The color information should be utilized. As is shown in Figure 5, we randomly cropped some palm skin and background image patches to build a training set for segmentation. A SVM- based binary classifier is learned from the training dataset; the segmented palm
Database Error rate (%)
Hard samples
IITD Left 0.38 0037_0006, 0107_0002, 0152_0003, 0181_0001, 0209_0003 Right 0.31 0137_0001, 0140_0004, 0204_0003, 0204_0004
COEP 0.31 0103_0004, 0145_0003, 0159_0006, 0164_0002
PolyU 0.54 0004, 0039, 0073, 0109, 0127, 0187, 0223, 0224, 0245, 0246, 0259, 0271, 0273, 0287, 0293, 0307, 0311, 0328, 0379
Table 2.
ROI localization results of different databases.
Figure 4.
ROI localization results on IITD. (a) Left hand; (b) right hand.
Figure 5.
Image patches cropped from the palm skin and the black box. (a) Patches of the palm surface; (b) patches of the black box.
3.2 Abnormal detection and iterative localization
The keypoint detection method described above is based on local vision. The algorithms know whether the predefined keypoints,p1p6andvp1vp2, are obtained. But they cannot tell whether the detected points are the correct ones.
Background noise and abnormal palm poses will cause error localizations and thus generate abnormal ROIs. Those ROI images should be removed to avoid security risks. For example, during the process of sample registration, if a ROI falsely located in the background region, the black ROI image will be extracted and registered. This may cause big risk in real-world applications, since everyone can pass the system by a black image. Many deep learning-based image denoising methods have been proposed [54, 55], but, to some extent, they are too time-consuming to the target of palmprint image preprocessing. Hence, a high-speed abnormal ROI detection method is required. Here, the angle and scale of the ROI, the ratio of the back- ground region (if there exist background regions in the located ROI), and the ratio of the width of the two finger valleys are selected as features for abnormal detec- tion. They are denoted asθ,lt,rbg, andrh, respectively. The area of the ROI stands for the scale information, so we use the tangent line lengthltinstead. Then, for each time ROI localization, the feature vectorθ,lt,rbg,rh
can be obtained. To train a SVM-based abnormal detector, first, conduct the simple localization algorithm described above on the training set to generate different kinds of ROIs (as is shown in Figure 3). Then, separate the ROIs into normal and abnormal subsets. Last, a binary classifier can be trained by them. According to our experiments, all the false ROIs in Figure 3 can be successfully detected. With the abnormal detector, for line- scan-based method, once the current localized ROI is refused by the detector, it can move to the next position to iteratively detect the ROI. If the terminal condition is triggered, it means this image sample is unprocessable.
4. Experiments on different databases
In this section, the performance of the designed method is tested on different palmprint databases. More details and further updates can be found at [56]. The error rates of ROI localization are enumerated in Table 2. For IITD and COEP, the numbers in hard samples stand for PalmID_SampleID; for PolyU, the numbers stand for PalmID. After each experiment, the error cases are analyzed in detail.
4.1 ROI extraction for IITD Touchless Palmprint Database
Figure 4 shows the ROIs localized by the proposed method. Although the palm images in IITD Touchless Palmprint Database are captured in a black box, it is still
Figure 3.
Abnormal ROIs caused by complex background objects, difficult hand poses, and bad illuminations.
difficult to segment the palm region. The strong light source and the small enclosure lead to light reflections and ray occlusions, which generate many bright regions in the background and dark regions on the palm surface. Hence, the brightness infor- mation is not sufficient for segmenting the palm region. The color information should be utilized. As is shown in Figure 5, we randomly cropped some palm skin and background image patches to build a training set for segmentation. A SVM- based binary classifier is learned from the training dataset; the segmented palm
Database Error rate (%)
Hard samples
IITD Left 0.38 0037_0006, 0107_0002, 0152_0003, 0181_0001, 0209_0003 Right 0.31 0137_0001, 0140_0004, 0204_0003, 0204_0004
COEP 0.31 0103_0004, 0145_0003, 0159_0006, 0164_0002
PolyU 0.54 0004, 0039, 0073, 0109, 0127, 0187, 0223, 0224, 0245, 0246, 0259, 0271, 0273, 0287, 0293, 0307, 0311, 0328, 0379
Table 2.
ROI localization results of different databases.
Figure 4.
ROI localization results on IITD. (a) Left hand; (b) right hand.
Figure 5.
Image patches cropped from the palm skin and the black box. (a) Patches of the palm surface; (b) patches of the black box.
region can be seen in Figure 6. For palm skin patches, both the bright and dark regions are selected to learn a precise classification plane. In the color space, the palm can be easily segmented from the unicolor background. After palm region segmentation,vp1andvp2are detected by the local-extremum-based method.
Results: Figure 6 shows the IITD samples that are hard to process. If we cannot detect five fingertips and four finger valley points after palm segmentation, the system will directly return by giving an error code (as is shown in Figure 6(a) and (c)). If false keypoints are detected due to difficult palm poses, the finally extracted ROI images are abnormal images which should be discarded in real-world
applications (as is shown in Figure 6(b) and (d)).
4.2 ROI extraction for COEP Palmprint Database
The line-scan-based keypoint detection method is used for COEP. Figure 7 shows the ROI localization results on COEP database. Since the pegs used in their imaging setup may interfere the keypoint detection algorithm, we should delete
Figure 6.
Images cannot be localized in IITD database. (a) and (c) are the finger detection failed samples;
(b) and (d) are the ROI localization error samples.
Figure 7.
ROI localization result of the COEP database.
them first. The pegs’ colors are green, blue, and yellow. After removing the bright yellow pixels in the image, we extract the red channel from the original RGB image to conduct ROI localization algorithm. In this way, the green and blue pegs can be automatically removed (as is shown in Figure 7). Results: as is shown in Figure 8, after ROI localization, four images failed to be correctly localized. All of the four error cases are caused by closed fingers.
4.3 ROI extraction for PolyU Palmprint Database
For PolyU database, which contains 7752 images, the line-scan-based method is utilized to localize the ROI. At last, 42 samples failed to be localized. As is shown in Figure 9, most of them are caused by small finger valleys (palm pose) and unideal palm region segmentations (only grayscale information can be utilized). In Table 2, only the user ID is listed for the PolyU database.
Figure 8.
Images cannot be localized of COEP database. (a)–(d), (e)–(h), and (i)–(l) are the original, binary, and ROI localization images, respectively. The image ID of (a), (e), and (i) is 0103_0004; the image ID of (b), (f), and (j) is 0145_0003; the image ID of (c), (g), and (k) is 0159_0006; and the image ID of (d), (h), and (l) is 0164_0002.
Figure 9.
Hard samples of PolyU.
region can be seen in Figure 6. For palm skin patches, both the bright and dark regions are selected to learn a precise classification plane. In the color space, the palm can be easily segmented from the unicolor background. After palm region segmentation,vp1andvp2are detected by the local-extremum-based method.
Results: Figure 6 shows the IITD samples that are hard to process. If we cannot detect five fingertips and four finger valley points after palm segmentation, the system will directly return by giving an error code (as is shown in Figure 6(a) and (c)). If false keypoints are detected due to difficult palm poses, the finally extracted ROI images are abnormal images which should be discarded in real-world
applications (as is shown in Figure 6(b) and (d)).
4.2 ROI extraction for COEP Palmprint Database
The line-scan-based keypoint detection method is used for COEP. Figure 7 shows the ROI localization results on COEP database. Since the pegs used in their imaging setup may interfere the keypoint detection algorithm, we should delete
Figure 6.
Images cannot be localized in IITD database. (a) and (c) are the finger detection failed samples;
(b) and (d) are the ROI localization error samples.
Figure 7.
ROI localization result of the COEP database.
them first. The pegs’ colors are green, blue, and yellow. After removing the bright yellow pixels in the image, we extract the red channel from the original RGB image to conduct ROI localization algorithm. In this way, the green and blue pegs can be automatically removed (as is shown in Figure 7). Results: as is shown in Figure 8, after ROI localization, four images failed to be correctly localized. All of the four error cases are caused by closed fingers.
4.3 ROI extraction for PolyU Palmprint Database
For PolyU database, which contains 7752 images, the line-scan-based method is utilized to localize the ROI. At last, 42 samples failed to be localized. As is shown in Figure 9, most of them are caused by small finger valleys (palm pose) and unideal palm region segmentations (only grayscale information can be utilized). In Table 2, only the user ID is listed for the PolyU database.
Figure 8.
Images cannot be localized of COEP database. (a)–(d), (e)–(h), and (i)–(l) are the original, binary, and ROI localization images, respectively. The image ID of (a), (e), and (i) is 0103_0004; the image ID of (b), (f), and (j) is 0145_0003; the image ID of (c), (g), and (k) is 0159_0006; and the image ID of (d), (h), and (l) is 0164_0002.
Figure 9.
Hard samples of PolyU.