Given the current situation, the goal of this project is to implement a contactless palmprint recognition system as a means of more hygienic user authentication. The result of the project is the creation of a system that can perform contactless palm print recognition in four main stages through computer vision techniques and Siamese neural networks.
Problem Statement and Motivation
The ultimate goal of the project is to develop a system that can perform contactless palmprint verification using computer vision and deep learning techniques. Draw the bounding box for the palm ROI by connecting the reference points that are close to the palm ROI area.
Project Scope
In addition, this project demonstrates innovation in the design and use of a Siamese neural network to learn a similarity function to perform verification. In addition, the goals of this project will be evaluated and the challenges that the project faces will also be mentioned.
Literature Review
Contactless Palmprint Image Retrieval Techniques
- Small Sample Biometric Recognition Based on Palmprint and Face Fusion .1 Brief
- Weaknesses/Limitations
- Recommendation
- Developing a contactless palmprint authentication system by introducing a novel ROI extraction method
- Brief
- Recommendation
- Multisampling approach applied to contactless hand biometrics .1 Brief
- Weaknesses/Limitations
- Recommendation
- Multisampling approach applied to contactless hand biometrics .1 Segmentation and ROI Extraction
- Feature Extraction from ROI using OLOF
- Modified SIFT to curb the drawbacks
- Brief
- Weaknesses
- Recommendations
- Feature Extraction Methods for Palmprint Recognition: A Survey and Evaluation [6] studied and reviewed several mainstream feature extraction methods that were
- Matching using Hausdorff Distance .1 Brief
- Strengths
- Weaknesses/Limitations
- Recommendations
- Brief
- Strengths
- Weaknesses/Limitations
- Recommendations
- N-shot Palm Vein Verification Using Siamese Networks
The downside of this system is that the speed of the hand movement was not regulated. The authors pointed out that using ReLU as an activation function will affect the training phase of the model.
System Methodology and Design
System Design Diagram
Brief description: This use case describes the palm registration process for new/existing users Trigger: Users who want to register their palm in the system. Brief description: This use case describes a case where a user needs to enter their profile names to get a profile or register a new profile. User - Who wants to verify his palm with the system to get his profile/watch Brief description: This use case describes a process for real-time palm print verification Trigger: User who wants to verify his palm.
Brief Description: This use case describes the case where the administrator intends to perform administrative work on the system, including editing detection thresholds and profile management. Brief Description: This use case describes the case where the administrator wants to edit the detection settings, including profile detection threshold, average confidence threshold, matching threshold, and strict registration mode. Short description: This use case describes the case where the administrator wants to verify 2 palm ROIs directly and bypass the registration and verification steps.
Short description: This use case describes the case when the admin wants to perform profile management. Brief Description: This use case describes the process of deleting a user's profile Trigger: Admin wanting to remove an existing profile from the system. Brief Description: This use case describes the process of resetting a user's profile Trigger: Admin wanting to reset a user's profile.
System Implementation
Tools and Technologies
- Hardware Setup
- Overview
- Block Diagram
- Implementation Details
OpenCV Python Library OpenCV will be used for image processing related operations in the project. NumPy Python Library NumPy will be used together with OpenCV to represent the images captured in a format as a NumPy array. TensorFlow Python Library TensorFlow will be the main deep learning library to be used in this project.
TensorFlow will be used for dataset preparation, image processing operations, data loading, model design, training, testing and evaluation. PyQT5 Python Library The PyQT5 GUI library will be used to package the various modules of the system into a complete application with a user interface. The camera lens has auto-focus technology and will be helpful during the capture process as it will be able to lock onto the palm of your hand.
Once the camera can be accessed by OpenCV, the next step will be to retrieve the video images from the connected camera. Then the frame will be flipped along the y-axis so that the video frame appears as a mirror image to the user. If the image quality is unsatisfactory, some adjustments to the camera lens will be required.
ROI Segmentation and Localization .1 Overview
- Block Diagram
- Overview
- Block Diagram
Otherwise, the video frame can then be passed to the ROI extraction module for segmentation. Referring to the example above, the region of interest (ROI) is a square area in the center of the palm that contains most of the wrinkles, outlines, and other textural features. Previous works suggested using traditional image processing techniques such as calculating the image's center of gravity and maximum inscribed circle to extract the ROI from the palm [11].
In this project MediaPipe Hands will be used to extract the ROI from the palm. The ROI extraction is modularized into a single process that accepts a video frame containing a palm, and then outputs the ROI extracted from the video. The function will translate the landmark points needed to draw the bounding box of the ROI into the actual coordinates relative to the video frame.
The ROI extraction module can then be used during application development to segment ROIs from camera inputs in real-time. Two databases namely, COEP and MediaPipe are extracted and compiled by themselves using the ROI extraction module. 88 On the other hand, MediaPipe ROI images were obtained by extracting ROIs from the palms of some volunteers of this project.
Model Design, Training and Fine-Tuning .1 Overview
- Implementation Details
- Overview
- Block Diagram
- Implementation Details
- Overview
- Block Diagram
The top layer of the original EfficientNet was removed as transfer learning will be adopted as the training method. The L1 distance is used as the distance layer in the Siamese network, the L1 distance calculates the absolute difference between 2 feature vectors and returns a. Since the output of the model is to predict whether both input palm images match, therefore.
The input to this module is the test partition of the dataset and also the trained conjoined model. The model was accepted for use, as no overfitting was detected, and the performance of the model was very satisfactory. This section walks through the implementation of the application that uses the trained Siamese model for inference.
The result of this module is to deliver an application that packages all the functionalities of the previous modules. QtDesigner was used as the main tool to develop the UI components of the application. Administration submenu that allows the administrator to adjust various detection thresholds and determine the stringency of the verification at runtime.
System Evaluation and Discussion
System Verification Plan .1 Overview
- Contactless Palm Capture via Camera
- Siamese Model
- VeriPalm App
- Overview
- Contactless Palm Capture via Camera
1 The "Back" button is clicked. The registration page closes and the window returns to the main menu. Test case Description User places his palm in front of the camera and is streamed live. Expected output Recorded video frame can display the palm on the window. Test case Name User removes palm in front of camera Test case Description User removes palm in front of camera.
Test Case Name Video Frame with Invalid Objects Passed to Segmentation Module Test Case Description A video frame containing objects other than palm images will be displayed. Test Case Name Two different identical palm images are input to the model Test Case Description Two prepared and different ROIs are used for verification. Test Case Description The Siamese embedding model is used to extract features from a palm image.
Test Case Description The user accesses the registration page from the main menu. Expected Output The registration page is displayed. Test Case Name Testing Valid ROI Segmentation in VeriPalm Test Case Description A palm is placed in front of the camera. Test Case Name Update Model Detection Thresholds Test Case Description Admin updates threshold values.
Concluding Remark
Additionally, a robust ROI extraction algorithm using MediaPipe Hands was also developed and was able to extract square ROIs from the palmprints as long as they were detected in the video frame. In addition, the ROI extraction algorithm was also used to extract ROIs from still palmprint images to be used as training samples. This move helped tremendously during model training, as the model benefited from more unique training samples.
In addition, a Siamese neural network was implemented that performs verification by computing the distance vector between 2 embeddings of 2 palm features. The Siamese neural network could generalize well to unseen patterns because it did not learn a predetermined number of palms, but instead learned to extract features that will determine whether the two input palms match. The EfficientNet CNN was also fine-tuned by building a custom upper layer to learn palmprint-independent features.
The Siamese Neural Network was subjected to a rigorous evaluation by analyzing its classification metrics to determine whether the model is suitable for application use. Finally, the last goal was achieved, which is to develop an application that uses all the developed modules. An application named VeriPalm has been supplied which contains a wide range of functionality as indicated in the sub-objectives.
Conclusion and Recommendation
Conclusion
Future Work
BIBLIOGRAPHY
Veldhuis, "Few-Shot Learning for Palmprint Recognition via Meta-Siamese Network," IEEE Transactions on Instrumentation and Measurement, vol. Salakhutdinov, “Siamese neural networks for single-photo image recognition,” In ICML Deep Learning Workshop, vol.
APPENDICES
FINAL YEAR PROJECT WEEKLY REPORT
- WORK DONE
- SELF EVALUATION OF THE PROGRESS Progress is on track
- WORK TO BE DONE
- PROBLEMS ENCOUNTERED None
- WORK TO BE DONE None
- SELF EVALUATION OF THE PROGRESS Progress is on track. Project could be delivered on time
- Poster
Designed by the Siamese model; the selected CNN embedding model is MobileNetV2 Trained this Siamese model with a subset of data prepared due to memory limitations Evaluated the model, performance was not satisfactory. Evaluated model, performance was satisfactory; However, performance can be increased if the entire dataset can be used for training. Start your application design by creating UML diagrams such as use case diagram and class diagram.
Studied how to add functionality to QT UI components through event-driven programming. Strictly tested VeriPalm application, all functions could work according to their requirements.
PLAGIARISM CHECK RESULT
Program / Course BACHELOR OF COMPUTER SCIENCE (HONOURS) Final Year Title Project Contactless Palm Verification Using Siamese Networks. Required originality parameters and limits approved by UTAR are as follows:. i) Overall similarity index is 20% and below, and. ii) Matching of individual listed sources must be less than 3% each, and (iii) Matching texts in consecutive block must not exceed 8 words. Note: Parameters (i) – (ii) must exclude citations, bibliography and text matches that are less than 8 words.
Note Supervisor/candidate(s) are expected to provide a soft copy of the complete set of originality report to Faculty/Institute. Form title: Supervisor's comments on originality report generated by Turnitin for submission of final year project report (for undergraduate programs). Based on the above results, I hereby declare that I am satisfied with the originality of the Final Year Project Report submitted by my student(s) as mentioned above.
FYP2 CHECKLIST
UNIVERSITI TUNKU ABDUL RAHMAN
