This project/internship titled "Navigation System for Blind People Using Machine Learning", submitted by Md. Abdullah Al-Baky, ID no. to the Department of Computer Science and Engineering, Daffodil International University has been accepted as satisfactory for the partial fulfillment of requirements for the degree of B.Sc. We hereby declare that this project has been carried out by us under the supervision of Md.
We also declare that neither this project nor any part of this project has been submitted elsewhere for the award of any degree or diploma. His endless patience, expert guidance, constant encouragement, constant and energetic supervision, constructive criticism, valuable advice, reading many inferior drafts and correcting them at all stages made it possible to complete this project. Professor, and Head, Department of CSE, for his kind help in completing our project and also to other faculty members and the staff of the CSE Department of Daffodil International University.
We would like to thank all our course fellow at Daffodil International University who participated in this discussion during the completion of the course work. Blind people interact with the entire system through a speech recognition module and all information is stored in the cloud.
LIST OF TABLES
INTRODUCTION
- Introduction
- Motivation
- Rationale of the Study
- Expected Outcome
- Project Management and Finance
- Report Layout
Nowadays it is becoming an important term to develop a system for blind people to navigate like normal people. This will not only provide guidance, but also ensure the perception of the environment as much as possible like normal people, such as which object is present in front of the blind person (eg person, car, bus, currency, chair, etc.), his current location, recognize familiar faces in front of him, etc. The stereo camera is used as a virtual eye for the blind, an ultrasonic sensor is used to detect obstacles in front of it and measure the distance to the blind person.
The camera captures the video sequence from the surroundings and then processes the query through the system with voice provided by the blind. The speech recognition module provides smart interaction with the system for blind people who can easily communicate through their formal English language. To overcome the challenges of blind people and ensure high reliability, credibility and safe navigation, we have developed a system that navigates them with safety, reliability and can get the environment in as much detail as normal people.
It is also more challenging to navigate the blind in perceiving the environment with advanced technology because it is not easy to navigate safely in the blind. Our research work aims to help blind people navigate safely and perceive the environment indoors and outdoors as much as possible with cost-effectiveness.
BACKGROUND
- Terminologies
- Related Works
- Comparative Analysis and Summary
- Scope of the Problem
- Challenges
In our project we worked with a huge amount of data with different types of identification. When we thought of this project we felt we wanted to make it very easy and we tried to add more features and we did. Our advisor helped us to move forward and advised that we need to learn a lot of things about machine learning.
We began to learn about the system for object detection, voice recognition, relative identity, focal length counting, etc. We had to learn all the basics of these topics and try to implement them. Again and again we encounter problems and learn again and again with the help of our consultant, with the help of Google, YouTube and various websites.
With the help of our advisor, we started our work and thought that our university authority would give us Raspberry pi. The total cost is about 15,000 taka, we have a lot of problems to collect even this amount of money.
RESEARCH METHODOLOGY
Research Subject and Instrumentation
Data Collection Procedure
In this research, we collected a currency dataset from BDT (Taka) notes to real-time images.
Statistical Analysis
Proposed Methodology
For data training we have used google colab because our machine can't do training data with GPU and as we know google colab provides free GPU to do our higher task. For coding we have used pycharm IDE and general text editor on pi os to implement and. To detect real world objects we have used coco image dataset which has 330k+.
We installed the required packages such as opencv-contrib-python, numpy, scikit-image, GPIO, pytesseract, etc. For the first one, detecting objects, we used a pre-trained yolov4 model (yolov4.model and yolov4 -tiny.model). In this part, we measured the distance of each person detected for the blind.
To determine the distance from the camera to a person, to use the triangle similarity and the triangle similarity goes something like this, we have an object with a known width W[17]. Let us briefly discuss the step-by-step face recognition procedure that we have implemented in our project. The basic idea comes with 68 specific points (called landmarks) that exist on every face - the tip of the chin, the outer edge of each eye, the inner edge of each eyebrow, etc.
In our thesis, we have used OCR to recognize text from roadside images or directions or text. We have used pytesseract python library to recognize text from images or videos in real time and pytesseract is a wrapper for Tesseract-OCR Engine. We have used the Ublox Neo GPS 7M module to get the latitude and longitude to get the exact location of blind people to track where they travel.
In our thesis, we implemented an API to send the data in the cloud to manage blind data for customer service and track it from anywhere. In a web application we retrieve the data from the database stored with API then what we used in our application.
Implementation Requirements
If blind people are in danger or need urgent communication with their guides, the blind will be able to contact the guide by voice command. Then an email will be sent to the guide's inbox with the current location of the blind.
Experimental Setup
Experimental Results
Distance measurement can be divided into two parts: one measures the distance to the nearest object using an ultrasonic sensor, and the other measures the distance to the person when a person's face is available to a blind camera. Ultrasonic provides the distance to the object and sensor, as shown in Figure 4.2.6, and the distance using the camera to the person, as shown in Figure 4.2.7.
Discussion
IMPACT ON SOCIETY, ENVIRONMENT, AND SUSTAINABILITY
- Impact on Society
- Impact on Environment
- Ethical Aspects
- Sustainability
Some people in our society try to misbehave with blind people and always try to fool them. So we have to do something for them and it is for them that will change their life. We want to add more features to our project and we will work widely and for more people.
SUMMARY, CONCLUSION, AND FUTURE WORK
Summary of the Study
Conclusion
In the coco dataset, 330 thousand datasets were used where more than 270 thousand datasets were labeled. But if we focus on the accuracy of the system, we can see that the accuracy of the system is more than 98%. This is an ultrasonic sensor and system based on raspberry pi and it also uses GPs and web application which is really effective and works really well.
We use deep learning which is really effective for object detection and currency recognition in many places. While this is a really good navigation system for the blind, there are some places where we can actually improve. We use ultrasonic sensors that can detect objects between 5 meters, which is not a very long range.
In the future, we will add more features and more sensors to make it more efficient.
Future Work
So it can guarantee the safe navigation of blind people in some complex and emergency situations with emergency contact method. Android and web applications will improve the system's user interface, more interactive interfaces and many options that allow the blind guide to track and manage his blind.
34;Object Recognition and Descriptive Analytics." In Computational Linguistics and Intelligent Systems: Proceedings of the 4th International Conference p. 34;Object Recognition for the Visually Impaired Using the SSD Algorithm." In 2020 International Conference on Systems, Computing, Automation and Networking (ICSCAN), p. Rosebrock, "Find distance from camera to object using Python and OpenCV", PyImageSearch, 2021 [Online]. Available: https://www.pyimagesearch .com find-distance-camera-objectmarker-using-python-opencv/.
Available: https://medium.com/@ageitgey/machine-learning-is-fun-part-4-modern-face-recognition-with-deep-learning-c3cffc121d78.
PLAGIARISM REPORT
![Figure 3.4.11: The above figure shows the affine transformation of different angles of the image[18]](https://thumb-ap.123doks.com/thumbv2/filepdfnet/10861965.0/37.918.194.784.703.870/figure-figure-shows-affine-transformation-different-angles-image.webp)
