1
ARDUINO BASED SMART GLOVES FOR BLIND, DEAF AND DUMB PEOPLE USING SIGN AND HOME AUTOMATION MODE
1K Pujakumari K Ravi Shankar, 2Er. Devendra Kuril Research Scholar, Assistant Professor
Department of Computer Science & Engineering, LNCT, Indore, India
Abstract- Clever Hand Gloves help incapacitate people to live with normal people. As dumb person cannot speak then this smart gloves helps him to convert his hand gesture into text and pre-recorded voice. This also help normal person to understand what he is trying say and reply accordingly. This Smart Gloves has facility of Home Appliance control from which a physically impaired person become independent to live. The main objective of the implemented project is to develop a reliable, easy to use, light weight smart hand gloves system which can minimizes the obstacles for disable people where they can stand with the race. Combination of fingers will be used for demonstrating multiple gestures.
Keywords: Gesture to text, Gesture to voice, wireless home.
1. INTRODUCTION
In our life we meet many disable people, some of them are partially and some are completely disables. The partially impaired people like dumb, deaf, paralysis in one leg or hand manages their life with difficulties and feel separate from others. Here communication plays major role to feel someone better and indulging them in an activity where they may say themselves as independent person. By this thought the project Smart Hand Gloves for Disable People is developed so that disable person can live his life as he wants. In this project, Flex Sensor plays the major role.
The glove is fitted with flex sensors along the length of each finger and the thumb.
The flex sensors give output in the form of voltage variation that varies with degree of bend.
This flex sensor output is given to the ADC channels of microcontroller. It processes the signals and perform analog to digital signal conversion. Further the processed data is sent in a wireless manner to the receiver section. In this section the gesture is recognized and the corresponding output is displayed on LCD and simultaneously a speech output is play backed through speaker.
The portability of this project is a major advantage. Thus with the help of this project, the barrier faced by these people in communicating with the society can be reduced to a great extent. In first mode single fingers will be used to show the gesture then combination of two fingers and finally combination of three fingers will be used. By closing fist the working will change to home automation mode through this a person can use home appliances.
2. LITERATURE REVIEW
Tushar Chouhan et al. implemented wired interactive glove, interfaced with a computer running MATLAB or Octave, with a high degree of accuracy for gesture recognition. The glove maps the orientation of the hand and fingers with the help of bend sensors, Hall Effect sensors and an accelerometer. The data is then transmitted to a computer using automatic repeat request (ARQ) as an error controlling scheme. The system is modeled for the differently abled section of the society to help convert sign language to a more human understandable form such as textual messages. The hardware section of their proposed design has its constituent electronic components as bend sensor, hall-effect sensor, accelerometer and Machine Learning Algorithms Used for Gesture Recognition
Jan Fizza Bukhari et al. proposed a system consisting of 21 sensors, out of which 9 were flex sensors, 11 were contact sensors and one for measuring acceleration. At least 9 analog voltage channels for flex sensors and at least 10 digital channels for contact sensors were required. To enhance the performance and accuracy of the system, signal conditioning for accelerometer readings was needed. Baud rate was set at 9600 and sampling rate at 500 samples per second. The device used was: NI PCI-6250. SCB-68A I/O block was used for interfacing signals to plug in DAQ devices with 68 pin connectors. The NI PXI 8330 module was used to connect an external controller (our desktop computer) to a PXI chassis. It was
2
paired with a PCI 8330 in desktop computer with a MXI cable running between the two. It contained a MXI-3 Multi-System Extension Interface for PCI.
Vajjarapu Lavanya et al. presented a system which can be made using small metal strips that are fixed on the five fingers of the glove. A copper plate is fixed on the palm as ground. According to their design implementation it is better to use a ground plate instead of individual metal strips because the contact area for ground will be more facilitating easy identification of finger position. The copper strips indicate a voltage level of logic 1 in rest position. But when they come in contact with the ground plate, the voltage associated with them is drained and they indicate a voltage level of logic 0.
Wang et al. presented a sign language recognition system that uses tensor subspace analysis to model a multi-view hand gesture. The hand recognition process is achieved through color segmentation. Input image that is in RGB color space is converted to YCbCr color space to ease the process of detecting the skin that employs the Back Propagation (BP) networks model. The sign language recognition is modeled and recognized using tensor.
Then, the matching process is carried out to identify the input hand gesture.
Harmeet Kaur, et al. in their paper, presented a brief description about the past attempts that were made to convert sign language to understandable form. In their paper, they have thoroughly scrutinized the previous attempts over this technology and also suggested various possible ways to implement the design of a simple smart glove.
Ajit Manware , Rajnish Raj , Amit Kumar , Tejaswini Pawar In order to share thoughts and to communicate with person with disability (dumb and deaf) the communication is the only medium so that they can convey the message to others. But there are lots of issues in communication with the person with disability. Therefore, a person with disability is not able to stand in the race with normal person. As we know that communication for a person who cannot hear is visual, not auditory. Usually dumb people use sign language for communication but they find difficulty in communicating with others who don’t understand sign languages.
So this creates barrier in the communication between these two communities. This paper aims to lower this barrier in communication with normal person. The main aim of the proposed system is to develop a cost effective system which can give voice to voiceless person with the help of Smart Gloves. It means that using smart gloves communication will not be barrier between two different communities and they will be able to communicate easily with the normal person. Use of smart glove by person with disability makes nation grow and also they will not differ themselves from the normal people.
Everyday communication with the hearing population poses a major challenge to those with hearing loss. For this purpose, an automatic sign language recognition system has been developed using Random Forest Classifier as a machine learning algorithm, and to translate the sign alphabets and common words into text and sound. A glove circuit has been designed with flex sensors, 3-axis accelerometer and gyroscope to capture the gestures or signs data. The finger bend data has been obtained from flex sensors on each finger while the accelerometer and gyroscope provided the trajectories of the hand motion. The data from the sensors has been passed through the trained model to recognize the gesture. The main purpose of Smart Glove is to provide an ease of sharing basic ideas, minimize communication gap and an easier collaboration for the hard of hearing people.
Everyday communication with the hearing population poses a major challenge to those with hearing loss. For this purpose, an automatic sign language recognition system has been developed using Random Forest Classifier as a machine learning algorithm, and to translate the sign alphabets and common words into text and sound. A glove circuit has been designed with flex sensors, 3-axis accelerometer and gyroscope to capture the gestures or signs data. The finger bend data has been obtained from flex sensors on each finger while the accelerometer and gyroscope provided the trajectories of the hand motion. The data from the sensors has been passed through the trained model to recognize the gesture. The main purpose of Smart Glove is to provide an ease of sharing basic ideas, minimize communication gap and an easier collaboration for the hard of hearing people.
3
Everyday communication with the hearing population poses a major challenge to those with hearing loss. For this purpose, an automatic sign language recognition system has been developed using Random Forest Classifier as a machine learning algorithm, and to translate the sign alphabets and common words into text and sound. A glove circuit has been designed with flex sensors, 3-axis accelerometer and gyroscope to capture the gestures or signs data. The finger bend data has been obtained from flex sensors on each finger while the accelerometer and gyroscope provided the trajectories of the hand motion. The data from the sensors has been passed through the trained model to recognize the gesture. The main purpose of Smart Glove is to provide an ease of sharing basic ideas, minimize communication gap and an easier collaboration for the hard of hearing people.
1.2 Problem Definition
Problems faced by the disable person regarding employment can be overcome by our method. So in the implemented work an intelligent microcontroller based system using Flex sensors is developed which is able to:-
Convert gesture into voice and text.
Help a person to control his home appliances if he could not walk to switchboard.
In today’s technology wireless gloves are not yet reliable because to be used as wireless, the gloves should have inbuilt battery and some electronics controller board which makes gloves heavier and may cause irritation. Thus wired equipment’s are preferred for patients and partial disable people.
2. PRACTICAL IMPLEMENTION 2.1. Block Diagram (SENDER)
4
Above block diagram represents the complete architecture of Smart Gloves for Disable People where it consist of two modes:-
1. Transmitter 2. Receiver
Transmitter includes components such as Flex Sensors, Arduino Mega, 16*2 LCD Screen, APR33A3 Voice Playback module, Transmitter, Receiver with Electronic Switches. The Arduino Mega Microcontroller Board is the heart of smart gloves device, it has interfaced with flex sensor, voice module, transmitter, and LCD screen. This whole assembly works on voltage of 5 volts and 9 volts supplied by power supply blocks. Receiver includes component such as power supply, receiver HT 12 D, Arduino Uno R3, relay and home appliances 3. WORKING
The work of this project start from movement of hand gloves where the flex sensors are attached, and the value of sensor changes when its experiences the bending. The flex sensor is another type of potentiometer are attach to the fingers when we bend the figure the value of the sensor get changes. The changing value of the sensor is depend upon the resistance and applied angle of the bending when we bend the sensor at some particular angle we can see the value of the resistance is increase and accordingly the output get reduced. On the other way we can say that it’s like an inversely proportional when the resistance of the sensor is increase at that instant the value of output decrease and accordingly we can make project by getting the advantage of this process.
3.1 Mode 0
When system get ON, it stays initially in Mode 0 where glove give status about user; his finger sensor value and position of palm and all these get displays on LCD Screen Attached to it. The user can generate different gesture and user can change Mode by particular patterns as coded. According the programming we have made there are three modes and modes are change when all the sensor give low output. Here we are trying to create the project such that it can work in the two different applications.
3.2 Mode 1
In this Mode, user can use feature of Voice and Playback by generating desired gestures then recorded voice can be played. When the mode gets activated, the sensor gives some value to the Arduino and according to the programming the signal passes from the Arduino to the voice and playback recorder (APR33A3). The recorder check which port or section is active at that time and sound which already recorded in the recorder it play. The sound we hear from the speaker which attached to the recorder.
3.3 Mode 2
In this mode a user can control the home appliances where we deal with the major part of the project. Here the output of the sensor is recorded to the Arduino and this value is matched with the programming by the Arduino. The Arduino check the value and matched us the programming and the output we can see from the LCD attached to the Arduino. The output value is send to the transmitter for transmit the data. The transmitter is attached with IC HT12E which encodes the data and finally sends from the antenna. At Receiver end the data get decode with the IC HT12D and send to the relay for switching purpose. The relay we use only for switching purpose which use to ON or OFF the switch
4. RESULTS
As a solution to problem definition, our results are more realistic and affordable than other research paper had claimed. Our Smart Gloves Prototype not only displays the gesture into text, but it also able to convert in voices. The results of our prototype are mentioned below.
4.1 Mode 1 (SIGN MODE)
The status of user’s palm and fingers, and Gesture for changing Modes is obtain as:-
5
Through bending finger user will be able to show his need.combination of fingers will be divided into seven parts thus seven basic need of user can be satisfied.working of finger is as below:
Finger 1 -first gesture command for first voice
“I am feeling thirsty”
Finger 2- second gesture command for second voice
“I am hungry”
Finger 3- Third gesture command for Third voice
“I want medicines”
Finger 4- fourth gesture command for fourth voice
“I want to go outside”
Finger (1+2)- fifth gesture command for fifth voice
“I want someone help”
Finger (2+3)- sixth gesture command for sixth voice
“I am not feeling well call doctor”
Finger (3+4)- seventh gesture command for seventh voice
“I want blind stick for blind people”
Bending all the fingers together will change the mode.mode 1 will shift to mode 2 i.e.home automation mode.
First gesture command for first Appliance:- a. For Switching ON: Switching ON first Appliance.
b. For switching OFF: Switching OFF first Appliance.
Second gesture for second Appliance:- a. For switching ON
b. Switching ON second Appliance.
c. For switching OFF
d. Switching OFF second Appliance.
5. CONCLUSIONS
This paper introduced the Smart Hand Gloves for Disable People. It will provide the more reliable, efficient, easy to use and light weight solution to user as compare to other proposed papers. This will responsible to create meaning to lives of Disable People. During this project we face various types of challenges. We have tried to minimize the problem. One problem is there to make it Wireless.
So, we observed and analyzed different research papers and products available in market which are bulky and used Wi-Fi technology which sometimes create a problem of irregular network, difficult of handle, and delicate in structure. Since this was a prototype our focus was to build a model, which can solve or minimize the communication problem for the disable people.
REFERENCES
1. K. V. Fale, Akshay Phalke, Pratik Chaudhari, Pradeep Jadhav. “Smart Glove: Gesture Vocalizer for Deaf and Dumb People”. International Journal of Innovative Research in Computer and Communication Engineering, Vol. 4, Issue 4, April 2016.
2. Ms. Pallavi Verma, Mrs. Shimi S. L., D r. S. Chatterji. ”Design of Smart Gloves”. International Journal of Engineering Research & Technology (IJERT), ISSN: 2278-0181.Vol. 3 Issue 11, November-2014.
6
3. Harmeet Kaur, Amit Saxena, Abhishek Tandon, Keshav Mehrotra, Khushboo Kashyap. “A Review Paper On Evolution Of Smart Glove”. International Journal of Scientific Research and Management Studies (IJSRMS), ISSN: 2349-3771, Volume 3 Issue 3, pg: 124-128.
4. Rafiqul Zaman Khan and Noor Adnan Ibraheem, “Hand Gesture Recognition: A Literature Review”, International Journal of Artificial Intelligence & Applications (IJAIA), Vol.3, No.4, July 2012.
5. Priya Matnani, “Glove Based and Accelerometer Based Gesture Control: A Literature Review”, International Journal of Technical Research and Applications e-ISSN: 2320-8163, www.ijtra.com Volume 3, Issue 6 (November-December, 2015), PP.
6. Abhinandan Das, Lavish Yadav and Mayank Sighal, 2016. “Smart Glove for Sign Language Communication”, IEEE conference.
7. Suraksha Devi and Suman Deb, 2017. “Low Cost Tangible Glove for Translating Sign Gestures to Speech and Text in Hindi Language”, 3rd IEEE International Conference on Computational Intelligence and Communication Technology, IEEE-CICT.
