It is hereby certified that ______CHAI GUO QIANG ID No: __18ACB01504 ) has completed this final year project entitled "Smart Home Appliance Control System Using Augmented Reality Technology_" under the supervision of Dr. I declare that this report titled "SMART HOME APPLIANCE CONTROL SYSTEM USING AUGMENTED REALITY TECHNOLOGY" is my own work except as cited in the references. Therefore, this project regarding control systems for smart home appliances using augmented reality technology is proposed to overcome this problem .
As the augmented reality trend is gradually growing, people are doubting whether they will control household appliances using augmented reality technology one day in the future. It is possible to achieve as there are some augmented reality based home appliance control systems today.
INTRODUCTION
- Problem Statement and Motivation
- Project Objectives
- Project Scope and Direction
- Contributions
- Background Information
- Report Organization
The project aims to develop an AR-controlled mobile application for smart home appliances to help the physically disabled and the elderly to control their home appliances effectively. AR application will really help partially physically disabled users to easily control their home appliances [1]. Users can manage their smart home devices with AR technologies, just as they can physically control the household appliances.
The proposed system can help people with physical disabilities to control home appliances in wheelchairs. In addition, the voice control function provides a hands-free way for users to control home appliances.
LITERATURE REVIEW
- Augmented reality-based marker tracking for Smart Home control .1 Brief Introduction
- Strength
- Weaknesses/Limitations
- Augmented Reality Control Home (ARCH) for Disabled and Elderlies .1 Brief Introduction
- Strengths
- Weaknesses/ Limitations
- ARIoT : Scalable Augmented Reality Framework for Interacting with Internet of Things Appliances Everywhere
- Brief Introduction
- Strengths
- Weaknesses/Limitation
- Smart AR Home .1 Brief Introduction
- Strength
- Weaknesses/Limitations
- Summary
Then the X10 sends the command to the server and communicates with the user's smartphone via the web service. The system will transfer the collected information from the scanned QR code to the web service when the user scans the QR code. If the web service is down, the user loses the ability to control the household appliances.
The system will then add the information of nearby IoT objects to the user's smartphone. Only the information of the selected IoT objects closest to the user will be displayed and controlled by the user.
SYSTEM METHODOLOGY / APPROACH
Design Specification
- Methodology
- Use Case Diagram
- Activity Diagram Login and Register
Brief Description: This use case describes the process of logging a user into their account in the system. Brief description: This use case describes the process of selecting a user space in the system. Brief description: This use case describes the process of user management of household appliances in the system.
Brief description: This use case describes the operation of a user control device in a system. Brief Description: This use case describes the process of a user device to control using the voice control feature in the system. Brief Description: This use case describes the process of a user control device in a system using AR.
The login and registration activity diagram shows how the user logs in and registers in the system. If the email and password exist in the database, then the system will display an error message. After logging into the account, the system will show the user a list of rooms.
When the user presses the voice button, the system redirects the user to the voice control page. For users who select the AR button, the system will redirect the user to the AR page. After pressing the microphone button, the system will direct the user to the voice command page.
Timeline
SYSTEM DESIGN
- System Block Diagram
- System Block Diagram Overview
- System Components Specification .1 Hardware
- Software Android Studio
- Circuits and Components Design
- AR button
Raspberry Pi needs to connect to Wi-Fi to get data from cloud database and send signal to relay switches to control on/off function of home appliances. The Raspberry Pi's GPIO (General Purpose Input/Output) pins will first be connected to the 5V relay. A python script is written to control the GPIO pins and update the user entry in the database.
Then the home appliances, connected to the 5V relay, receive on/off commands from the Raspberry Pi. The relay is connected to specific GPIO (General-Purpose Input/Output) pins on the Raspberry Pi. The other end of the relay is connected to the phase wire and the neutral wire.
The relay will receive a signal from the Raspberry Pi via the GPIO pins and turn on/off. In addition, the Python programming language uses the Visual Studio code editor to control the GPIO pins of the Raspberry pi and send data to the server. The 5V relay is connected to the Raspberry Pi GPIO connector and the household appliance wire.
To enable the 5V relay, one wire is connected from the GPIO 5V pin to the relay's VCC (Voltage Common Collector) and another wire is connected from the GPIO ground pin to the relay's ground pin. When the user presses , the system sends the to data to Firebase's real-time database and the Raspberry Pi retrieves the to data from the database in real time. The user can place his hand directly on the green button if he wants to open the light and place his hand on the red button if he wants to turn off the light.
SYSTEM IMPLEMENTATION AND EVALUATION
Hardware setup
- ARCore setup
- Firebase setup
Next, a wire is needed to connect to the GND (ground) relay from the Raspberry Pi GPIO ground pin. An AR button is created using Android Studio and the ARCore plugin, as shown in Figure 5.6. After adding the ARCore plugin to flutter, the button image is uploaded to the folder.
So, when the user places the AR button, the system will update the Firebase real-time database and the Raspberry Pi will receive the data and trigger on or off the relay. When a user registers with the system, the system will send the user's registration email address and password to the Firebase authentication. If user is new to the system, he or she can register with their email address and password and be able to control the home appliances after login.
The real-time database is set up by setting the read and write rules to true. This is because the system needs to get the results from the home appliances and write them to the database. Without setting the read and write rules to true, the system cannot read and write the database results.
To allow voice command, packages are imported in the specified file and the 'android.permission.RECORD_AUDIO' permission is added to the AndroidManifest.xml file since speech recognition requires access to the microphone. In addition, keywords such as "living room open fan", "bedroom open fan" and so on are added to the system. When the user says the keywords, the words will be highlighted and send the opt-out data to the Firebase database in real-time.
Setting and Configuration ARCore
The asynchronous function is designed to recognize the user's voice and extract the keyword to send to the firebase function. The firebase feature will recognize the highlighted keyword and enable or disable it on the home devices.
System Operation
After entering the email and password, the user can press the login button and the system will direct them to the Home AR main page. If the user has forgotten the password, he can press the forgotten password option and the system will allow the user to reset the password. If the user is new to the application, they can select the login option and register by entering their username, email and password.
Then the system sends a link to the user's registered email address and the user can reset their password by entering a new password of six and above digits. After logging in to the application, the system lets the user choose which home appliances in the room they want to take over. To control the home appliances, the user can use the power button or the AR button.
If the user selects the AR button, the system will direct the user to the AR page. The user can change the home appliances by pressing the icon with 3 dots in the upper right corner. To return to the home page, the user can press the home page icon and return to the home page, which is the page for the selected room.
Remote control of home appliances can be done as long as the phone has internet connection. The user can remotely turn on or off home appliances by pressing the power button. After the user sets the on or off button, the user can press the button and perform the on or off action.
System Testing and Performance Metrics Functionalities Testing
Test Action Expected Result Meets Expectation (✓/) User presses AR button The system will appear on. User select button The system will open the smartphone camera and scan any aircraft surface. User places the on button The system has successfully detected the level surface and the on button has been placed on the level surface.
User select off button The system opens the smartphone camera and scans any flat surface. The user places the off button The system has detected the flat surface and the off button is placed on the flat surface. The system will detect the aircraft and the AR button able to perform on or off function.
Conclusion
Project Review, Discussion and Conclusion
Novelties and Contributions
Future Work
APPENDIX A-1
FINAL YEAR PROJECT WEEKLY REPORT
- WORK DONE
- WORK TO BE DONE
- PROBLEMS ENCOUNTERED None
- SELF EVALUATION OF THE PROGRESS So far so good
- WORK TO BE DONE - Complete User Interface
- SELF EVALUATION OF THE PROGRESS Still on the track
- WORK TO BE DONE - Report writing
- SELF EVALUATION OF THE PROGRESS Still on the track, So far so good
- WORK TO BE DONE - Prepare for presentation
Learn how to use all the tools in Android Studio - Complete the installation of all hardware and software. WORK TO DO - Full UI - Full UI - Full AR feature - Full on-off feature. SELF-ASSESSMENT OF PROGRESS Still on the track, so good so far Still on the track, so good so far.
Appendix A-2: Poster
PLAGIARISM CHECK RESULT
Required originality parameters and UTAR approved limits are as follows:. i) The overall similarity index is 20% and below, and. ii) The matching of the individual sources mentioned must each be less than 3%, and (iii) The matching of texts in continuous blocks must not exceed 8 words. Note: Parameters (i) – (ii) exclude citations, bibliography, and text matches that are less than 8 words. Note: Supervisor/Candidate(s) is/are required to provide a copy of the full set of Originality Report to Faculty/Institute.
Based on the above results, I declare that I am satisfied with the authenticity of the Final Year Project Report submitted by my student(s) as mentioned above. Form title: Supervisor's Comments on Originality Report Generated by Turnitin for submission of Final Year Project Report (for Undergraduate Programs).
UNIVERSITI TUNKU ABDUL RAHMAN FACULTY OF INFORMATION & COMMUNICATION
TECHNOLOGY (KAMPAR CAMPUS)
