4.1 System Block Diagram
4.1.1 System Block Diagram Overview
The processes of the project were categorized into three parts: hardware, software and web service. The hardware used in this system will be a microprocessor (raspberry pi), 5V relay to support the AR control system. As Figure 4.1 shows, when the system receives input from the user, the system will use computer vision to detect and track home appliances objects using ARCore and some object detection algorithm. ARCore will only detect the plan surface like floor and table. Then user can place the AR button to the detected surface and the system will send on or off signal to the database. The database is updated in real-time to the cloud. Any changes made by voice command, button control or AR control from the system to the database will update in the user smartphone in real-time. Microprocessor, raspberry pi will connect to the relay switches to control the home appliances functionality. Moreover, the voice command provided by the user will be recognized and update in the database. Thus, user can either use their smartphone or voice to control the home appliance via the system design proposed.
Figure 4.1 Block diagram of proposed system
37
Bachelor of Computer Science (Honours)
Faculty of Information and Communication Technology (Kampar Campus), UTAR
Microprocessor
The microprocessor that used in this project is Raspberry Pi 3 Model B. Raspberry Pi must connect to Wi-Fi to receive data from cloud database and send signal to relay switches to control on/off function of the home appliances. GPIO (General-Purpose Input/Output) pins of Raspberry Pi will first connect to the 5V relay. A python script is written to control the GPIO pins and updates user input to the database. Then, the home appliances, which connect to the 5V relay, will receive on/off commands from the Raspberry Pi.
5V Relay Switch
The Relay is connected to specific GPIO (General-Purpose Input/Output) pins on the Raspberry Pi. The other end of the relay will connect to the live wire and neutral wire.
The relay will receive the signal from the Raspberry Pi through the GPIO pins and turn on/off.
AR Core
Through AR Core's plane detection APIs, ARCore allows a user's phone to observe its surroundings, understand the world, and interact with information. When the system identifies a plane surface, a bunch of dotted spots appears on the smartphone, and the user may then place the created AR button on the dotted area to operate household appliances.
38
Bachelor of Computer Science (Honours)
Faculty of Information and Communication Technology (Kampar Campus), UTAR
4.2 System Components Specification 4.2.1 Hardware
The hardware involved in this project is computer, raspberry pi, switches, actuators and android mobile device. A computer issued for the process of implementing the program and then it also used for applying AR technology on the system. A mobile device is used for testing and deploying this AR application.
Description Specifications
Model Acer AN515-51
Processor Intel Core i5-7200U Operating System Windows 10
Graphic NVIDIA GeForce GTX 1050 4GB DDR5
Memory 8GB DDR4 RAM
Storage 256GB SSD, 1TB SATA HDD
Table 4.1 Specifications of laptop
Description Specifications
Model Raspberry Pi 3 Model B
Processor 1.2 GHz 64-bit quad core ARM Cortex-A53 processor Operating System Raspberry Pi OS
Memory 1GB RAM
Table 4.2 Specifications of Raspberry Pi
Description Specifications
Model Huawei Nova 3i
CPU Hisilicon Kirin 710
Operating System Android 9
GPU Mali-G51 MP4
RAM 4GB
Storage 128GB
Table 4.3 Specifications of Android phone
39
Bachelor of Computer Science (Honours)
Faculty of Information and Communication Technology (Kampar Campus), UTAR
4.2.2 Software Android Studio
Android Studio is an official integrated development environment used for building android applications. Android Studio is used in this project for developing the application. The version of Android Studio used in this project is version 4.1, released in August of 2020. The programming languages supported by Android Studio are Java, C++, C and so on.
Firebase
Firebase is a platform that provides a wide range of services to developers, including analytics, authentication, databases, configuration, file storage, and push messaging.
Firebase is a cloud-based real-time database for storing, accessing, and managing data.
Account authentication and registration are also supported via the Firebase authentication service. Furthermore, Firebase supports C++ and Unity, which are required in the project.
Unity 3D
Unity is a powerful cross-platform 3D engine and a user-friendly development environment. It is used to build 2D, 3D and VR applications. Unity, established in the year 2005, is known as the world's most popular game engine. Many popular mainstream AR applications were developed using Unity, such as Pokemon Go. Unity offers a wide range of scripting languages for developers, such as C#, JavaScript, or Boo scripting. Some widely known IDEs that Unity supports are Visual Studio and MonoDevelop.
ARCore
Google's ARCore software development kit allows developers to construct Augmented Reality apps. In Android applications, it's also known as Google Play Service. The smartphone camera may be used to track motion using ARCore. It can also identify the size and position of any surface, including horizontal and vertical surfaces such as the ground, tables, and walls. Furthermore, light estimation is used to estimate the illumination conditions in the area.
Visual Studio Code
Visual Studio Code is a code editor develop by Microsoft. Visual Studio Code supports multiple programming languages including Java, JavaScript, Node.js, Python, C++ and so on. Visual Studio Code remote development using SSH method are used in this
40
Bachelor of Computer Science (Honours)
Faculty of Information and Communication Technology (Kampar Campus), UTAR
project to remote Raspberry pi. Besides, Python programming language is using Visual Studio Code editor to control GPIO pins of Raspberry pi and send data to the server.
4.3 Circuits and Components Design
Figure 4.2 2 Channel 5V Relay Module
Figure 4.3 Raspberry Pi GPIO pin design
The figure above shows the Raspberry Pi GPIO pin design. The 5V Relay is connected to the Raspberry Pi GPIO pin and the home appliances wire. To power on the 5V Relay, a wire is connected from the GPIO 5V pin to the relay’s VCC (Voltage Common Collector) and another wire is connected from the GPIO ground pin to the relay’s ground pin. The other side of the relay is connected to the wire of the home
41
Bachelor of Computer Science (Honours)
Faculty of Information and Communication Technology (Kampar Campus), UTAR
appliances. The Raspberry Pi GPIO pin on and off is control by the user smartphone.
When user press on, the system will send the on data to the Firebase real-time database and the Raspberry Pi will fetch the on data from the database in real-time.
4.4 AR button
Figure 4.4 AR detect the plane surface
Figure 4.1.1 above show the AR button to control the light. The green button indicates on, and the red button indicates off. User can directly place their hand to the green button when they want to open the light and place their hand to the red button when they want to turn off the light.
Figure 4.5 User turning on the light Figure 4.6 User turning off the light
42
Bachelor of Computer Science (Honours)
Faculty of Information and Communication Technology (Kampar Campus), UTAR