This paper describes the process of developing an augmented reality (AR) color mobile application using storybook. It can enhance children's interaction in the physical coloring book by creating Augmented Reality mobile application that contains 3D interactive animated objects and interesting storybook-based audio.
Introduction
Background
What is colour?
According to (Wikipedia, Color, n.d.) color is the visual perceptual property that corresponds to humans and arises from the spectrum of light interacting in the eye with the spectral sensitivity of the light receptors. Another definition of color is the object-obsessed property of producing different sensations on the eye due to the way it reflects or radiates light (Oxford Dictionary).
Problem Statement
Learning colors starts at an early age where the children can notice the similarities and differences in shape and size. In addition to knowing the basic colors, they can also distinguish and identify the colors based on the different objects.
Objective of Study
Significant of the project
They are exposed to the advancement of different technologies, which is also an impetus to have different expectations in the learning process. Thus, an interactive digital storybook coloring book should be developed to allow children to have fun and engage in the learning process.
The Relevancy of the Project
The problems must be tackled as children are more advanced these days and easily get bored if the learning process does not run efficiently. With the success of the project, the learning process should definitely be more fun, exciting and interesting.
Scope of Study
Feasibility of Project within timeframe and scope
The project aims to allow children to understand more, facilitate and speed up the learning of color education. Meanwhile, the second phase of the project will be more focused on the development and implementation of the application, then the testing phase.
Literature Review
Theory of Augmented Reality
Concept of Augmented Reality
Hardware and Software of Augmented Reality
It is used to develop video games for web plugins, desktop platforms, consoles and mobile devices. The powerful Unity 3D is used to compile all the above list of software, programs and SDKs to design the application.
Types of Augmented Reality
Marker-based AR
Albright mentioned that the marker-based application typically uses images or visual markers that will produce a 3D image when directed by the user, normally using a camera on a smartphone. Examples of the use of marker-based AR can be seen in the Mini Cooper ad, as in the image below.
Markerless AR
Aurasma is available for Android and iOS users to download an app that uses GPS, camera, and other smartphone features to recognize surroundings and display interactive AR content without scanning a barcode (Albright, n.d.). Layar is available for iPhone and Android smartphones, as well as Aurasma, which displays digital information called “layers” in the users' field of view through their mobile device (Albright, n.d.).
Devices used for Augmented Reality
Augmented Reality Application
- Medical Application
- Games
- Advertising and Marketing
- Travel and Tourism
- Education
In medical learning or training, AR can be used to give students a better understanding of human bodies and how it works. Augmented Reality itself enhances the real-time physical environment, which is vivid and direct or indirect display of the real world, the elements of which are augmented by computer-generated sensory data such as video, sound, GPS location and images. Today, augmented reality has been dominated by advertising and marketing fields widely, especially in car and furniture advertising.
The AR app allows users to view virtual objects, seemingly sharing their space, which can be detected and manipulated using natural movements and hand gestures. Like other applications, Augmented Reality can also be used in the field of education in many types of study levels. It is closely related to the application of augmented reality in education, where students can participate and learn directly.
Current Mobile Apps of Augmented Reality on Colour in Market
- Animal Pants
- PLAY123
- Peek-a-Zoo
- Colour Memory Match
- AR Flashcard Shapes and Colours
- The Very Hungry Caterpillar 3D pop-up picture and story book app
The app offers different types of animals that need the help of children to find their correct pants size. A fact about this fun geometry app is that it won an Editor's Choice Award from Children's Technology Review in 2013. The app is purposely built for preschoolers and is also suitable for 2 year olds.
The application is most likely like this project in terms of matching the animal colors. The application was created by cooperation of parents and teacher, especially for their children and children. It also includes professional narration, fun music and positive feedback that will inform both children and parents of the children's actions.
Comparison on available apps in market
Storytelling in Children Education
Digital Storytelling with Augmented Reality
In general, storytelling is very important as it provides entertainment and education to human being (Zhou, Cheok, Pan, & Li, 2004). Traditional storytelling is the use of storybook that allows users to engage in multi-sensory experiences such as speech which is the story, vision by looking at the book and also touch by turning the pages and pointing to the books. The goal to implement AR in storybooks is to improve interactions and interactive learning in the sense of human-computer interaction, especially children, while maintaining the advantages of traditional physical books.
By using AR technology, it can potentially increase student interaction through experiences of various modalities, including speech, 3D audio, 3D graphics, and even touch.
Colour Learning for Children
Importance of Colour for Children Learning
Research Methodology
Methodologies
- Requirement planning phase
- User design phase
- Construction phase
- Cutover phase
In addition, the statement of the project problem (Section 1.2), objectives (Section 1.3) and importance and relevance of the project (Section 1.4) are also clearly stated. Existing products on the market are also considered as a reference for project initiation and future improvements. During the Final Year Project II (FYPII) period, some changes were made in terms of the story and design of the book.
This includes the mobile app interfaces, storybook storyline, and augmented reality development such as the 3D modeling part and programming. Creating the characters involves modeling, texturing, rigging and animating the models, which was a very challenging and learned activity. The snapshot of the character modeling process was also shown as Section 3.3.3 below.
Tools Development
Hardware and Equipment
Software, Platform and SDKs
Vuforia is an augmented reality software development kit (SDK) for mobile devices that enables the creation of augmented reality applications. This was used to edit and create the logo, interaction buttons and finally a markerless marker specifically for the color storybook. Unity 3D is a cross-platform game engine with a built-in integrated development environment (IDE) developed by Unity Technologies.
In this project, it is useful to create the AR application that integrates with the 3D models created using Autodesk Maya.
Project Activities
Mobile Application
The final designs stick to a simple yet efficient design, which has a good contrast graph and most of the markings for the final designs contain more than 3 stars, hence an efficient flow in performance.
Storybook Design
Model Development
In other words, 3D rigging is skeletal animation, the technique in computer animation of rendering a character in two parts: a surface rendering used to draw the character (called skin or mesh) and a hierarchical set of interconnected bones (the skeleton or rig) used to animate the mesh (pose and keyframe). Below is the example of ninja model rigging where bones are made to control the model for animation. Animation stage is very important to create the animation clips for the models to make movements.
Overall, in this project the animation controller known as Mecanim in Unity 3D was used to edit the animation flow to ensure a proper and smooth animation after setting the main animation frames. These procedures must be followed in sequence to obtain the functions of the application.
Challenges Encountered
Program Installation, Compilation & Learning Curve
Model Development
Time Constrain
Methods of Survey and Observation
Questionnaires Survey
Tinjauan proyek ini juga diperkenalkan dan umpan balik mereka disimpan untuk tujuan analisis. Tujuan kuesioner pada dasarnya untuk mengetahui kepekaan orang tua terhadap anak yang sedang belajar, selain untuk mengumpulkan tanggapan dan pendapat mereka tentang penemuan teknologi ini. Apakah Anda lebih suka mengajar anak-anak Anda menggunakan tradisional (buku) atau teknologi (aplikasi seluler)).
Adakah anda bersetuju jika teknologi tersebut diaplikasikan dalam pendidikan anak-anak supaya dapat meningkatkan minat dan kefahaman anak-anak? Teknologi semasa telah memperkenalkan realiti tambahan, di mana pengguna boleh melihat objek dalam 3D. Adakah anda bersetuju jika teknologi tersebut diaplikasikan dalam pendidikan anak-anak untuk meningkatkan minat dan pemahaman anak-anak?).
User Perception Survey
Observation
Key Milestone and Gantt Chart
Result and Discussion
Analysis of Survey Result
The questionnaires survey for parents
The rest assume that the smartphone or gadget is for their children to release tension. The figure above said that most of the parents agreed that their children spend more time on smartphones or devices especially playing games compared to reading books. Most of the parents mentioned that they used physical books as the way to teach their children about color.
The figure above indicates that most parents prefer to teach their children colors using a mobile app as they provide them with a smartphone. Referring to charts 6 and 7, parents believe that their children have their own interesting colors, especially when choosing toys and clothes. Based on the performance, they also trust their children to perform well in the area of color learning because it is a child's nature.
User Testing Results
The user perception survey result
This shows that the respondents agreed that the product can help children learn colors. The image above shows the graph of the respondent's opinion of the fun and interactivity of the product. This shows that most respondents agree that the product is fun and interactive.
The figure above shows the graph on the respondent's rate of acceptance and ease of use of the product. This shows that most respondents agree that the product is easy to use. The chart shows that all respondents agreed that children would like to use the product to learn color by rating it a 5.
Prototype
Storybook Design
AR Mobile Application
Conclusion and Recommendation
Conclusion
Recommendation
