Visual Media Development of Wayang Culture Based on 2D Object Motion using Kinect Sensor
Awang Pradanaa,1,*, Syaiful Nugrahab,2, Salya Raterc,3
a Universitas Borneo Tarakan, Jl Amal Lama NO.1, Tarakan and 77111, Indonesia
b Universitas Musamus Merauke, Jl Kamizaun Mopah Lama, Merauke and 99600, Indonesia
c Polytechnic of South Aceh, Merdeka Street, Komplek Reklamasi Pantai, Tapaktuan city, 23751, Indonesia
1 [email protected] *; 2 [email protected]; 3 [email protected]
I. Introduction
The material on the diversity of arts and culture in the archipelago presents several elements of diversity. One of these elements is material and interaction in understanding cultural arts. Various events and occurrences in culture are composed of seven interconnected elements: social systems and organizations, knowledge systems, living equipment systems and technologies, livelihood systems, arts, and language systems [1]. When examining the cultural elements of a community group, one of the easiest ways to recognize this is through their creations. In this case, it focuses on literary works, namely folklore.
Computer graphics is a science closely related to digital images and graphics, the science that studies how simple shapes can be transformed from 2D images into 3D shapes[2], the process of which cannot be separated from image manipulation, image processing and pattern recognition. The programming language and the eXtensible Application Markup Language (XAML) are the scopes of work of this research. The programming language is a set of syntax or code that can be understood by computer machines, the programming language used for the compiler in this study is the C# language with the Windows WinFX framework to display attractive UI. The advantages of the open source Microsoft NET Framework make this source popular among developers and programmers worldwide, where they can develop and edit as they please without any restrictions. Today's media use increasingly encourages researchers to improve the efficiency of the impact of media methods with the delivery of material. The current use of media increasingly encourages researchers to increase efficiency on the effect of media methods in delivering material [3]. Three factors affect efficiency in
ARTICLE INFO A B S T R A C T
Article history:
Accepted
One of the works of art inherited by the ancestors of the Indonesian state, which is most in demand by the public, is Wayang art, which comes from folklore or fairytale. This concern for preserving the art of Wayang, a characteristic of Javanese regional traditions, is essential to preserving the history of folklore, especially in Indonesia.
However, preserving the Wayang art culture is not easy, sometimes;
sometimes,e difficulties or obstacles in learning how to introduce the art of Wayang to early childhood. In this new technology, art can be mediated or cast through interaction in the digital world. Inseparable from the concept of a system being developed at the time, one of the system concepts uses interaction technology for motion detection or motion sensors to the sensor. Microsoft introduced the Kinect sensor on gaming consoles in the Xbox retail package. Therefore, researchers use these sensors with the existence of these sensors and use the field of artistic graphic interaction to manipulate 2D image forms digitally.
Creation of simulations in digital form using the C# programming language and several OpenCV (Open Source Computer Vision) tools.
It is hoped that in the future, this visualization can be used by parents to introduce their children to the art of Wayang culture so that children can learn about some of the characters in Wayang art at an early age.
Copyright © 2022 Politeknik Aceh Selatan.
All rights reserved.
Keywords:
Microsoft Kinect 2D Image OpenCV C# Programming Art Of Wayang
of the Gatotkaca legend. The second is time efficiency in understanding the storyline material, and the third is designing and creating characters from the Gatot Kaca puppet character. The addition is efficient in determining whether the object's movement results follow the camera sensor capture.
Computer graphics is a science closely related to digital images and graphics, knowledge that learns how to transform a simple form of a 2D canvas image into a 3D canvas [4], the process not involving image manipulation, image processing, and recognition patterns. The programming language and OpenGL are the scopes of work of this research, the programming language itself is a set of syntax or code that computer machines can understand, and the programming language used for the compiler in this research is the C++ language with several additional libraries for Character rendering process support already built.
The explanation related to this research cannot be solved by relying on the provided library, but there is a factor from sensory acquisition to understand the concept of motion. Several things need a plot, scheme or Image capable of visualizing the character in a simulation, with the rules and parameters being precisely the same as the actual plot. Therefore, in this study, the author proposes a design that proposes a visual medium to explore the cultural elements of a community group. One of the easiest ways to see this is through their creations, and in this case, it focuses on literary works, namely folk tales made up of characters Gatot Kaca and Sri Kandi. The reason for choosing this object is that literary works can be viewed as a sign system that creates meaning through linguistic consciousness, culture, or individual consciousness in its creation.
II. The Proposed Method/Algorithm A. Folklore theory
Humans are born with an aesthetic ability, namely human nature, to enjoy all forms of beauty. People create an activity called art with reason and a desire for beauty. Art is the result of collaboration between the mind, becoming an imaginative force armed with a desire for beauty[5].
B. Storyboard
• Theme Background
Cultural art is one of the inheritances of ancestors or ancestors, representing the diversity of a tradition and belonging to an area. Along with the development of the world in the current era of globalization, culture is also experiencing a change in life values such as manners, manners, habits, behavior, environment and perspectives of people. The changes experienced by society are the carriers of the appearance of stories, one of which can be found in fairy tales.
• Character in the puppet
Wayang is an art form in Indonesia that has existed since ancient times. Wayang develops and focuses on the islands of Java and Bali with the influence of Hindu culture. On November 7, 2003, UNESCO, the United Nations agency responsible for culture, named Wayang the famous shadow puppet theater of Indonesia. A Priceless World Heritage in the Art of Speech (Masterpiece of the Oral and Intangible Heritage of Humanity) and by Presidential Decree No. 30 of 2018, December 17, 2018, has been designated National Puppet Day[6]. Wayang is an Indonesian culture that is almost deserted, especially for children. The things that cause Wayang art culture to be abandoned are the less attractive appearance and stories considered old-fashioned or boring. The author wants to create an exciting form for Wayang stories to be requested by many people, especially children, by creating story visualizations using 2D interactive visualization media, raising the story of Gatot Kaca and Sri Kandi guided by visualization media.
• Backsound
This study uses the background of the sound of gamelan music to maximize visualization and integrate with folklore artwork. In Javanese society, gamelan musical orchestras are usually called "Karawitan," meaning intricate, smooth, and trim. Javanese art at this time is still exhibited at official events such as weddings, harvest festivals and others[7].
C. Motion Capture
Motion capture is widely used in entertainment, education, sports and medicine. In the film, motion capture is used to capture human movements as actors and then use this information to move digital characters in a virtual environment[8]. The marker motion capture system uses markers attached to moving objects [9]. Marks are placed on any desired human body part to obtain movement data. At the same time, the markless motion detection system does not require a mark attached to a moving object[10]. This system was first developed at Stanford, MIT and the Max Planck Institute.
D. Virtual Environment
In the world of technology, digital development significantly impacts the virtual environment. The journey of the computerized world in the field of 2D or 3D becomes very fast and fast when developers compete for a revolution in the field of graphics [11] . Other research attempts try to create an excellent virtual environment in 2D visualization. Some things that need to be considered are style and texture [12].
III. Method
The flow of this research is how to design the system, starting from; Determining header parameters, variables used, and some of the system's primary functions (2D objects). Then implement characters and templates and design visualization interfaces. At the stage of the visualization experiment with several people of different heights, then analysis of the given visualization results, then consideration of the value of the articulated skeleton, whether it corresponds to the portion?
A. System Design
In the system design process in this sub-chapter, the design method in the primary process of the proposed research project is explained. The process of implementing the proposal is described using a flow chart for easy understanding and application of the proposed follow-up method. The explanation can also be seen in Figure 1.
Figure 1 Program Design
B. Application interface design
The communication mechanism between the user and the system, the application interface, can receive and provide information to the user to help track the problem until a solution is found. It is described in Figure 2, which depicts the application interface design in this study.
Figure 2 UI Program
The characteristics of the application design in this study are as follows. According to the numbering in Figure 2:
• The title of the scene’s visualization of the plot
• Previous, previous storyline button
• Next, the button to continue the story.
• Control to adjust the position of the Kinect camera.
• Character selection
• Sounds from the background of the storyline
C. Implementation of program development tools and materials
T This stage implements the software method. This implementation process involves writing program code and compiling it until a working program is created. The design freedom in implementing this algorithm consists of hardware and software. The hardware and software specifications used in this study are listed in Table 1.
Table I. Spesification Device
Scope Specifications
Hardware Processor AMD A4, 2.0 GHz
RAM 4096 MB
VGA 1024 MB
Sensor Kinect Kinect V1
Software Operation System Windows 10
Tools OpenCV 4.5.1 stable
D. Calibration test and maintenance programs
Tests are performed to determine whether the program's feature meets this study's requirements and the program design. When testing functions, the following functions should be used:
• Can recognize objects, maximum 2.
• The position of the object affects the height.
• For the object of the Gatot-Kaca character, the movement of the hands, feet and body, the real-time visualization can also see the joint skeleton of the hands, body and legs.
• For the object of Sri Kandi's movement of the hands, feet and body, the real-time visualization can also see the articulated skeleton of the hands, body and legs.
After functional testing, the next step is to maintain a comprehensive distribution program for the program implementation process, starting with calibrating the articulated skeleton, conducting user testing training, and ending with user handling of BUGs or sensor errors in the visualization program.
IV. Results and Discussion A. Interaction on application
The initial appearance of the program can be seen in Figure 3, and the application interface displays where several functions are used to run the application.
Figure 3. Application main page
• The functions of some features of the application include the following.
• Scene 1 begins the historical story of a Gatot Kaca and a heroine. It can be seen in Figure 4.
• Scene 2 feature is a story of a dispute between kingdoms. The Pringgodani kingdom, led by Gatot Kaca, is angry at having its territory conquered by "Kurawa," as seen in Figure 5. Five background storylines have been combined.
• Scene 3 is the final display of the story of the two puppet characters. It can be seen in Figure 6.
• Multiple buttons like Start, Next, Previous, Reload and Exit. It can be seen in figure 7.
§ Start to run programs/animations after detecting objects A and B.
§ "Next" is used to move to the slide/background of the story.
§ "Prev" is for going back to the previous story.
§ "Muat Ulang" function to return to the original story.
§ "Keluar" function to end the program/application.
Figure 4. "Scene 1" with two storylines
Figure 5. "Scene 2" contains five storylines
Figure 6. Scene 3 has two storylines
Figure 7. Features to run the application
B. Testing on application 1) Scene I
When running the application. The program or animation runs according to the capture of the moving object that will be carried out and is seen in the results. It can be seen in figure 8. in this scene, there are two background slides. The first one tells about Gatot Kaca, and "slide 2" tells the history of the Srikandi.
Figure 8. Screenshot scene I
2) Scene II
Upon entering "Scene 2", this 2D object of movement occurs according to the story's content. The results are shown in Figure 9. There are five slides in this scene with each story, and in this scene, the "Srikandi" character or object B appears.
Figure 9. Scene II with five storylines
3) Scene III
In scene 3, there is a plot that is summarized in 2 slides, and this scene is also a concluding short story of the journey of the story of Gatotkaca and Srikandi. It can be seen in Figure 10, a screenshot of the 3rd scene program.
Figure 10. Scene III, end of the storyline
C. User test result
Table 2. In the table, the test results are valid for all processes in the program and by the use case design that has been made, obtained from the suitability test results based on the percentage value, the value of which can be converted into a statement in the form of a predicate to show the quality [13]
of the hardware, as for the predicate that has been stated that is very good, good, and not good
Table 2. User Trial
No Point Testing Result Point
Very Good Good Poor
1. Calibration to Skeleton √
2. Suitability Joint Skeleton √
3. • Characteristic Object Recognition A
√
• Characteristic Object Recognition A & B same time
√
4. Progression move slide √
5. • 2D movement according to the movement of Object A
√
• 2D movement according to the movement of Object B
√
• 2D movement according to the movement of Object A & B Same time
√
V. Conclusion
Program tests show valid results for most use case design processes. The results of the Object Motion application compatibility test show that the application runs well on the Windows operating system. and the usability test results get pleasing scores for users who try or use it, and the user says that the user interface of this application is good and easy to use by the user.
Acknowledgment
Thanks to the faculty of engineering, computer engineering department, LPPM teams and the University of Borneo Tarakan for the research grants sourced DIPA/PNBP UBT funds from 2022.
References
[1] K. Jayanti, A. Loita, S. Al, and H. Sariwangi, “ANALISIS 7 UNSUR KEBUDAYAAN MASYARAKAT KECAMATAN CIGALONTANG KABUPATEN TASIKMALAYA Helda Safaat,” J. Pendidik. Seni, vol. 1, no. 2, pp. 2620–8598, 2018.
[2] P. Nurtantio Andono and T. (Universitas dian nuswantoro semarang) Sutojo, Konsep Grafika Komputer. penerbit andi, 2016.
[3] S. U. S. Supardi, L. Leonard, H. Suhendri, and R. Rismurdiyati, “Pengaruh Media Pembelajaran dan Minat Belajar Terhadap Hasil Belajar Fisika,” Form. J. Ilm. Pendidik. MIPA, vol. 2, no. 1, pp. 71–81, 2015, doi: 10.30998/formatif.v2i1.86.
[4] A. Pradana, A. Fadllullah, T. Elektro, F. Teknik, U. B. Tarakan, and G. Komputer, “Mekanika Fisika Untuk Gerak Objek 3D Berbasis Opengl,” vol. 9, no. 3, 2020.
[5] J. P. Maran, “Etnomatematika Dalam Ritus Agama Katolik Larantuka,” Pros. Sendika, vol. 5, no. 1, p.
473, 2019.
[6] Keputusan Presiden Republik Indonesia, KEPUTUSAN PRESIDEN TENTANG HARI WAYANG NASIONAL. 2018.
[7] R. Supanggah, “Bothekan Karawitan II: Garap,” Surakarta ISI Press Surakarta, 2009.
[8] J. Blomberg and M. Burrell, "An Ethnographic Approach to Design," pp. 1025–1052, 2012, doi:
10.1201/b11963-52.
[9] A. Kulshreshth, C. Zorn, and J. J. Laviola, "Poster: Real-time markerless Kinect based finger tracking and hand gesture recognition for HCI," IEEE Symp. 3D User Interface 2013, 3DUI 2013 - Proc., pp.
187–188, 2013, doi: 10.1109/3DUI.2013.6550241.
[10] Z. Liu, J. Huang, S. Bu, J. Han, X. Tang, and X. Li, "Template deformation-based 3-D reconstruction of full human body scans from low-cost depth cameras," IEEE Trans. Cybern., vol. 47, no. 3, pp. 695–
708, 2017, doi: 10.1109/TCYB.2016.2524406.
[11] A. Ardiyan, M. S. Mansuan, and J. Putra, “Sinematografi Animasi Dalam Media Virtual Reality Bermuatan Hantu Lokal Indonesia,” J. Dimens. DKV Seni Rupa dan Desain, vol. 4, no. 1, p. 87, 2019, doi: 10.25105/jdd.v4i1.4563.
10.30998/jurnaldesain.v5i02.2164.
[13] T. A. Ghaffur, “Analisis Kualitas Sistem Informasi Kegiatan Sekolah Berbasis Mobile Web Di Smk Negeri 2 Yogyakarta,” Elinvo (Electronics, Informatics, Vocat. Educ., vol. 2, no. 1, pp. 94–101, 2017, doi: 10.21831/elinvo.v2i1.16426.
