By
Dimitri Nathanael Sunaringtyas 11501006
BACHELOR’S DEGREE in
MECHANICAL ENGINEERING - MECHATRONICS CONCENTRATION FACULTY OF ENGINEERING AND INFORMATION TECHNOLOGY
SWISS GERMAN UNIVERSITY The Prominence Tower
Jalan Jalur Sutera Barat No. 15, Alam Sutera Tangerang, Banten 15143 - Indonesia
STATEMENT BY THE AUTHOR
I hereby declare that this submission is my own work and to the best of my knowledge, it contains no material previously published or written by another person, nor material which to a substantial extent has been accepted for the award of any other degree or diploma at any educational institution, except where due acknowledgement is made in the thesis.
Dimitri Nathanael Sunaringtyas
__________________________________________________________________________________________
Student Date
Approved by:
Prof. Dr.-Ing. Andreas Schwung
__________________________________________________________________________________________
Thesis Advisor (Fachhochschule Südwestfalen Soest) Date
Dr. Rusman Rusyadi
__________________________________________________________________________________________
Thesis Co-Advisor (Swiss German University) Date
Dr. Maulahikmah Galinium, S.Kom, M.Sc
__________________________________________________________________________________________
Dean Date
ABSTRACT
CREATING AN INTERACTIVE VISUAL PROCEDURE FOR PLC IN A HOLOLENS
By
Dimitri Nathanael Sunaringtyas Prof. Dr.-Ing. Andreas Schwung, Advisor
Fernando Arévalo, M.Sc., Co-Advisor Dr. Rusman Rusyadi, Co-Advisor
SWISS GERMAN UNIVERSITY
In this thesis, an application is created for the HoloLens, a head wear which enables users to experience mixed reality. The application proves that it is plausible to create an interactive visual guide in the form of a Unity application for the HoloLens to aid engineers in the fault detection assessment of a machine. The HoloLens application utilizes WorldAnchors and Vuforia engine to make the holograms stand still. The HoloLens application shows the steps to solve a fault when detected through animations and CAD models. Users can interact with buttons to go through the guide and the HoloLens will automatically check if the fault is solved. The fault detection system is located in the core system, which gets data from an OPC-UA server connected to a bulk good system. This system is proven to be useful, as users rated
© Copyright 2019
by Dimitri Nathanael Sunaringtyas All rights reserved
DEDICATION
I dedicate this work to myself, because I did it.
ACKNOWLEDGEMENTS
I am most grateful to God for every morning I can wake up and breath.
I wish to express my sincere gratitude to Prof. Schwung for giving me the opportunity to do my thesis in Germany.
I am grateful to Fernando and Dr. Rusman for guiding me and encouraging me.
I also am grateful to my parents for always believing in me.
A warm gratitude to my beloved partner Leticia for always being patient with me, being by my side through it all, and never letting this thesis get the better of me.
I thank Chris for helping and supporting me throughout the work, specially with the CAD models and the animations.
Last but not least, I thank Albert for being there throughout all the hardships with all the visa and permission issues.
TABLE OF CONTENTS
Page
STATEMENT BY THE AUTHOR... 2
ABSTRACT... 3
DEDICATION... 5
ACKNOWLEDGEMENTS... 6
TABLE OF CONTENTS... 7
LIST OF FIGURES...12
LIST OF TABLES... 17
CHAPTER 1 - INTRODUCTION... 18
1.1. Background... 18
1.2. Research Question...19
1.3. Hypothesis...19
1.4. Objectives...20
CHAPTER 2 - LITERATURE REVIEW... 21
2.1. Theoretical Perspectives...21
2.1.1. Determining which faults in a system should be detected... 21
2.1.2. Determining how the HoloLens should communicate with the core system...21
Maintenance and Repair [6]... 23
2.2.2. Augmented Reality Application for Training in Maritime Operations. A Proof of Concept AR Application Developed for Microsoft HoloLens [12] ...25
2.2.3. Comparative Study of Augmented Reality SDKs [1]... 25
2.2.4. Augmenting Microsoft's HoloLens with vuforia tracking for neuronavigation [4]... 26
2.2.5. Fault Detection Assessment using an extended FMEA and a Rule-based Expert System [3]...27
CHAPTER 3 - RESEARCH METHODS... 28
3.1. Concept Development... 28
3.1.1. System Overview... 29
3.1.2. System Functionality...31
3.1.3. Component Overview...32
3.1.3.1. Communication Protocols... 33
3.1.3.2. Bulk Good System [11]...34
3.1.3.3. Core System... 37
3.1.3.4. HoloLens [8]...38
3.2. System Design and Implementation...39
3.2.1. Schedule Planning... 39
3.2.2. System Requirements...40
3.2.3. Fault Detecting Algorithm Design... 41
3.2.4. Hololens Unity Application Design... 42
3.2.5. Implementation...47
3.2.5.1. Fault Detecting Algorithm...47
3.2.5.2. HoloLens Unity Application... 53
3.2.5.2.2. Health... 63
3.2.5.2.3. Start... 64
3.2.5.2.4. Step...69
3.2.5.2.5. AnimationManager...72
3.2.5.2.6. TrackableEventHandler...74
3.2.5.2.7. Animator...77
3.3. Performance Test...80
3.3.1. Sufficiency Test...80
3.3.2. OPC-UA Data Collection and Processing Time... 81
3.3.3. MQTT Message Transmission Time...81
3.3.4. Marker Recognition Capability...82
3.3.5. User Experience Survey... 82
CHAPTER 4 - RESULTS AND DISCUSSIONS...83
4.1. Test 1: OPC-UA Data Collection and Processing Time... 83
4.2. Test 2: MQTT Message Transmission Time...87
4.3. Test 3: Marker Recognition Capability...89
4.4. Test 4: User Feedback... 91
4.5. Test 5: Final Application Testing 1 with the Bulk Good System... 92
4.6. Test 6: Final Application Testing 2 with the Bulk Good System... 100
4.7. Test 6: Sufficiency Test...105
