Design and Construction of an Automatic Guided Vehicle Based on Computer Vision for Navigation

(1)

By

Rheza Andika Prasetya 1-1111-042

BACHELOR’S DEGREE in

MECHANICAL ENGINEERING-MECHATRONICS CONCENTRATION FACULTY OF ENGINEERING AND INFORMATION TECHNOLOGY

SWISS GERMAN UNIVERSITY EduTown BSDCity

Tangerang 15339 Indonesia

Revision after the Thesis Defense on 3^rd August 2015

(2)

Rheza Andika Prasetya STATEMENT BY THE AUTHOR

I hereby declare that this submission is my own work and to the best of my knowledge, it contains no material previously published or written by another person, nor material which to a substantial extent has been accepted for the award of any other degree or diploma at any educational institution, except where due acknowledgement is made in the thesis.

Rheza Andika Prasetya _________________

Student Date

Revision after Thesis Defense on August 3^rd, 2015 Approved By:

Dr. Rusman Rusyadi, B.Eng., M.Sc. _________________

Thesis Advisor Date

Erikson F. Sinaga, S.T, M.Kom _________________

Thesis Co-Advisor Date

Dr. Ir. Gembong Baskoro, M. Sc _________________

Dean Date

(3)

Rheza Andika Prasetya ABSTRACT

DESIGN AND CONSTRUCTION OF AUTOMATIC GUIDED VEHICLE

BASED ON COMPUTER VISION FOR NAVIGATION SYSTEM

By

Rheza Andika Prasetya

Dr. Rusman Rusyadi, B.Eng., M.Sc., Advisor Erikson F. Sinaga, S.T, M.Kom., Co-Advisor

SWISS GERMAN UNIVERSITY

The main purpose of this thesis project is to design and construct a three wheeled AGV based on computer vision for navigation system, which will enable the robot to detect signs and follow the sign according to the sign direction. The navigation system is designed to be implemented for indoor use. Using camera as the main sensor with perspective transformation to transform the perspective view of the robot into Bird’s Eye View perspective. The communication system of the robot is established by the connection between the host computer and the microcontroller through serial communication using Firmata protocol. The system has been tested to follow the sign direction and the AGV robot is able to move forward and follow the direction of the sign.

Keywords: Automated Guided Vehicle, Three wheeled AGV, Bird’s Eye View, Signs Detection, Computer Vision, Navigation System, Distance Estimation, Firmata.

(4)

(5)

Rheza Andika Prasetya DEDICATION

I dedicate this work to my God, Jesus Christ,

My parents and my family, who always supports me all the time, And all of my friends and lecturers who helps me to finish this research.

(6)

Rheza Andika Prasetya ACKNOWLEDGEMENTS

First of all, I would like to thank God because of the blessing, this thesis work could be completed.

I also would like to express my gratitude to all peoples which support me in finishing my thesis work, DR. Rusman Rusyadi, B. Eng, M.Sc., as the thesis advisor because of his guidance and patience during this thesis work and to the co-advisor Mr. Erikson F.

Sinaga, S.T, M.Kom., for his input and encouragement. And also to Mr. Cepi M.

Hanafi, MT for his continuous support and help.

Furthermore, I would also like to thank to all of my colleagues in Mechatronics batch 2011 especially Felix, Meiviana Charisa whom work together with me in developing the AGV robot. And also not forget to thank all my lab-mates who always helping each other there is a problem occurred with the work and tools sharing: Muhammad Rizky, Benarivo, Kevin Inar, Brilly Nurhalim, Aditya, Sebastian Suwisar, Yogi Hamdani, Steven Leonardo, Michael Wijanarko.

I also would like to thank my parents, family and my special friend Amanda Pricillia for all of their motivation and support to complete the thesis work.

Sincerely, from the bottom of my heart, I thank all of you. Without all of your help and support this thesis would not be finished.

(7)

Rheza Andika Prasetya TABLE OF CONTENTS

ABSTRACT ... 3

DEDICATION ... 5

ACKNOWLEDGEMENTS ... 6

TABLE OF CONTENTS ... 7

LIST OF TABLES ... 10

LIST OF FIGURES ... 11

CHAPTER 1 - INTRODUCTION ... 13

1.1 Background ... 13

1.2 Thesis Objective... 14

1.3 Research Problem ... 14

1.4 Thesis Scope ... 14

1.5 Thesis Limitation ... 15

1.6 Thesis Organization ... 15

CHAPTER 2 - LITERATURE REVIEW ... 17

2.1 Introduction ... 17

2.2 Theoritical Perspective ... 17

2.2.1 A Vision-Based Automated Guided Vehicle System with Marker Recognition for Indoor Use [2] ... 17

2.2.2 Computer Vision ... 19

2.2.3 Camera Calibration [5] ... 19

2.2.4 Bird’s Eye View [6] ... 21

2.2.5 HSV Color Conversion [7] ... 21

2.2.6 Erosion and Dilation [8] ... 22

2.2.7 Lubuntu [9] ... 22

2.2.8 Secure Shell Connection ... 23

CHAPTER 3 - RESEARCH METHODS ... 24

3.1 Introduction ... 24

3.2 System Description ... 25

3.3 Mechanical Design... 26

3.3.1 Robot Base Plate ... 27

3.3.2 Camera Bracket ... 27

3.3.3 Casing ... 28

3.4 Electrical Components ... 29

3.4.1 Camera ... 29

3.4.2 Mini-PC ... 30

3.4.3 Wi-Fi module ... 31

(8)

3.4.4 Storage Device ... 32

3.4.5 Arduino Mega 2560 ... 32

3.4.6 DC Motor ... 33

3.4.7 Motor Driver ... 36

3.4.8 Power Source ... 36

3.5 Development Tools ... 38

3.5.1 Arduino IDE ... 38

3.5.2 Win32 Disk Imager for Odroid ... 39

3.5.3 Qt Creator [12] ... 40

3.5.4 OpenCV [8] ... 41

3.5.5 Qt-OpenCV-Multithreaded for Image Processing ... 42

3.5.6 Firmata ... 42

3.5.7 Secure Shell Remote Connection... 43

3.6 Software Design ... 43

3.6.1 Preliminary Programs ... 44

3.6.1.1 Camera Calibration ... 45

3.6.1.2 Bird’s Eye View Perspective Transformation... 46

3.6.2 Main Program Design ... 48

3.6.2.1 Firmata Communication Initialization ... 49

3.6.2.2 Image Perspective Transformation ... 50

3.6.2.3 Image Extraction ... 51

3.6.2.4 Sign Detection ... 53

3.6.2.5 Distance Estimation ... 55

3.6.2.6 Arduino Command ... 56

CHAPTER 4 - RESULTS AND DISCUSSIONS ... 58

4.1 Mechanical Result ... 58

4.1.1 Robot Assembly ... 58

4.1.2 Camera Bracket ... 59

4.1.3 Casing ... 61

4.2 Electrical Result ... 62

4.2.1 Motor Characteristic Test ... 62

4.3 Programming Result ... 65

4.3.1 Camera Calibration ... 65

4.3.2 Bird’s Eye View Perspective Transformation ... 66

4.3.3 Firmata Communication Setup Result ... 67

4.3.4 Image Extraction Result ... 67

4.3.5 Sign Detection Result ... 69

4.3.6 Distance Estimation Result ... 70

4.3.7 Qt-opencv-multithread Program ... 72

(9)

4.3.8 Secure Shell Remote Connection... 73

4.4 Test Result ... 73

4.4.1 Sign Detection Result due to the Light Intensity ... 73

4.4.2 The Angle of Error Analysis from Distance Estimation Algorithm ... 75

4.4.3 Qt-opencv-multithreaded Performance Test Result ... 78

4.4.4 Straight Line Movement Result ... 80

4.4.5 Curvy Line Movement Result ... 81

CHAPTER 5 - CONCLUSIONS AND RECOMMENDATIONS ... 83

5.1 Conclusion ... 83

5.2 Recommendation ... 83

GLOSSARY ... 84

REFERENCES ... 85

APPENDICES ... 86

APPENDIX A – Mechanical Design ... 87

A.1 Robot’s Base Plate Parts ... 87

A.2 Camera Bracket ... 92

A.3 Casing ... 97

APPENDIX B – Datasheets ... 105

B.1 Arduino Mega 2560 ... 105

B.2 Arduino Motor Shield L298 Motor Driver ... 108

B.3 Odroid-U3 ... 111

APPENDIX C – Qt Programming Code ... 112

C.1 qt-opencv-multithreaded.pro ... 112

C.2 fdevice.h ... 113

C.3 ProcessingThread.h ... 115

C.4 Structures.h ... 117

C.5 Util.h ... 118

C.6 CameraView.cpp ... 118

C.7 fdevice.cpp ... 127

C.8 FrameLabel.cpp ... 133

C.9 ProcessingThread.cpp ... 136

C.10 Util.cpp ... 146

APPENDIX D – Bill of Material ... 148

CURRICULUM VITAE ... 149