DIFFERENCE
By
Jeffrey Ardi Santoso 11110042
A thesis submitted to the Faculty of
ENGINEERING AND INFORMATION TECHNOLOGY
in partial fulfillment of the requirements for the
BACHELOR’S DEGREE in
MECHATRONICS ENGINEERING
SWISS GERMAN UNIVERSITY EduTown BSD City
Tangerang 15339 Indonesia
Revision after the Thesis Defense on 14 July 2014
Jeffrey Ardi Santoso STATEMENT BY THE AUTHOR
I hereby declare that this submission is my own work and to the best of my knowledge, it contains no material previously published or written by another person, nor material which to a substantial extent has been accepted for the award of any other degree or diploma at any educational institution, except where due acknowledgement is made in the thesis.
Jeffrey Ardi Santoso
_____________________________________________
Student Date
Approved by:
Dr. Rusman Rusyadi, B.Eng, M.Sc
_____________________________________________
Thesis Advisor Date
Dipl.-Ing. Wahyu Guritno, M.Si
_____________________________________________
Thesis Co-Advisor Date
Dr. Ir. Gembong Baskoro, M.Sc
_____________________________________________
Dean Date
Jeffrey Ardi Santoso ABSTRACT
DESIGNING AND CONSTRUCTING COMPUTER VISION SUBSYSTEM FOR CHESS-PLAYING ROBOT USING SUCCESSIVE IMAGE DIFFERENCE
By
Jeffrey Ardi Santoso
Dr. Rusman Rusyadi, B.Eng, M.Sc, Advisor Dipl.-Ing. Wahyu Guritno, M.Si, Co-Advisor
SWISS GERMAN UNIVERISTY
Chess is a popular board game, played by many people worldwide since the 15th century and along with the development of computer technology, it is possible to play against the Artificial Intelligence in a computer. In spite of playing it in computer simulation, this thesis gives feeling of playing a real chess game with a robot as the opponent.
Computer vision system is applied to detect the chessboard corners and the movement made by the player. Using successive image difference method, the areas where the movement takes place are revealed. The centroids of these areas are compared to the chessboard corners and their cells are recognized. These data is used as the input for the Chess Engine and the best move to counter the player is sent to the robotic manipulator to move the chess piece.
Through several testing, this thesis delivers acceptable result. 88.09 % of the movements made by the player are detected successfully. It may be concluded that successive image difference method is applicable for chess-playing robot. However, it is limited to detect the movement without detecting the type of the chess piece.
Keywords: chess-playing robot, computer vision, image difference, SCARA robot.
Jeffrey Ardi Santoso
© Copyright 2014 by Jeffrey Ardi Santoso
All rights reserved
Jeffrey Ardi Santoso DEDICATION
I dedicate this works for my family, my advisor Dr. Rusman Rusyadi, B.Eng, M.Sc, and my co-advisor Dipl.-Ing. Wahyu Guritno, M.Si.
Jeffrey Ardi Santoso ACKNOWLEDGEMENTS
The author wishes to God for His grace during the thesis work so that I could finish this thesis on time.
I would like to express my gratitude to my advisor Dr. Rusman Rusyadi, B.Eng, M.Sc, for his guidance during my thesis work, and my co-advisor Dipl.-Ing. Wahyu Guritno, M.Si who has taught me the mechanical design. Also many thanks for their advice that motivates me to do this thesis work.
I want to give my special thanks to my parents, my brother, and my sister who always pray for me.
I also want to thank all of my colleagues in Mechatronics 2010, who always help and support me during this thesis work, especially Tommy Gunawan, Alfin Timothius, Ryan Iskandar, Steven Wibisono, Gandhi Winata, Dirga Pratama and Glenn Vialli.
Jeffrey Ardi Santoso
Jakarta, Indonesia.
Jeffrey Ardi Santoso TABLE OF CONTENTS
Page
STATEMENT BY THE AUTHOR ... 2
ABSTRACT ... 3
DEDICATION... 5
ACKNOWLEDGEMENTS ... 6
TABLE OF CONTENTS ... 7
LIST OF FIGURES ... 11
LIST OF TABLES ... 13
CHAPTER 1 - INTRODUCTION ... 14
1.1 Background ... 14
1.2 Thesis Purpose ... 15
1.3 Design Problem ... 15
1.4 Thesis Scope ... 15
1.5 Thesis Structure ... 16
CHAPTER 2 - LITERATURE REVIEW ... 17
2.1 Chess ... 17
2.2 Chess Graphical User Interface (GUI) ... 17
2.3 Chess Engine ... 18
2.4 Chess AI ... 19
2.5 Universal Chess Interface (UCI) ... 21
2.6 Robotic Arm... 21
2.6.1 Gantry ... 21
2.6.2 Articulated Robot ... 22
Jeffrey Ardi Santoso
2.6.4 Gripper ... 23
2.7 Motor... 23
2.8 Microcontroller ... 25
2.9 Single Board Computer (SBC) ... 25
2.10 Serial Communication ... 26
2.11 Computer Vision ... 27
2.11.1 Image Acquisition ... 27
2.11.2 Pre-processing ... 27
2.11.3 Feature Extraction ... 31
2.11.4 Detection/Segmentation ... 34
2.11.5 High-level Processing ... 34
2.11.6 Decision Making ... 34
CHAPTER 3 - METHODOLOGY ... 35
3.1 System Block Diagram ... 35
3.2 Process Block Diagram ... 36
3.3 Mechanical Design... 37
3.3.1 The Base... 38
3.3.2 Camera and Frontlight Mounting... 38
3.3.3 SCARA Robot Manipulator ... 38
3.3.4 Linear Actuator ... 39
3.3.5 Gripper ... 39
3.3.6 The Motor ... 39
3.4 Electrical Design ... 40
3.4.1 The Processor ... 40
3.4.2 The Microcontroller ... 40
Jeffrey Ardi Santoso
3.4.3 Webcam ... 41
3.4.4 The Motor Driver ... 42
3.5 Software Design ... 42
3.5.1 Chess GUI ... 42
3.5.2 Chess Engine ... 42
3.5.3 Computer Vision System ... 43
3.5.4 Camera Calibration ... 43
3.5.5 Pre-processing ... 44
3.5.6 Chessboard Detection ... 45
3.5.7 Chess Piece Movement Detection ... 49
CHAPTER 4 - RESULTS AND DISCUSSIONS ... 51
4.1 Lighting ... 51
4.2 Image Capture ... 52
4.3 Camera Calibration ... 52
4.4 Chessboard Detection ... 53
4.5 Chess piece movement Detection ... 58
4.6 K3Chess ... 62
4.7 Chess GUI and Computer Vision System Integration ... 63
4.8 Serial communication test ... 66
CHAPTER 5 - CONCLUSIONS AND RECCOMENDATIONS... 67
5.1 Conclusions ... 67
5.2 Recommendations ... 67
GLOSSARY... 68
REFERENCES ... 70
APPENDICES ... 72
APPENDIX A – Data Sheets ... 73
Jeffrey Ardi Santoso APPENDIX C – Bill of Material ... 139
CURRICULUM VITAE ... 140
Jeffrey Ardi Santoso LIST OF FIGURES
Figures Page
Figure 2.1 Chess GUI display ... 18
Figure 2.2 Possible moves tree ... 19
Figure 2.3 White player score prediction ... 20
Figure 2.4 White player score evaluation ... 20
Figure 2.5 Overhead gantry ... 21
Figure 2.6 Articulated robot ... 22
Figure 2.7 SCARA robot ... 22
Figure 2.8 Gripper ... 23
Figure 2.9 DC Servo motor ... 24
Figure 2.10 Microcontroller chip ... 25
Figure 2.11 Single Board Computer ... 26
Figure 2.12 Result of image smoothing ... 28
Figure 2.13 Threshold binary ... 29
Figure 2.14 Threshold binary inverted... 29
Figure 2.15 Truncate ... 29
Figure 2.16 Threshold to zero ... 30
Figure 2.17 Threshold to zero inverted ... 30
Figure 2.18 Dilated image... 30
Figure 2.19 Eroded image ... 31
Figure 2.20 Canny edge detector working principle ... 32
Figure 2.21 Line detected using hough transform ... 33
Figure 2.22 Blob detection process ... 33
Figure 3.1 System block diagram ... 35
Figure 3.2 Process block diagram ... 36
Figure 3.3 Chess-playing robot mechanical design ... 38
Figure 3.4 Linear actuator design ... 39
Figure 3.5 Odroid U-3 Single Board Computer... 40
Figure 3.6 Arduino Mega 2560 ... 41
Jeffrey Ardi Santoso
Figure 3.8 Radial distortion effect ... 43
Figure 3.9 Chessboard detection process sequence ... 45
Figure 3.10 Gray values around chessboard corner ... 46
Figure 3.11 Upper left chessboard corner calculation ... 47
Figure 3.12 Perspective warp result ... 48
Figure 3.13 Chess piece movement detection process... 49
Figure 4.1 Frontlight effect to the detection process ... 51
Figure 4.2 Over exposure image ... 52
Figure 4.3 Undistorted Image ... 53
Figure 4.4 Line detection on the chessboard ... 53
Figure 4.5 Gray values around a chessboard corner ... 54
Figure 4.6 Detected internal chessboard corner ... 55
Figure 4.7 Chessboard corner numbering sequence possibilities ... 55
Figure 4.8 Determined ROI ... 57
Figure 4.9 Warp perspective result ... 57
Figure 4.10 Chessboard corner detection on the warped image ... 58
Figure 4.11 Image before and after chess piece movement ... 58
Figure 4.12 Difference image ... 59
Figure 4.13 Eroded difference image... 59
Figure 4.14 Blob detection on the difference image ... 60
Figure 4.15 Hand detected during moving process ... 60
Figure 4.16 Castling movement blob detection ... 61
Figure 4.17 United difference image ... 61
Figure 4.18 Image difference separation using erosion ... 62
Figure 4.19 K3Chess User Interface ... 63
Figure 4.20 Possible moves and detected move ... 64
Figure 4.21 K3Chess and computer vision system synchronization ... 65
Figure 4.22 PC to Arduino communication result ... 66
Jeffrey Ardi Santoso LIST OF TABLES
Table Page
Table 4.1 K3Chess and computer vision system integration result………65
