Jeffrey Ardi Santoso

(1)

DIFFERENCE

By

Jeffrey Ardi Santoso 11110042

A thesis submitted to the Faculty of

ENGINEERING AND INFORMATION TECHNOLOGY

in partial fulfillment of the requirements for the

BACHELOR’S DEGREE in

MECHATRONICS ENGINEERING

SWISS GERMAN UNIVERSITY EduTown BSD City

Tangerang 15339 Indonesia

Revision after the Thesis Defense on 14 July 2014

(2)

Jeffrey Ardi Santoso STATEMENT BY THE AUTHOR

I hereby declare that this submission is my own work and to the best of my knowledge, it contains no material previously published or written by another person, nor material which to a substantial extent has been accepted for the award of any other degree or diploma at any educational institution, except where due acknowledgement is made in the thesis.

Jeffrey Ardi Santoso

_____________________________________________

Student Date

Approved by:

Dr. Rusman Rusyadi, B.Eng, M.Sc

_____________________________________________

Thesis Advisor Date

Dipl.-Ing. Wahyu Guritno, M.Si

_____________________________________________

Thesis Co-Advisor Date

Dr. Ir. Gembong Baskoro, M.Sc

_____________________________________________

Dean Date

(3)

Jeffrey Ardi Santoso ABSTRACT

DESIGNING AND CONSTRUCTING COMPUTER VISION SUBSYSTEM FOR CHESS-PLAYING ROBOT USING SUCCESSIVE IMAGE DIFFERENCE

By

Dr. Rusman Rusyadi, B.Eng, M.Sc, Advisor Dipl.-Ing. Wahyu Guritno, M.Si, Co-Advisor

SWISS GERMAN UNIVERISTY

Chess is a popular board game, played by many people worldwide since the 15^th century and along with the development of computer technology, it is possible to play against the Artificial Intelligence in a computer. In spite of playing it in computer simulation, this thesis gives feeling of playing a real chess game with a robot as the opponent.

Computer vision system is applied to detect the chessboard corners and the movement made by the player. Using successive image difference method, the areas where the movement takes place are revealed. The centroids of these areas are compared to the chessboard corners and their cells are recognized. These data is used as the input for the Chess Engine and the best move to counter the player is sent to the robotic manipulator to move the chess piece.

Through several testing, this thesis delivers acceptable result. 88.09 % of the movements made by the player are detected successfully. It may be concluded that successive image difference method is applicable for chess-playing robot. However, it is limited to detect the movement without detecting the type of the chess piece.

Keywords: chess-playing robot, computer vision, image difference, SCARA robot.

(4)

(5)

Jeffrey Ardi Santoso DEDICATION

I dedicate this works for my family, my advisor Dr. Rusman Rusyadi, B.Eng, M.Sc, and my co-advisor Dipl.-Ing. Wahyu Guritno, M.Si.

(6)

Jeffrey Ardi Santoso ACKNOWLEDGEMENTS

The author wishes to God for His grace during the thesis work so that I could finish this thesis on time.

I would like to express my gratitude to my advisor Dr. Rusman Rusyadi, B.Eng, M.Sc, for his guidance during my thesis work, and my co-advisor Dipl.-Ing. Wahyu Guritno, M.Si who has taught me the mechanical design. Also many thanks for their advice that motivates me to do this thesis work.

I want to give my special thanks to my parents, my brother, and my sister who always pray for me.

I also want to thank all of my colleagues in Mechatronics 2010, who always help and support me during this thesis work, especially Tommy Gunawan, Alfin Timothius, Ryan Iskandar, Steven Wibisono, Gandhi Winata, Dirga Pratama and Glenn Vialli.

Jeffrey Ardi Santoso

Jakarta, Indonesia.

(7)

Jeffrey Ardi Santoso TABLE OF CONTENTS

Page

STATEMENT BY THE AUTHOR ... 2

ABSTRACT ... 3

DEDICATION... 5

ACKNOWLEDGEMENTS ... 6

TABLE OF CONTENTS ... 7

LIST OF FIGURES ... 11

LIST OF TABLES ... 13

CHAPTER 1 - INTRODUCTION ... 14

1.1 Background ... 14

1.2 Thesis Purpose ... 15

1.3 Design Problem ... 15

1.4 Thesis Scope ... 15

1.5 Thesis Structure ... 16

CHAPTER 2 - LITERATURE REVIEW ... 17

2.1 Chess ... 17

2.2 Chess Graphical User Interface (GUI) ... 17

2.3 Chess Engine ... 18

2.4 Chess AI ... 19

2.5 Universal Chess Interface (UCI) ... 21

2.6 Robotic Arm... 21

2.6.1 Gantry ... 21

2.6.2 Articulated Robot ... 22

(8)

2.6.4 Gripper ... 23

2.7 Motor... 23

2.8 Microcontroller ... 25

2.9 Single Board Computer (SBC) ... 25

2.10 Serial Communication ... 26

2.11 Computer Vision ... 27

2.11.1 Image Acquisition ... 27

2.11.2 Pre-processing ... 27

2.11.3 Feature Extraction ... 31

2.11.4 Detection/Segmentation ... 34

2.11.5 High-level Processing ... 34

2.11.6 Decision Making ... 34

CHAPTER 3 - METHODOLOGY ... 35

3.1 System Block Diagram ... 35

3.2 Process Block Diagram ... 36

3.3 Mechanical Design... 37

3.3.1 The Base... 38

3.3.2 Camera and Frontlight Mounting... 38

3.3.3 SCARA Robot Manipulator ... 38

3.3.4 Linear Actuator ... 39

3.3.5 Gripper ... 39

3.3.6 The Motor ... 39

3.4 Electrical Design ... 40

3.4.1 The Processor ... 40

3.4.2 The Microcontroller ... 40

(9)

3.4.3 Webcam ... 41

3.4.4 The Motor Driver ... 42

3.5 Software Design ... 42

3.5.1 Chess GUI ... 42

3.5.2 Chess Engine ... 42

3.5.3 Computer Vision System ... 43

3.5.4 Camera Calibration ... 43

3.5.5 Pre-processing ... 44

3.5.6 Chessboard Detection ... 45

3.5.7 Chess Piece Movement Detection ... 49

CHAPTER 4 - RESULTS AND DISCUSSIONS ... 51

4.1 Lighting ... 51

4.2 Image Capture ... 52

4.3 Camera Calibration ... 52

4.4 Chessboard Detection ... 53

4.5 Chess piece movement Detection ... 58

4.6 K3Chess ... 62

4.7 Chess GUI and Computer Vision System Integration ... 63

4.8 Serial communication test ... 66

CHAPTER 5 - CONCLUSIONS AND RECCOMENDATIONS... 67

5.1 Conclusions ... 67

5.2 Recommendations ... 67

GLOSSARY... 68

REFERENCES ... 70

APPENDICES ... 72

APPENDIX A – Data Sheets ... 73

(10)

Jeffrey Ardi Santoso APPENDIX C – Bill of Material ... 139

CURRICULUM VITAE ... 140

(11)

Jeffrey Ardi Santoso LIST OF FIGURES

Figures Page

Figure 2.1 Chess GUI display ... 18

Figure 2.2 Possible moves tree ... 19

Figure 2.3 White player score prediction ... 20

Figure 2.4 White player score evaluation ... 20

Figure 2.5 Overhead gantry ... 21

Figure 2.6 Articulated robot ... 22

Figure 2.7 SCARA robot ... 22

Figure 2.8 Gripper ... 23

Figure 2.9 DC Servo motor ... 24

Figure 2.10 Microcontroller chip ... 25

Figure 2.11 Single Board Computer ... 26

Figure 2.12 Result of image smoothing ... 28

Figure 2.13 Threshold binary ... 29

Figure 2.14 Threshold binary inverted... 29

Figure 2.15 Truncate ... 29

Figure 2.16 Threshold to zero ... 30

Figure 2.17 Threshold to zero inverted ... 30

Figure 2.18 Dilated image... 30

Figure 2.19 Eroded image ... 31

Figure 2.20 Canny edge detector working principle ... 32

Figure 2.21 Line detected using hough transform ... 33

Figure 2.22 Blob detection process ... 33

Figure 3.1 System block diagram ... 35

Figure 3.2 Process block diagram ... 36

Figure 3.3 Chess-playing robot mechanical design ... 38

Figure 3.4 Linear actuator design ... 39

Figure 3.5 Odroid U-3 Single Board Computer... 40

Figure 3.6 Arduino Mega 2560 ... 41

(12)

Figure 3.8 Radial distortion effect ... 43

Figure 3.9 Chessboard detection process sequence ... 45

Figure 3.10 Gray values around chessboard corner ... 46

Figure 3.11 Upper left chessboard corner calculation ... 47

Figure 3.12 Perspective warp result ... 48

Figure 3.13 Chess piece movement detection process... 49

Figure 4.1 Frontlight effect to the detection process ... 51

Figure 4.2 Over exposure image ... 52

Figure 4.3 Undistorted Image ... 53

Figure 4.4 Line detection on the chessboard ... 53

Figure 4.5 Gray values around a chessboard corner ... 54

Figure 4.6 Detected internal chessboard corner ... 55

Figure 4.7 Chessboard corner numbering sequence possibilities ... 55

Figure 4.8 Determined ROI ... 57

Figure 4.9 Warp perspective result ... 57

Figure 4.10 Chessboard corner detection on the warped image ... 58

Figure 4.11 Image before and after chess piece movement ... 58

Figure 4.12 Difference image ... 59

Figure 4.13 Eroded difference image... 59

Figure 4.14 Blob detection on the difference image ... 60

Figure 4.15 Hand detected during moving process ... 60

Figure 4.16 Castling movement blob detection ... 61

Figure 4.17 United difference image ... 61

Figure 4.18 Image difference separation using erosion ... 62

Figure 4.19 K3Chess User Interface ... 63

Figure 4.20 Possible moves and detected move ... 64

Figure 4.21 K3Chess and computer vision system synchronization ... 65

Figure 4.22 PC to Arduino communication result ... 66

(13)

Jeffrey Ardi Santoso LIST OF TABLES

Table Page

Table 4.1 K3Chess and computer vision system integration result………65