Development Of A Mind Game Using Electroencephalogram (EEG).

DEVELOPMENT OF A MIND GAME USING ELECTROENCEPHALOGRAM (EEG)

LIM KEAN THENG

This Report Is Submitted In Partial Fulfilment of Requirements for the Bachelor Degree of Electronic Engineering (Computer Engineering)

Faculty of Electronic and Computer Engineering University Technical Malaysia Malacca

ii

UNIVERSTI TEKNIKAL MALAYSIA MELAKA

FAKULTI KEJURUTERAAN ELEKTRONIK DAN KEJURUTERAAN KOMPUTER

BORANG PENGESAHAN STATUS LAPORAN

PROJEK SARJANA MUDA II

Tajuk Projek : DEVELOPMENT OF A MIND GAME USING ELECTROENCEPHALOGRAM (EEG)

Sesi Pengajian : 1 4 / 1 5

Saya LIM KEAN THENG mengaku membenarkan Laporan Projek Sarjana Muda ini disimpan di Perpustakaan dengan syarat-syarat kegunaan seperti berikut:

1. Laporan adalah hakmilik Universiti Teknikal Malaysia Melaka.

2. Perpustakaan dibenarkan membuat salinan untuk tujuan pengajian sahaja.

3. Perpustakaan dibenarkan membuat salinan laporan ini sebagai bahan pertukaran antara institusi pengajian tinggi.

4. Sila tandakan ( √ ) :

SULIT*

*(Mengandungi maklumat yang berdarjah keselamatan atau kepentingan Malaysia seperti yang termaktub di dalam AKTA RAHSIA RASMI 1972)

TERHAD** **(Mengandungi maklumat terhad yang telah ditentukan oleh organisasi/badan di mana penyelidikan dijalankan)

TIDAK TERHAD

Disahkan oleh:

__________________________ ___________________________________

(TANDATANGAN PENULIS) (COP DAN TANDATANGAN PENYELIA)

iii

“I hereby declare that this report entitled Development of a Mind Game using Electroencephalogram (EEG) is a result of my own work except for the notes that

have been cited clearly in the references.”

Signature :

iv

“I hereby declared that I have read this report and in my opinion this report is sufficient in terms of the scope and quality for the award of Bachelor of Electronics

Engineering (Computer Engineering) with Honours”

Signature :

v

vi

ACKNOWLEDGEMENT

vii

ABSTRACT

Electroencephalogram (EEG) based technology are becoming more popular in games design and development as new EEG equipment with portability, stylish wearable and easier usages are available in the market. Such technologies could be implemented in e-learning, entertainment and medical application for the disabled people. This project is to identify the importance of the EEG systems and seek the requirements to make a real-time, manageable and user-friendly BCI for gamers. The main objective of this project was to develop a playable game controlled by EEG signals. This includes building a mind game algorithm that making use of the available EEG device in the market. To achieve the aim, EEG signals were acquired, analyzed and classified. Finally, the classification signal was used to control a real-time feature in the 3D concentration mind game developed by the Orge3D game engine.

viii

ABSTRAK

Electroencephalogram (EEG) teknologi menjadi semakin popular dalam reka bentuk permainan dan pembangunan peralatan EEG baru dengan kelebihan mudah alih, kelaziman pakai dan bergaya. Kini, electroencephalogram (EEG) teknologi sebegini boleh didapati di pasaran. Teknologi seperti itu boleh dilaksanakan dalam pembelajaran, hiburan dan aplikasi perubatan untuk orang kurang upaya. Projek ini mengenal pasti kepentingan sistem EEG untuk mereka bentuk sistem permainan yang menarik untuk pemain. Fokus utama projek ini adalah mereka satu permainan yang boleh dimain kawal oleh isyarat EEG. Ini termasuk membina algoritma sistem permainan menggunakan alat EEG yang terdapat di pasaran. Untuk mencapai matlamat ini, isyarat EEG telah diperolehi, dianalisis dan dikelaskan. Akhir sekali, isyarat klasifikasi itu digunakan untuk mengawal entiti yang terdapat dalam permainan yang telah direka guna Orge3D permainan enjin.

ix

TABLE OF CONTENT

CHAPTER CONTENT PAGE

TITLE i

CONFORMATION REPORT STATUS ii

SUPERVISORS CONFORMATION iii

DECLARATION iv

DEDICATION v

ACKNOWLEDGEMENT vi

ABSTRACT vii

ABSTRAK viii

TABLE OF CONTENT ix

LIST OF TABLE xii

LIST OF FIGURES xiii

LIST OF ATTACHMENTS xv

1 INTRODUCTION 1

1.1 Problem Statement 3

1.2 Objectives 3

1.3 Scope of Work 4

2 LITERATURE REVIEW 5

2.1 Theories of Electroencephalogram (EEG) 5

2.2 Brain Activity Patterns 10

x

2.4 Techniques 14

2.4.1 Classification of EEG system 14

2.4.1.1 Emotion 14

2.4.1.2 Movement 15

2.4.2 Signal Processing Algorithms 15

2.4.2.1 Asynchronous vs. Synchronous 15

2.4.2.2 Spatial Filtering 16

2.4.2.3 Frequency and ERP analysis 17 2.4.2.4 Fractal Dimension Model 17

2.4.2.4.1 Higuchi 18

2.4.2.4.2 Box-counting Method 18

2.5 EEG-based Game Design 19

2.5.1 ORGE3D Game Engine 20

2.6 OpenViBE software platform 24

2.6.1 OpenViBE in Touch Approach 25

2.6.2 Brain Activity 25

2.6.3 Virtual Movements 26

3 METHODOLOGY 27

3.1 Project Implementation 27

3.1.1 EEG System Implementation 29

3.1.1.1 Virtual Reality Peripheral Network(VRPN) 30

3.1.2 Game Algorithm Design 30

3.1.3 Signal Processing Technique and Classification 32 3.1.4 System Evaluation and Enhancement 32 3.2 Electroencephalogram (EEG) System Specification 33

4 RESULTS AND DISCUSSION 35

4.1 Game Implementation 36

4.2 Data Acquisition 36

xi

4.2.1.1 EEG Signal Data for Subject 1 39 4.2.1.2 Beta Band Power Subject 1 40 4.2.1.3 EEG Signal Data for Subject 2 41 4.2.1.4 Beta Band Power Subject 2 42

4.3 Data Processing 43

4.4 Game Control 44

4.5 Discussions 45

5 CONCLUSION AND RECOMMENDATION 50

REFERENCES 53

APPENDIX A 73

APPENDIX B 75

APPENDIX C 77

APPENDIX D 88

APPENDIX E 91

APPENDIX F 104

APPENDIX G 107

APPENDIX H 109

APPENDIX I 130

APPENDIX J 133

APPENDIX K 135

xii

LIST OF TABLES

NO TITLE PAGE

xiii

LIST OF FIGURES

NUMBER FIGURE PAGE

2.1 Photo of Hans Berger 5

2.2 Functional Magnetic resonance Imaging (fMRI) 6

2.3 Basic functional brain map 8

2.4 Sample EEG cap technology 9

2.5 Standardized placement of electrodes on scalp 10

2.6 Delta waves 11

2.7 Theta waves 11

2.8 Alpha waves 12

2.9 Beta waves 12

2.10 Gamma waves 12

2.11 ORGE3D frame samples 20

2.12 ORGE3D frame details 21

2.13 ORGE3D main menu 22

2.14 ORGE3D frame examples 23

2.15 Example of EEG signal display platform 24

2.16 Olivier Bertrand photos 25

2.17 OpenViBE system based grid of letters 26

3.1 Flow Chart of the Project Implementation 28

3.2 Block Diagram of EEG System 29

xiv

3.4 TMSi Porti32 33

3.5 OpenViBE logo 34

3.6 ORGE3D logo 34

4.1 Subject 1 EEG data acquisition 37

4.2 Subject 2 EEG data acquisition 38

4.3 Subject 1 Filtered EEG data of all channels (4-40Hz) 39 4.4 Subject 1 Filtered EEG data of selected channels (4-40Hz) 39 4.5 Subject 1 Raw EEG data of selected channels 39

4.6 Beta Power Spectrum for Subject 1 40

4.7 Subject 2 Filtered EEG data of all channels (4-40Hz) 41 4.8 Subject 2 Filtered EEG data of selected channels (4-40Hz) 41 4.9 Subject 2 Raw EEG data of selected channels 41

4.10 Beta Power Spectrum for Subject 2 42

4.11 Image displayed for relaxation phase 44

4.12 Game Development Diagram 45

4.13 Subject 1 controlling game features 47

xv

LIST OF ATTACHMENTS

NUMBER TITLE PAGE

A Project Proposal 73

B Project Milestone 75

C OpenViBE Documentation 77

D OpenViBE Designer Algorithm 88

E ORGE3D SDK Coding 91

F 3D developed Game 104

G 3D Game User Manual 107

H EEG Data for 10 Subjects 109

I VRPN Coding 130

J TMSi Porti-32 Specification 133

K INOTEK Poster 135

CHAPTER 1

INTRODUCTION

Generally, mind games is undertaken for educational aims and sometimes for medical applications besides entertainment. Such game could be a very useful tool for e-learning and discoveries in the medical field. An electroencephalogram (EEG) is the detection test of brain electrical impulses activities using electrodes attached to the scalp. Recently, EEG-based technology is used in neurofeedback games and brain computer interface (BCI). Availability of advanced BCI makes possible for the EEG system to spread over the areas such as digital media, learning, healthcare and other areas of life.

2

patients by mapping brain signals to human sensory motor functions. [1] Besides, EEG based game technology development are gaining concentration towards ordinary healthy people, especially gamers. Gamers has become the target group of EEG technology, giving them new experiences and incentive in gameplay. There is a shift of focus not just constrained to treatment for patients, but also entertainment. With this shift, patients could be beneficial as more and more powerful gaming industry get involved, the newly available driver for EEG technology improvements can be designed with better game engines and faster processors.

A phenomenon known as Event-Related Desynchronization (ERD) was utilized in order to use EEG as a control signal. In a relaxed state, the subject’s sensorimotor EEG signals will show a low peak in amplitude range. However, if the subject imagines activity, movement, the sensorimotor EEG signal amplitude will show an increase trend. Subjects can learn to control ERD, which allow the EEG signals to be used as a control signal.

3

1.1 Problem statement

There is a lack of EEG based game application for healthy and disabled people. People with severe disabilities, especially to those who suffer from devastating neuromuscular injuries or paralyze which strips their muscular activity cannot perform everyday routine. Individuals who unable to speak and/or use their limbs cannot communicate or manipulating objects. This designed mind game seeks to provide the general medical community discoveries and practical useful clinical application besides entertainment. However, the constraint of the stability of neuronal recordings is difficult to maintain and easy to be interfered by loss making the design difficult to classify the brain waves using neural network. Besides, there is uncertainty about the EEG signal classification in a real-time application. A computational algorithms are needed to reconstruct the activity of large neuronal populations under a range of conditions to allow stable and accurate performance for the BCI interface platform (GUI) to communicate with brain signal.

1.2 Objectives

4

1.3 Scope of work

CHAPTER 2

LITERATURE REVIEW

2.1 Theories of Electroencephalogram (EEG)

Figure 2.1: Photo of Hans Berger (1873-1941) [1]

6

Figure 2.2: Functional Magnetic resonance Imaging (fMRI) [3]

7

Table 2.1: Summary of neuron recording used in Non-invasive approach

8

Figure 2.3: Basic functional brain map [4]

9

Figure 2.4: Sample EEG cap technology [2]