View of IDENTIFYING EMOTIONAL RESPONSE OF VIDEOS ON BRAINWAVES USING EEG: A REVIEW

(1)

Vol.03, Issue 09, Conference (IC-RASEM) Special Issue 01, September 2018 Available Online: www.ajeee.co.in/index.php/AJEEE

1

IDENTIFYING EMOTIONAL RESPONSE OF VIDEOS ON BRAINWAVES USING EEG: A REVIEW

Dharmender¹, ²Ramavtar Jaswal, ³Parminder Saini

1Mtech Student Krukshetra University, Krukshetra

2Assisistant Professior EE, Krukshetra University, Krukshetra

3Assisistant Professior EE, Krukshetra University, Krukshetra [email protected], [email protected],

[email protected]

Abstract- Emotions are psycho-physiological phenomena associated with a wide variety of expressed subjective feelings, observable behaviors and changes in autonomic body state.

There is presently no universally accepted model to categorize emotions. Given the complexity of emotions it is not surprising that there is no universal method to measure them. Different emotions differ in their elicitors, appraisals, physiology and behavioral responses. Selection of the measurement method depends on factors such as the targeted affective theory, the emotional aspects of interest, the context, and the final goal of the evaluation. There have been presented some suggestive literatures on the frequency and phase synchronization of the alpha and theta waves related to Emotion Identification. It is also crucial to evaluate the effect of the mental work on the fundamental EEG rhythms.

Key Words:EEG, Brain Waves, Alpha , Beta , Gamma.

1. INTRODUCTION

A BCI is a communication system that allows a person to convey her/his intention to the external world by merely thinking without depending on the brain's normal output channels of nerves and muscles [2].

A BCI can be functionally described as illustrated in Fig. 1.

Monitoring of the user's brain activity results in brain signals (e.g. electric or hemodynamic brain activity indicators), which are processed to obtain features that can be grouped into a feature vector n modulate her/his brain activity to accomplish her/his intents.

Figure 1: Functional model of a BCI.

The latter is translated into a command to execute an action on the BCI application (e.g. wheelchair, cursor on the screen, spelling device). The result of such an action can be perceived by the user who ca. Brain activity produces various phenomena which can be measured using

appropriate sensing technology. Of particular relevance for BCIs are electrical potential and hemodynamic measurements. Electrical potential measurements include action and eld potentials which can be recorded through invasive (e.g. electrocorticography) and non-invasive techniques (e.g. electro encephalogram and magneto encephalogram). Hemodynamic measurements include functional magnetic resonance imaging (fMRI), positron emission tomography (PET), and functional near infrared brain monitoring (fNIRS). Because of its high time resolution, noninvasiveness, ease of acquisition, and cost effectiveness, the electroencephalogram (EEG) is the preferred brain monitoring method in current BCIs.

To gain control of the BCI, the user executes mental activities which appear as distinctive patterns in the EEG.

These are automatically recognized by the BCI and associated with certain actions that depend on the applications.

Examples of application include: spellers, wheelchair control, and cursor movement.

The type of mental activities and their corresponding EEG correlates are termed as electrophysiological sources of control [3]. The main sources are listed below:

 Sensorimotor activity. Mu and beta rhythms (8-12 Hz & 13-30 Hz respectively) originate in the primary sensorimotor cortex and are more

(2)

2 prominent when a person is not engaged in processing sensorimotor inputs or in producing motor outputs. A voluntary movement results in a desynchronization in the mu and beta bands which is termed event related desynchronization (ERD), and begins in the contralateral rolandic region about 2 seconds prior to the onset of a movement and becomes bilateral before execution of movement. After the movement, the power in the brain rhythm increases (event related resynchronization, ERS).

Motor imagery elicits similar patterns of activity [4]. The imagination of motor movements, in particular limb movements is used in several BCIs which identify the type of motor imagery (right/left hand/foot movement) using a classification algorithm that takes as features the power in the mu and beta bands at electrodes located over the primary sensorimotor cortex (i.e. electrodes C3, C4, Cz of the EEG ten-twenty international system of electrode placement [5]).

For convenience, we refer hereafter to BCIs relying on motor imagery as ERD/ERS based BCIs.

 P300. Infrequent or particularly signicant auditory, visual, or somatosensory stimuli, when interspersed with frequent or routine stimuli, typically evoke in the EEG over the parietal cortex a positive peak at about 300 milliseconds after the stimulus presentation. This peak is called P300. P300 based BCIs operate by presenting the user with a set of choices (usually in a matrix form) and randomly highlighting all different choices. A P300 appears in the user's EEG when her/his selected choice is highlighted.

Detecting the choice for which a P300 was elicited allows the BCI to know the user's selected choice and execute the corresponding action.

The P300 detection algorithm takes as features the signal samples (after band-pass filtering and subsampling) at parietal electrode sites [6].

 Steady State Visual Evoked Potentials (SSVEPs). When subjects

are presented with repetitive visual stimuli at a rate > 5 Hz, a continuous oscillatory response at the stimulation frequency and/or harmonics is elicited in the visual cortex. This response is termed steady-state visual evoked potential (SSVEP). The SSVEP is more prominent at occipital sites. SSVEP based BCIs operate by presenting the user with a set of repetitive visual stimuli at different frequencies which are associated with actions. To select a desired action, the user needs to focus her/his attention on the corresponding stimulus. The SSVEP corresponding to the focused stimulus is more prominent and can be automatically detected by the BCI. Detection of SSVEPs in current BCIs relies on the application of spatial filters (across electrodes) and temporal filters (e.g. comb filters centered around the stimulation frequencies) [7].

 Slow cortical potentials. SCPs are slow non-movement potential changes voluntarily generated by the subject. They react to changes in cortical polarization of the EEG lasting from 300 ms up to several seconds. Operation of SCP based BCIs is often of binary nature and relies on the subject's ability to voluntarily shift her/his SCP [8].

 Responses to mental tasks. BCI systems based on no movement mental tasks assume that different mental tasks (e.g., solving a multiplication problem, imagining a 3D object, and mental counting) lead to distinct, task-specific distributions of EEG frequency patterns over the scalp.

Most of current noninvasive EEG based BCI implementations use the three rst electrophysiological sources of control.

Thus, in the following we focus on ERD/ERS, P300, and SSVEP based BCIs.Present BCI applications have as primary goal to restore communication for the physically challenged. With the considerable expansion of BCI research during the last decade, BCI technology has been increasingly proposed in applications for a wider range of users.Interest in BCI technology stems from the unique advantage of having

(3)

3 access to the user's ongoing brain activity which enables applications spanning a variety of domains such as entertainment (e.g. brain-activity based gaming [9]), safety (e.g. detecting the level of alertness [10]), security (e.g. brain activity based biometrics [11]), and neuro-economics (e.g. neural correlates of consumer choices for marketing [12]). Applications relying on the use of brain activity as an additional input, allowing the real time adaptation of the application according to the users mental state are categorized as passive BCIs [13].

The rest of research paper is design as follows. The overall previous work is described in Section II. Section III describes problem formulation.

performance parameter describe in section IV. Finally, Section V describes the conclusion of paper.

2. LITERATURE REVIEW

This section will provide the brief description and highlights the contribution, remarks and factors of the work done by the researchers. Many attempts have been made in the past to achieve the maximum accuracy of video signals.

Liu, Ning-Han, Cheng-Yu Chiang 2013[1] Driving safety has become a global topic of discussion with the recent development of the Smart Car concept.

Many of the current car safety monitoring systems are based on image discrimination techniques, such as sensing the vehicle drifting from the main road, or changes in the driver's facial expressions. Through the in-car stereo system, this music recommendation mechanism can help prevent a driver from becoming drowsy due to monotonous road conditions. Experimental results demonstrate the effectiveness of our proposed drowsiness detection method to determine a driver's state of mind, and the music recommendation system is therefore able to reduce drowsiness.

Liu, N.-H 2013[2] With rapid growth in the online music market, music recommendation has become an active research area. In most current

approaches, content-based

recommendation methods play an important role. Estimation of similarity between music content is the key to these approaches. A distance formula is used to calculate the music distance measure,

and music recommendations are provided based on this measure. However, people have their own unique tastes in music.

This paper proposes a method to calculate a personalized distance measure between different pieces of music based on user preferences.

Campisi, P., & La Rocca 2014[3]

investigated within the medical field for more than a century to study brain diseases like epilepsy, spinal cord injuries, Alzheimer's, Parkinson's, schizophrenia, and stroke among others.

They are also used in both brain computer and brain machine interface systems with assistance, rehabilitative, and entertainment applications. In this paper, we further speculate on those issues, which represent an obstacle toward the deployment of biometric systems based on the analysis of brain activity in real life applications and intend to provide a critical and comprehensive review of state-of-the-art methods for electroencephalogram-based automatic user recognition, also reporting neurophysiological evidences related to the performed claims.

Rhiu, Ilsun 2015[4] reviewed both studies on general smart car technologies and human–computer interaction (HCI)/human–vehicle interaction studies that were published in journals and conferences so that the current status of research can be identified and future research directions can be suggested.

Furthermore, previous studies on elderly drivers were reviewed, as these drivers could be the most vulnerable social group in terms of new technology acceptance. A total of 257 articles for HCI research and 45 articles for elderly drivers were selected and reviewed from 11,267 collected articles (2010–2014).

Zheng, Wei-Long, and Bao-Liang 2015[5] investigated critical frequency bands and channels, this paper introduces deep belief networks (DBNs) to constructing EEG-based emotion recognition models for three emotions:

positive, neutral and negative. Four different profiles of 4, 6, 9, and 12 channels are selected. The recognition accuracies of these four profiles are relatively stable with the best accuracy of 86.65%, which is even better than that of the original 62 channels. We compare the performance of deep models with shallow models. The average accuracies of DBN,

(4)

4 SVM, LR, and KNN are 86.08%, 83.99%, 82.70%, and 72.60%, respectively.

Sanyal, S., Banerjee, A., Pratihar, R 2015[6] studied reports the effect of two different types of Hindustani music which are supposed to evoke contrasting emotions on brain activity using Electroencephalography (EEG) data.

Two different sets of Hindustani music raga clips of contrasting emotion (romantic/sorrow) were used in the study.

EEG was performed on five male subjects while they listened to the clips. The scaling exponent (α) values determined for the different experimental conditions show different levels of neural activity when the two different types of music are played. The implications are discussed in detail.

Maity, A. K., Pratihar, R., Mitra 2015[7] Electroencephalography (EEG) was performed on 10 participants using a simple acoustical stimuli i.e. a tanpura drone. The tanpura drone is free from any semantic content and is used with a hypothesis that it provides a specific resting environment for the listeners. The EEG data was extracted for all the frontal electrodes viz. F3, F4, F7, F8, Fp1, Fp2 and Fz. Empirical Mode Decomposition (EMD) was applied on the acquired raw EEG signal to make it free from blink as well as other muscular artifacts. The importance of this study lies in the context of emotion quantification using multifractal spectral width as a parameter as well as in the field of cognitive music therapy. The results are discussed in detail.

De Smedt, T., Menschaert 2016[8] Previous research has shown positive correlations between EEG alpha activity and performing creative tasks. In this study, expert classical musicians (n=4) were asked to play their instrument while being monitored with a wireless EEG headset. Data was collected during two rehearsal types: (a) in their regular, fixed ensemble;; (b) in an improvised, mixed ensemble with unfamiliar musicians and less rehearsal time.

Finally, the real-time capabilities of wireless EEG monitoring were explored with a data visualisation during live performance on stage.

2.1 Problem Formulation

Various methods on the stress measurement previously proposed utilize

changes of physiological quantities such as blood pressure, heart rate and salivary amylase in stressed conditions. These quantities can be measured noninvasively by use of inexpensive instruments.

However, since these quantities can be affected by many physiological causes and noises other than the stresses, it should be noted these methods have possibility of overestimate or underestimate on quantity of stress human receives.

Measurement of brain activities has advantages including direct observation of emotional response to stressors in the limbic system and hypothalamus. Specifically, this type of measurement is expected to take details on the intensity of the stresses she or he receives as well as to identify an underlying stressor.There are some imaging techniques which measure the brain activities; e.g., positron emission tomography (PET), functional magnetic resonance imaging (fMRI), near-infrared spectroscopy (NIRS), magneto

encephalogram (MEG) and

electroencephalogram (EEG). However, the temporal evolution of PET and fMRI is not sufficient to elucidate dynamics of the emotional response in the brain to stressors. Moreover, the measurement environment of PET, fMRI and MEG strictly limits stressors applicable to measurements. Although the size of measurement equipment for NIRS is small and its temporal evolution is relatively high, its cost is expensive, having difficulty in use in many clinical sites.

EEG, a conventional measurement method, detects electrical activities of neuronal populations in the cerebral cortex and may reflect the emotion elicited by the stressors more directly than the above physiological quantities. Although the EEG has neither sufficient spatial resolution, a few centimeters, nor takes information in depth direction, the EEG has high temporal resolution and an EEG measurement system is small enough to be portable. The EEG has also other advantages including in dissemination in clinical sites and open-endedness in experimental environment.

2.2 Sounds and Brainwave Bands

Sounds made by humans can be divided into verbal sound such as speaking and singing and non-verbal sound such as footsteps, breathing and snoring. Even

(5)

5 without visual information, sound recognition abilities by humans and animals assist in detecting behaviorally relevant events [33].

Sound emerges when the vibrations of an object trigger the air surrounding it to vibrate, and this in turn causes the vibration of the human eardrum signaling the brain to interpret it as sound. Technically, a sound wave is any disturbance that is propagated in an elastic medium which may be a liquid, solid or gas and can be perceived by the hearing sense of a human being [34].

Sound is transmitted through a medium as longitudinal waves and transverse waves where the wavelength is expressed as 1. Wavelength = Sound waves are usually visualized as sine waves and they have fixed frequencies and amplitudes, perceived as pure tones. Most sound waves consist of multiple overtones or harmonics and can be represented mathematically, as follows

2. p(t) = p0 Sin (2πf)

Where:-p(t) : Instantaneous, incremental, sound pressure (above and below atmospheric pressure)

po: Maximum amplitude of the instantaneous sound pressure

f : Frequency (the number of cycles per second) (Hz)

t : Time (s)

Sound waves comprise of pure tones (single frequency), a combination of single frequencies which are harmonically related and also a combination of single frequency which are not harmonically related either as finite or infinite in numbers. The frequency of sound can be determined by dividing the velocity of sound by its wavelength. The emission of sound-source into the surrounding air will produce a measurable amount of sound power (W) [34], expressed as (3):

Where:-Ws: total sound power radiated by source (Watts)

Is : maximum sound intensity (W/m2) r: distance (m)

Sound intensity is defined as a continuous flow of power carried by sound waves. A normal person’s ear responds to sound pressure at 105 or more. The sound power level may be mathematically derived, as (4):

3. CONCLUSION

Several action observation/imagery training studies have been conducted in patients with limited physical activity showing improvements in motor function.

However, most studies compared effects of action observation and imagery, so little is known about the changes caused by subsequent observation of target objects.

Moreover, few studies analyzed brain wave changes in the EEG rhythm.

Healthy adults participated in this study, and were divided into age groups and provided: ‘Visual Stimuli’ of movies of various Genres. EEG amplitude in the various frequency bands over the sensorimotor cortex was evaluated.

Significant EEG suppression was obtained in the action observation trials.

Results showed a main effect of visual stimuli, with decreased power during action observation, and increased power post-observation in both conditions.

Comparing the ‘Classical’ and ‘Non- Classical’ Groups during the post- observation period.

REFERENCES

1. Liu, Ning-Han, Cheng-Yu Chiang, and Hsiang-Ming Hsu. "Improving driver alertness through music selection using a mobile EEG to detect brainwaves." Sensors 13.7 (2013)

2. Liu, N.-H. Comparison of content-based music recommendation using different distance estimation methods. Appl. Intell.

2013, 38, 160–174.

3. Campisi, P., & La Rocca, D. (2014). Brain waves for automatic biometric-based user recognition. IEEE transactions on information forensics and security, 9(5), 782-800.

4. Rhiu, Ilsun, et al. "Research Issues in Smart Vehicles and Elderly Drivers: A Literature Review." International Journal of Human- Computer Interaction 31.10 (2015): 635- 666.

5. Zheng, Wei-Long, and Bao-Liang Lu.

"Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks." IEEE Transactions on Autonomous Mental Development 7.3 (2015): 162-175.

6. Sanyal, S., Banerjee, A., Pratihar, R., Maity, A. K., Dey, S., Agrawal, V., ...&Ghosh, D.

(2015, September). Detrended Fluctuation and Power Spectral Analysis of alpha and delta EEG brain rhythms to study music elicited emotion. In Signal Processing, Computing and Control (ISPCC), 2015 International Conference on (pp. 205-210).

IEEE.

7. Maity, A. K., Pratihar, R., Mitra, A., Dey, S., Agrawal, V., Sanyal, S., ...&Ghosh, D.

(2015). MultifractalDetrended Fluctuation Analysis of alpha and theta EEG rhythms

(6)

6

with musical stimuli. Chaos, Solitons&

Fractals, 81, 52-67.

8. De Smedt, T., Menschaert, L., Heremans, P., Lechat, L., &Dhooghe, G. (2016). An EEG study of creativity in expert classical musicians. arXiv preprint arXiv:1612.06719.

9. Thomas, E.M.; Temko, A.; Lightbody, G.;

Marnane, W.P.; Boylan, G.B. A Gaussian Mixture Model Based Statistical Classification System for Neonatal Seizure Detection. Proceedings of IEEE International Workshop on the Machine Learning for Signal Processing, Grenoble, France, 1–4 September 2009; pp. 1–6.

10. Yazdani, A.; Ebrahimi, T.; Hoffmann, U.

Classification of EEG Signals Using Dempster Shafer Theory and a K-Nearest Neighbor Classifier. Proceedings of the 4th International IEEE/EMBS Conference on Neural Engineering, Antalya, Turkey, 29 April–2 May 2009; pp. 327–330.

11. Zhuang, Tianbao, Hong Zhao, and Zheng Tang. "A study of brainwave entrainment based on EEG brain dynamics." Computer and information science 2.2 (2009): 80.

12. Cuingnet, R.; Chupin, M.; Benali, H.; Colliot, O. Spatial and Anatomical Regularization of Svm for Brain Image Analysis. Proceedings of the Neural Information Processing Systems Conference, Vancouver, BC, Canada, 6–9 December 2010; pp. 460–468.

13. Li, M.-A.; Zhang, C.; Yang, J.-F. An Eeg- Based Method for Detecting Drowsy Driving

State. Proceedings of the 2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), Yantai, China, 10–12 August 2010; pp. 2164–2167.

14. Li, Y.; Zeng, X. Sequential multi-criteria feature selection algorithm based on agent genetic algorithm. Appl. Intell. 2010, 33, 117–131. [73]

15. Nanopoulos, A.; Rafailidis, D.; Symeonidis, P.; Manolopoulos, Y. Musicbox: Personalized music recommendation based on cubic analysis of social tags. IEEE Trans. Aud.

Spee. Lang. Proc. 2010, 18, 407–412. [74]

16. Jrad, N.; Congedo, M.; Phlypo, R.; Rousseau, S.; Flamary, R.; Yger, F.; Rakotomamonjy, A.

Sw-SVM: Sensor weighting support vector machines for EEG-based brain–computer interfaces. J. Neur. Eng. 2011.

17. Sulaiman, N.; Taib, M.N.; Lias, S.; Murat, Z.H.; Aris, S.A.M.; Hamid, N.H.A. EEG- Based Stress Features Using Spectral Centroids Technique and K-Nearest Neighbor Classifier. Proceedings of the 13th International Conference on Computer Modelling and Simulation (UKSim), Cambridge, UK, 30 March–1 April 2011; pp.

69–74.

18. Bogdanov, D.; Haro, M.; Fuhrmann, F.;

Xambó, A.; Gómez, E.; Herrera, P. Semantic audio content-based music recommendation and visualization based on user preference examples. Inform. Proc. Manag. 2012, 49, 13–33.