Rabiul Islam at Department of Electrical and Electronics Engineering, Khulna University of Engineering and Technology, Khulna, Bangladesh. This is to prove that the thesis entitled "Formation of relevant virtual images based on channel correlation from EEG sub-band data to recognize human emotions using convolutional neural network" by Md. I certify that this thesis fulfills part of the requirements for awarding the degree.
But the improvement of various human-interacted media mainly depends on the proper recognition of the human feeling which means what he or she wants at that very moment. To communicate the computer with the human, to find out the present feelings of the patient, to understand the customer's interest in purchasing, to understand the interest of people with physical disabilities or to detect the lie of a convict; emotion recognition is a basic prerequisite. We observed that the Convolutional Neural Network (CNN) based method showed superior performance for emotion classification.
Rabiul Islam entitled "Formation of meaningful virtual images based on channel correlation from EEG sub-band data to recognize human emotions using convolutional neural network" has been approved by the board of examiners for partial fulfillment of the requirements for the degree of Master of .
- Introduction
- Emotion Recognition from EEG Signal
- Definition of Emotion
- Classification
- Electroencephalogram (EEG) Signal
- Objectives
- Research Challenges and Contribution
- Organization of the report
The objectives, motivation of the work, importance and research challenges are illustrated in this chapter. The brain is a surprising organ of the human body, created by the almighty and which can control everything. The name of the electrodes comes from the name of the lobe to which the electrode is to be connected.
This type of signal can be found from the frontal lobe and distributed on two sides of the scalp. The feature-based approaches to emotion recognition consist of the features in time domain, frequency domain, joint time-frequency domain, etc. The history of the initial research on emotions and the application and importance of emotion recognition are described in detail in the second part. chapter.
Furthermore, the result of emotion recognition from the previous research and our research is compared in this chapter.
- History of research of Emotion
- Importance’s of Emotion Recognition
- Application of Emotion Recognition from EEG
- Why EEG is the Best?
- Existing Methods of Emotion Recognition
- Feature-Based Methods
- Deep Architecture-based Methods
- Relevant Research
- Modeling of Emotion
- Software for EEG Signal Analysis
Since the understanding and technology of emotion continues to improve, there are more and more branches of using automatic emotion recognition systems. Now, in the era of technological development, the research on emotion recognition through EEG has become very attractive and fundamental research due to its non-invasive nature. Since human behavior is largely dependent on emotion and the physiologist wants to analyze the state of the human mind; Recognition of emotions has become a hot topic in the field of physiology.
In addition, this type of emotion recognition method can easily be used to determine the driver's drowsiness while driving and the student's alertness during instruction. But not only for this indicated field, but also many more fields of artificial intelligence have significant importance for emotion recognition from EEG. Based on the facial expression, only this type of emotion can be classified, the effect of which will change the structure of the human face.
For that reason, this type of emotion recognition method is not applicable to mutism persons [21]. People who cannot speak or are unable to express their emotional statement through gestures and posture are not from this type of emotion recognition method. In the field of emotion recognition, the spectrum analysis is also a popular analysis that uses Fourier transform [30].
In recent years, there has been a trend of emotion recognition research by selecting the right features and shallow machine learning algorithm such as Support Vector Machine (SVM), k Nearest Neighbor (kNN), Decision Tree or Multi-Layer Percept (MLP). 52] used an unsupervised learning-type method, namely self-organization mapping, and investigated emotion recognition performance by collecting 8-channel EEG data from 26 right-handed subjects. According to this opinion, there are a total of three types of emotions i) positive emotion, ii) negative emotion and iii) neutral emotion.
The circumplex model of emotion indicates that each emotion is located distinctly in a circular two-dimensional space. The 3D model of emotions not only takes into account the level of valence and arousal, but also considers the values of dominance.
Introduction
Brief Overview of the Proposed Method
Dataset Description
- EEG Data Acquisition System
- Available dataset for Emotion Recognition
- Details Information of DEAP Dataset
Pre-processing
Reshaping
Decomposition
Formation of Virtual Images
- Data Segmentation
- Pearson’s Correlation Coefficients (PCC)
- No. of Participant, Video, Segmentation and Virtual Images
- Virtual Images for Emotion Recognition
Experiment Protocol
No control or automation from the outside of the body was related to the human brain. From the subbands, only three relevant subbands are taken for the formation of the significant virtual image. Stimuli are the exciter in the human brain, according to which the variation of the EEG signal will occur.
When research will be conducted on the same raw data, the performance or accuracy of the research can be compared. The electrodes are placed according to the rule of the international 10/20 system, which is shown in fig. It indicates for each video and each channel or electrode the number of the data point is 8064.
The name and content of the files are given in the following table for easy representation in Table 3.4. Both parts Names/YouTube links of the music videos used in the online self-assessment and the experiment + individual rating statistics from the online self-assessment. Without this downsampling, the size of the EEG data became extremely large, which was not really informative.
The final format of the transformed data in the previous section was channel×data×video is equal to 32×7680×40. So, for one participant, the number of the entire segmented partition will be 20 by 32 by 40, i.e. In our work, when we considered the data of 32 EEG channels, the PCC matrix size will be 32 × 32 for any single segmentation.
For example, the alpha subband PCC matrix of Participant 1 for Video 1 and for Segmentation 1 is shown in Fig. It can be observed that the correlation between two identical sets of data is 1, the diagonal data of the matrix thus marked. 3.19, the number of videos and segmentation remain fixed (video number 20, segmentation number 7) while the participant version is displayed.
As a result, the more the virtual image changes with the video number change, the more accuracy can be found.
- Introduction
- Convolutional Neural Network
- Working of CNN Model
- Our Proposed CNN Model
- Convolutional Layer
- Padding
- Stride
- Activation Functions
- ReLU Activation Function
- The ‘sigmoid’ Activation Function
- The ‘softmax’ Activation Function
- Pooling Layer
- Optimization Algorithm
- Loss Function
- Summary of CNN model
The basic structure of CNN is very similar to the neuron connection model of the human brain. One of the main advantages of CNN is that it requires little or sometimes no preprocessing like other traditional classification algorithms. The direction of the movement of the core throughout the image is shown in Fig.
The red, green and blue are the 1st, 2nd and 3rd pages of the image volume. Page | 57 As in the previous procedure, the third value of the output volume can be calculated. This 3x3 output image then passes through each layer of the activation function (say ReLU) and finally passes through a pooling layer (say maximum pooling).
Later, the final output of the first set of convolutions, ReLU and pooling layer is taken as the input of the second set of layers. For each single convolution layer if the size of the input image and the filter is expressed as (4.1) and (4.2), respectively; the output dimension after completing a convolutional neural network will be nH×nW×nC. Padding is an additional layer of zeros that provides the pixel significance of the image data corner.
We used 'ReLU', 'sigmoid' and 'softmax' activation function in the different stages of the emotion recognition algorithm. Each neural network learns from the training data and classifies the output according to the desire of the program. The optimization algorithm always updates the values of weights and biases to reduce the value of the loss.
It can be observed that the two architectures are almost identical except for the output shape of the last dense layer. Since we classified 2 and 3 different emotions in protocol 1 and 2, respectively, the shape of the last dense layer is 2 and 3, respectively.
- Introduction
- Result of Classification
- Protocol-1
- Protocol-2
- Comparison of Results
- Limitations
In our experiment, we used the 'DEAP' dataset of EEG signal to classify positive, negative emotions and positive negative and neutral emotions. Both protocols consist of two different tasks (i) valence recognition task and (ii) arousal recognition task. Each participant is shown 40 different emotional audiovisual stimuli and records data on valence, arousal, dominance, sympathy and familiarity.
From the 'DEAP' EEG dataset, the valence rating is first labeled into two categories. The 'DEAP' EEG data is preprocessed and relabeled according to the three-category separation rule written in the previous section. In this section, the arousal data for emotional EEG data are categorized into three classes.
Indeed, the overall accuracy of our proposed model is shown in the following table. From the above table 5.5, it is clear that the maximum accuracy of 76.82% is found in the two-class arousal classification task. Another important fact is that the percentage accuracy of protocol-1 is relatively higher than the accuracy of protocol-2.
Due to the wide range of variations in a number of channels, no emotions, extracted features, a method applied for classification, comparison of accuracy in this type of search is not really important. However, in this work, our contribution is to propose a CNN-based method to recognize emotions from multichannel EEG data, which reduces the human difficulty to manually extract the important feature.
Conclusion
Future Work
If you want to enjoy the benefits of emotion recognition in the practical field, a fundamental prerequisite is greater accuracy and compatibility of real-time operation. Ahmad, “Emotion Classification from EEG Signal Based on Wavelet Analysis,” 2nd International Conference on Electrical, Computer and Communication Engineering (ECCE 2019), Cox's Bazar, Bangladesh, February 2019. Exploring Critical Frequency Bands and Channels for EEG-Based Emotion Recognition from a deep neural network.
Emotion recognition system based on autoregressive model and sequential forward feature selection of electroencephalogram signals. DREAMER: A database for emotion recognition through EEG and ECG signals from low-cost off-the-shelf wireless devices. EEG-based emotion recognition using deep learning network with principal component-based conjoint shift adaptation.
