Second, I would also like to thank my labmates as well as my Vietnamese friends both in Ulsan and in Vietnam, who have always been kind to me and helped me a lot in my early days in Korea. I am thankful for all the trips we went on, all the conversations we had, all the birthday events, dinners and regular occasions we spent together. Finally, I would like to thank all the members of the thesis defense committee who helped me revise my thesis.
Content words such as nouns and verbs convey essential information about the overall meaning (semantics) of the sentence, while function words such as prepositions and pronouns have less meaning and support the syntax of the sentence. The findings of this study may provide insight into the differential contributions of some word types versus others to overall sentence comprehension.
INTRODUCTION
- Research basis and objective
- Basic brain anatomy and function
- The language-relevant regions
- Broca’s Area
- Wernicke’s Area
- Electroencephalography
In addition, the thematic role assignment that determines whether a verb's argument is an object or subject of the action provides another critical aspect of sentence comprehension [3, 4]. While listening to spoken sentences, listeners' attention applied to each component may vary as a function of its importance to understanding the meaning of the entire sentence. The cerebral cortex of the human brain is traditionally divided into four lobes: frontal, temporal, parietal and occipital purely based on anatomical aspect.
5 Brodmann's map is a cytoarchitectonic map of the human brain first introduced in 1909 by Korbinian Brodmann, a German neurologist. Regarding the human brain, only 43 cortical areas belonging to 11 regions are shown on Brodmann's map, while the number for monkeys and apes is 30. Broca's area is the brain region located in the frontal lobe of the dominant hemisphere, usually in the left side of the human brain, which is named after Paul Broca, an anatomist and surger.
The area was defined in terms of pars opercularis and pars triangularis as BA (Brodmann area) 44 and 45 areas corresponding to the Brodmann map. Wernicke's area is a region of the brain located in the back of the superior temporal gyrus (STG) of the dominant hemisphere of the brain, which is usually on the left. This area is in close proximity to the auditory cortex, which is part of the auditory system and is responsible for processing auditory information in humans.
In collaboration with Broca's area, Wernicke's area is a part of the cerebral cortex associated with spoken and written language comprehension. EEG, a non-invasive method, measures voltage fluctuations within neurons in the brain using a cap of electrodes placed on the scalp, as shown in Figure 6 below. To improve the SNR, ERPs (event-related potentials) obtained by averaging the EEG are usually used instead of the raw EEG signal.
BASIS THEORY
EEG recording system
- Experiment diagram
- Devices and toolboxes
- Experiment procedure
The experiment was conducted in a soundproof booth, where participants were seated facing a computer monitor and four speakers placed in front at a distance of 1 m. Routines in the Psychtoolbox core enable synchronization, support sub-millisecond timing, and capture low-latency audio important for cognitive processes such as language processing, which typically occur within milliseconds of stimulus onset. In preparation for the experiment, participants were asked to wear 64 Ag-AgCl electrode holder caps, the holders of which were split as a 10/20 layout; then all electrodes were placed in the appropriate holders.
After that, electrode holders were filled with electrode gel with a syringe; electrode impedance was monitored via ActiView software installed in PC1 (Figure 9) to ensure that the impedance was kept in the range of -50 to 50. Ten English sentences were used as auditory stimuli; each sentence has a length of no more than 2.5 seconds. During the experiment, all sentences were played in a random order with an interval of 3 seconds.
Participants were asked to follow a silent movie presented on the monitor to ensure passive listening to the speech sound. During the course of the experiment, participants were also required to restrict movements to minimize movement artifacts. All the movements of subjects and corresponding trial were recorded by manually monitored by camera in the soundproof room.
Stimuli analysis
- Sentence structure
- Sentence structure parser
- Phoneme impulse train
In other words, a language analyzer can analyze a sentence in part of speech, for example which word is a noun or verb and which word is subject or object of a verb. One of them is Natural Language Toolkit, which is a leading platform in Python that provides an API for completing the task of labeling POS for components in English sentence [ 39 ]. Another project that is also used for parsing is the Stanford Parser, a Java package, which can be adapted for many languages such as: English, Chinese, German, etc.
The parser uses Penn Treebank, which produced approximately 7 million words of POS-tagged text. Each phoneme in a spoken English sentence can be separated by its onset time using Praat software, which is used for speech analysis in phonetics [41]. To examine to what degree each POS component contributes to sentence comprehension, the phoneme associated with that component was removed to make the component-excluded phoneme pulse train.
The examined components include substantive words, functional words, nouns, verbs, objects and subjects, which resulted in 6 cases excluded from the component: substantive words-, functional words-, nouns-, verb-, objects- and subjects-excluded case, respectively. Each constituent was compared to the complete sentence with the other in pairs, which are content words versus this. The reason for this is the reciprocal relationship between the two components in each pair with the correct grammar of the sentence.
Importantly, there were no significant differences (Wilcoxon signed rank test, p>.05) in the number of phonemes in the case of each pair (i.e., noun vs. verb; object vs. subject).
Data processing procedure
- EEG recording and processing
- EEG analysis
EEG signals were then analyzed using epoch lengths of 3 s from 0.5 s before stimulus onset to 2.5 s after stimulus onset. The ERP was calculated by averaging all epochs from each sentence at each EEG electrode. ERP baseline was corrected by subtracting the mean amplitude between −200 and 0 ms relative to stimulus onset.
An example of the ERP obtained from the sentence "Maple syrup is made from sap" is shown in Figure 14C. Figure 14A and B show an illustration of phoneme-onset impulse trains of the sentence "Maple syrup is made of sap" for a whole sentence and for a noun exclusion, respectively. There were no significant differences (Wilcoxon signed rank test, p > 0.05) in the number of phonemes in the case of each pair (i.e., noun vs. verb; object vs. subject). Cross-correlation coefficients were calculated between phoneme-onset impulse trains and their corresponding averaged EEG signals as a function of delay that ranged from −200 to 700 ms (where a positive sign indicates that the EEG signal lags behind the phoneme-onset train).
The cross-correlation function then subtracted the mean value of the cross-correlation coefficients and passed through a 1 Hz high-pass filter.
RESULTS
In each panel, the exclusion cases are represented by black or red lines, while the dark gray line shows the entire sentence case. For example, in the case of content word exclusion, the N1 component has been significantly reduced compared to the whole sentence case in brain regions related to language processing (Table 3, Wilcoxon signed-rank test, p <0.05), whereas function word exclusion cases did not reveal such a significant difference (see Figure 15 A, panel F7). 23 Figure 15: Example of cross-correlation functions at the typical electrodes of AF7, F5 and F7 at.
Figure 16B, Figure 17B, and Figure 18B show the locations of electrodes that showed significant differences in P1, N1, and P2 peaks between whole sentences and each exclusion case. The red color indicates that the whole-sentence case elicited a larger peak amplitude (Wilcoxon signed rank test, p<0.05), while the blue color indicates that a component exclusion case elicited a larger P1, N1, or P2 peak amplitude, respectively. Compared to the whole-sentence case, the case of noun exclusion caused a significantly reduced N1 peak response, also at left temporal and central sites (Figure 17, Wilcoxon signed rank test, p<0.05), while the case of verb exclusion showed no such difference at N1 peak.
25 Figure 17: (A) Correlation coefficient function between speech-evoked EEG signals and whole-sentence, noun-exclusion, and verb-exclusion impulses at the typical delay of 285.2 ms. B) Dominance map showing full sentence versus noun and verb exception cases. Whereas the object-exclusion case showed significantly reduced correlation coefficients at broad fronto-central and temporal sites in the N1 peak, the subject-exclusion case now showed such a decrease. The object exclusion case showed significant reduction of N1 activity at the largest number of electrode sites (ie, 31 electrodes out of 64).
In contrast, there was no electrode site that showed a significant N1 difference in the verb exclusion case.
DISCUSSION AND CONCLUSION
ERP complexes, including the N100 (or N1), N400 and P600 are thought to correspond to different stages of language processing. Martin, Speech and language processing: an introduction to natural language processing, computational linguistics and speech recognition. Wang, J., et al., Identifying thematic roles from neural representations measured with functional magnetic resonance imaging. Cogn Neuropsychol, 2016.
Alday, P.M., M/EEG analysis of naturalistic narratives: an overview from speech to language processing. Language Cognition and Neuroscience, 2019. Porbadnigk, A.K., et al., Single-trial analysis of the neural correlates of speech quality perception.J Neural Eng, 2013. Crosse, MJ, et al., The Multivariate Temporal Response Function (mTRF) Toolbox: A MATLAB Toolbox for Re transfer Ne oral signals to continuous stimuli.
Groppe, D.M., et al., The phonemic reinstatement effect reveals an effect of supporting sentence context in pre-N400 speech perception. Brain Res, 2010. Ruschemeyer, S.A., et al., Processing of lexical semantic and syntactic information in first and second languages: fMRI evidence from German and Russian. Hum Brain Mapp, 2005. Kuperberg, G.R., et al., Distinct patterns of neural modulation during the processing of conceptual and syntactic anomalies. J Cogn Neurosci, 2003.
Choi, I., et al., Quantifying attentional modulation of auditory-evoked cortical responses from single-trial electroencephalography. Front Hum Neurosci, 2013.
