Outside the lab, I want to thank H Perry Hatchfield for being the most supportive partner I could ever ask for. The effect of task on the response of primary somatosensory cortex to visual information indicates a role of attention in mediating early cortical touch processing.
Introduction
To create a viable somatosensory BMI, a better understanding of the neural underpinnings of tactile experience, how it is influenced by multisensory factors, and how the ICMS is integrated into the somatosensory processing hierarchy is needed. In particular, we examine how the ICMS combines with visual information to elicit tactile percepts and how visual context influences the perceived timing of these percepts.
Background
The importance of tactile sensations
- Somatosensation in everyday life
- Tactile experiences and the sense of embodiment
Embodiment is the physical intuition of one's body and the feeling that its components belong to the whole (Longo et al., 2008). The embodiment of a fake hand has also been used to induce illusory tactile sensations in patients with SCI (Lenggenhager et al., 2013).
The sensorimotor loop
- Primary somatosensory cortex (S1)
- Motor and premotor cortex
- Posterior parietal cortex (PPC)
- Sensory remapping in the brain
In particular, activity in S1 is much stronger to physical touch than to observed touch (Pihko et al., 2010). Human electrophysiological work has shown that PPC representations include imagined movements, trajectories, and goals (Aflalo et al., 2015).
Brain-Machine Interfaces (BMIs)
- Methodology
- Intra-cortical microstimulation (ICMS)
- Benefits of a BMI with sensory capabilities
- Multisensory integration of ICMS
- Current challenges
Another study showed that JNDs do not increase proportionally with stimulation current amplitude, although only two amplitudes were assessed (Kim et al., 2015b). Increasing ICMS current amplitude or stimulation duration leads to more intense sensations overall (Armenta Salas et al., 2018; Flesher et al., 2016; Hughes et al., 2021a).
Imagined somatosensory percepts in human cortex
- Introduction
- Methods
- Participant
- Task
- Experiment design and data collection
- Statistics and analysis methods
- Results
- Classifying sensations
- Longitudinal representational of sensations
- Discussion
In the final 5-s imagery phase of the task, the fixation circle changed to green and the participant began the somatosensory imagery. We found significant classification accuracy for the cue, delay, and imagery phases of the task in SMG and the imagery phase in S1 (Figure 3.2a; Table 3.1). In SMG, we found significant classification accuracy in signal phase in several frequency bands.
We also found significant classification accuracy only in the delay phase in higher frequency bands. In training on Experiment 1 and testing on Experiment 2, SMG showed significant classification accuracy in. In S1, significant classification accuracy was observed in the image phase only at a low-frequency band (Figure 3.4a).
The role of visual information and the cortical homunculus in S1
Introduction
MEG studies have shown that the topographic mapping of the fingers shifts in S1 based on the relative timing of visual and tactile signals (Schaefer et al. Transcranial magnetic stimulation (TMS) over S1 negatively affects the ability to perceive or discriminate touch when the accompanying visual information includes human arm rather than a neutral object (Bolognini et al., 2011; Fiorio and Haggard, 2005; Rossetti et al., 2012).Recent experiments suggest that although S1 maintains a coarse topographical representation of the body, as shown in the earliest studies of human cortical stimulation, which has been observed several times since then (Cunningham et al., 2013; Ejaz et al., 2015; Kaas et al., 2019; Kolasinski et al., 2016; Penfield and Boldrey, 1937; Sanders et al. ., 2019) , also contains other more complex levels of tactile representation (Arbuckle et al., 2022; Enander and Jörntell, 2019; Thakur et al., 2012).
Studies of the nonhuman primate hand have shown that S1 neural activity contains nonlinear interactions across different digits (Arbuckle et al., 2022; Qi et al., 2016; Thakur et al., 2012), supporting the idea that S1 carries information beyond a linear report of input from tactile receptors. In humans, S1 has recently been shown to represent body parts outside their traditionally defined areas (Muret et al., 2022; Wesselink et al., 2022). To interrogate S1's representations of touch across body locations and multisensory contexts, electrophysiological recordings in a human tetraplegic patient with two microelectrode arrays (Blackrock Microsystems, Salt Lake City, UT) implanted in putative area 1 of the S1 arm region ( Armenta Salas et al., 2018) were collected.
Methods
- Participant and implant details
- Experimental paradigm
- Data collection
- Quantification and statistical analysis
The experimenter performing the touch was positioned at approximately the 1 o'clock position relative to the participant's head so that the participant could clearly see the experimenter and the approach path of the touch stimulus (Figure 4.1c, Supplementary Video Explanations). Gray lines between state icons are "rubber bands" whose thickness is based on the goodness of fit of the scaling. Singular value decomposition (SVD) was used to perform dimensionality reduction on the initial 96 multi-unit channels of the training dataset.
Within a condition, firing rates from all tuned channels were averaged into 50ms bins, and the 95th percentile of the distribution of mean baseline firing rates was calculated. The onset time for the condition was the middle of the first bin in which the firing rate rose above the 95th percentile of the mean baseline. The offset time was calculated as the middle of the first time bin in which the firing rate after onset fell below the 95th percentile of the mean baseline.
Results
- Condition identity decoding
- Representational Similarity Analysis (RSA)
- Location and touch type generalization decoding
- Individual channel tuning analysis
In total, the task included 9 conditions (table 4.1); the average firing rate of a single channel for each condition is plotted as an example (Figure 4.1d). FP and *BL vary in their level of separation from the non-physical touch types (Figure 4.2c, 0-0.5s bin). To examine whether body location information generalizes across touch types, LDA classifiers were trained to differentiate arm/finger states within one touch type and tested on another (Figure 4.3a).
Finally, the position of tuned channels across all arm and finger conditions was plotted on diagrams of microelectrode arrays (Figure 4.4f). The mean tuned response curves of channels tuned to *FP and *BL conditions were examined (Figure 4.5), calculated as deviation from the distribution of baseline activity:. In all conditions (Figure 4.5b), activity peaks sharply immediately after touch onset, followed by a gradual decline in activity back to baseline.
Discussion
- Neural activity is specialized for arm touches and represents finger touches more generally
- Visual information modulates neural activity if accompanied by a physical stimulus
- Conclusion
- Limitations of the study
From previous work with this participant, intra-cortical microstimulation (ICMS) of the S1 arrays studied here is known to elicit cutaneous and proprioceptive sensations mainly in the hand, with a much smaller number of sensations in the fingers (Armenta Salas et al., 2018 ). The neural response to finger touches detailed here adds to a growing body of literature suggesting that although S1 generally retains a coarse representation of the body along the lines of the homunculus, as represented in the earliest studies of human cortical stimulation and observed repeatedly since (Ejaz et al., 2015;. Finally, Muret et al. 2022) used MRI to show that different body locations are represented in S1 in areas outside their primary topographical area, both in area 3b specifically and S1 in general.
From studies examining Brodmann's areas more specifically, we see evidence that area 3b (Kuehn et al., 2018) and areas 1 and 2 (Kuehn et al., 2013) are able to respond to the observed. Thus, it is possible that a salience threshold, modulated by higher-order brain areas, must be exceeded in order for S1 to represent the observed touches (Dionne et al., 2013). Data from only one participant may confound individual differences with population trends, and although unlikely, it is possible that the participant's spinal cord injury caused an S1 re-entrainment (Kikkert et al., 2021; Makin and Bensmaia , 2017).
Supplemental video legends
The integration of ICMS and visual context
Introduction
Studies have shown that artificial tactile sensations have slower reaction times compared to real tactile input or visual stimuli (Caldwell et al., 2019; Christie et al., 2022; Godlove et al., 2014). Given that visual and tactile stimuli are often combined in the real world, the characteristics of the temporal binding window, or the time period during which two stimuli are perceived as simultaneous events, need to be mapped with respect to ICMS and visual stimuli. . It has been shown that the optimal timing required to perceive peripheral nerve stimulation and visual stimuli as simultaneous is not always the same.
This time appears to differ based on whether stimulation occurs in the upper or lower limb, with stimulation in the leg occurring earlier than in the hand (Christie et al., 2019b). Some work in lower limb amputees has shown that visual information can bias the localization of sensations elicited by peripheral nerve stimulation (Christie et al., 2019a). Visual stimuli are presented at different time offsets relative to ICMS to better characterize the window of temporal association between ICMS and vision, and S1 recordings are examined during catch trials to examine the effects of ICMS-related visual information on processing early tangible.
Methods
- Participant
- Experimental paradigm
- Data collection
- Quantification and statistical analysis
In the abstract condition, the dot moved downward to contact the end of the line. In the baseline condition (18 total trials per session), which always occurred first, the participant viewed rasters of the upper body and hand on a gray background that remained static throughout the task. In the realistic condition, the visual cue was the robotic arm performing a single tap on the participant's virtual arm, and in the abstract condition it was the dot moving along the line to tap the bottom (Figure 5.1c, Supplementary Video 5.1, Supplementary Video 5.2).
At each bootstrap iteration, these binomial distributions were sampled using the MATLAB binornd function and Gaussian curves were fitted to the resulting synthetic data. A more complex model risks overfitting the data and would require a larger number of time samples to better characterize the shape of the temporal binding window. At each time interval, normalized firing rates were compared to baseline firing rates, which were calculated as the mean firing rates in the ITI, 1750 ms to 75 ms before the onset of the “stimulation” phase (Figure 5.1b).
Results
- ICMS-elicited tactile sensations
- The temporal binding window between vision and ICMS
- S1 neural responses to visual stimuli
Performance was also not affected by the current amplitude (Supplementary Figure 5.1, p=0.72), but was affected by the relative deviations of the visual and ICMS (p=0.005). Across the 9 experimental sessions, the word "touch" was used to describe the sensation in realistic trials a greater proportion of the time than in the baseline (Figure 5.2d, Wilcoxon rank sum test, p=0.016), but the difference between abstract and baseline uses of "touch" were insignificant (p=0.13). The tuning of channels relative to baseline while visual stimuli were presented was assessed using a linear regression analysis (Figure 5.4a, b).
The overlap in channels tuned to realistic and abstract conditions during the period from 0 to 0.25 s was quantified (Figure 5.4c). Multidimensional scaling (MDS) was used to visualize the calculated distances between fitness phases (Figure 5.4e) (Nili et al., 2014). Qualitatively, this is clearly visible in the MDS, where icons are grouped by phase, but the conditions are mixed together (Figure 5.4e).
Discussion
- ICMS-elicited sensations are affected by visual information and ICMS current amplitude
- The biological relevance of visual stimuli influences the temporal binding window
- S1 represents ICMS-relevant visual content in a context-independent fashion
- Conclusion
- Limitations of the study
When ICMS is used on the same electrode, with the same parameters, in the same participant, widely divergent qualia often emerge (Figure 5.2d, baseline) (Armenta Salas et al., 2018; Flesher et al., 2016). A viable neural prosthesis would ideally be able to elicit naturalistic sensations of specific qualia when necessary (Tabot et al., 2015). ECoG stimulation has also been shown to be slower than tactile stimuli (Caldwell et al., 2019).
The longer reaction times may be due to the processing time required to understand this irregular activation pattern (Godlove et al., 2014). Previous work with the same participant tested in this study found that S1 did not respond to visually displayed touch without the presence of a physical tactile stimulus (Rosenthal et al., 2023). Visual information unrelated to the tactile stimulation also does not modulate the somatosensory cortex (Espenhahn et al., 2020).
Supplemental figure
Supplemental video legends
Conclusion
Hand posture, but not view of the hand, influences tactile spatial resolution in the grating orientation discrimination task. Rostrocaudal gradients in the complexity of the neuronal receptive field in the finger region of the postcentral gyrus of the alert monkey. Judging roughness by vision - A 7 tesla fMRI study of the responsiveness of the primary somatosensory cortex during perceived touch of self and others.
Visual perception of body interactions in the primary somatosensory cortex: Observing touch in the primary somatosensory cortex. Neural mechanisms of visual guidance of hand action in monkey parietal cortex. Bilateral receptive field neurons in the hind limb region of the postcentral somatosensory cortex in awake macaque monkeys.
