Solid red line indicates AD for LKF, green is that for OLE, and blue is that for PVA. The networks' left input (L input) specifies the naive firing rates, and the network's right input (R input) specifies the kinematic variables: x and y velocity and velocity. Individual traces of the actual neural representations over time in each trial of the test data (gray lines) are overlaid with the corresponding estimates of the linear rate tuning model (red lines).
GENERAL INTRODUCTION
- I NTRODUCTION
- O UTLINE OF D ISSERTATION
- T HEORETICAL B ACKGROUND OF N EURAL C ODING
- Review of Spike Sorting
- Firing Rate Estimation
- Neural Coding
- Latent Factor Analysis of Neural Population Activity
- E XPERIMENTAL S ETUPS AND D ATA S PECIFICATIONS
- Experimental Setups and Behavioral Tasks for Animals
- Arm Movements and Kinematic Tuning Properties
Finally, considering the relationship between intrinsic neural states of neural population activity and activation patterns of the target muscles, I propose a new approach to extract kinematics-dependent neural representations from neural population activity. It first estimates the temporal firing rates and stores the parameters of the local kernel for spiking events. Another parameter to consider in predicting the neural firing rates is the bin width for the spikes.
EFFECTS OF ENSEMBLE PROPERTIES OF MOTOR CORTICAL NEURONS ON
- D ESIGN OF S IMULATION P ROCEDURE
- B EHAVIOR T ASKS FOR D IRECTIONAL T UNING M ODEL
- S ETTING N EURONAL E NSEMBLE P ROPERTIES
- Initialize Signal to Noise Ratio
- Initialize Distribution of Preferred Directions
- Initialize Non-stationarity of Preferred Directions in Neuronal Ensemble
- P OISSON S PIKE G ENERATOR
- T ESTED D ECODING M ODELS
- D IRECTIONAL D ECODING E VALUATION
- E XPERIMENTAL R ESULTS
- D ISCUSSION
The mean parameter of the Poisson process was defined by the firing rate estimated from the cosine tuning model. However, this study did not investigate the effect of the number of neurons on decoding performance, as it focused more on other neuronal ensemble properties, such as the number of well-tuned neurons (high SNR neurons). The SNR was defined as a logarithmic ratio of the power of st to that of t.
It has already been shown that BMI performance can be influenced by the uniformity of the PDs within the ensemble [97]. I included this point in our simulation to investigate the effect of the proportion of well-tuned neurons (PWTN) and the uniformity of well-tuned neurons (UWTN) on decoding performance (see Figure 2-3). The matrix of the linear tuning model, Ht, is estimated using the least squares method.
The red square dot indicates a significant interval between the AD of LKF and that of OLE. For example, P on the LKF plot indicates that the AD of PVA is statistically different from that of the LKF (p < 0.01). Furthermore, the stability of PVA against changes in UWTN was greatly influenced by the state of PWTN, which was not the case for LKF and OLE.
I also compared the decoding performance of the models with respect to changes in DP over time.
FEASIBILITY STUDY ON DEEP CANONICAL CORRELATION ANALYSIS FOR
N EURAL REPRESENTATION EXTRACTION VIA SUPERVISED LEARNING METHODS
- Linear canonical correlation analysis
- Deep canonical correlation analysis
CCA is one of the multivariate statistical methods that extracts joint canonical variables from random vectors z and x. An LCCA aims to achieve linear mappings from z and x to canonical variables by maximizing the correlations between canonical variables. Using the eigenvectors corresponding to the largest eigenvalues, I repeatedly computed a pair of canonical variables, {𝑢̂𝑍, 𝑢̂𝑋}, until the number of pairs equals m or 3.
DCCA finds nonlinear mappings of z and x into canonical variables through stacked nonlinear transformation layers, as shown in Figure 3-1 (B) (Andrew et al., 2013). The parameter set 𝛉, which includes W and b, is estimated by maximizing the correlation between the functional outputs as follows:. To find 𝛉Z∗ and 𝛉X∗, a backpropagation algorithm is used to optimize the parameters W and b based on the gradient 𝜌(∙).
The deep neural canonical variables can be calculated as 𝑜̂𝑍 = 𝑓𝑍(𝐳, 𝛉𝐙), and the deep kinematic canonical variables can be calculated as 𝑜̂𝑋 = 𝑓𝑋, 𝐉(𝐉). In that case, I call the deep neural (𝑜̂𝑍) and kinematic (𝑜̂𝑋) canonical variables ZDCV and XDCV, respectively. In addition to 𝛉, I also need to optimize the hyperparameters of DNNs, for which I used the Bayesian optimization method [126].
While other parameters determine the learning and architecture of a general DNN, the trade-off parameter is used to regularize correlations with a quadratic penalty, which is uniquely associated with DCCA.
N EURAL REPRESENTATION EXTRACTION VIA UNSUPERVISED LEARNING METHODS
- Principal component analysis
- Factor analysis
- Linear dynamical system for M1 states
To determine the number of PCs to be included in the set of neural representations, I followed the procedure proposed by Yu et al. Briefly, using the eigenvectors obtained from the training set, I extracted all PCs (i.e., how many neuronal units) for the test set, which were then ranked by the magnitude of the corresponding eigenvalues in a descending order. As a result, the minimum reconstruction error was achieved with the first 5 PCs for both monkey C and M datasets (Figure 3-1 (D)), which constituted the neural representation by PCA, denoted as ZPCA .
Note that the smoothing process was applied again to the final set of PCs before decoding. The filled circle indicates the appropriate dimensionality corresponding to the minimum prediction error for each dimensionality reduction method [8]. Then, in a manner similar to PCA, I determined the number of factors involved in a set of neural representations using the reconstruction error of the test set.
I found the minimum error with 20 factors for both monkey C and M, which was further used as the set of neural representations by FA, denoted ZFA. Through the LDS-based neural state estimation approach proposed by Kao and colleagues, I estimated dynamic neural latent states from the population activity [10]. I set the dimensionality to 20 for both monkey C and M, achieving the minimum reconstruction error.
A vector of these neural states was used as neural representations of LDS, denoted as ZLDS.
N EURAL REPRESENTATION ANALYSIS
To estimate latent factors of FRs, I adopted the FA method proposed by Yu et al. were proposed, which adapted FA for neural data [8], [10]. Observed neuronal population activity can be interpreted as a noisy observation of low-dimensional and dynamic neural states [10] , [89] .
L INEAR AND N ONLINEAR D ECODING M ODELS
- Kalman filter
- Long short-term memory in recurrent neural networks
The hyperparameters of LSTM-RNN were optimized by Bayesian optimizer in the same way as DCCA. In our analysis, I set the gradient decay factor to 0.95 and the squared gradient decay factor to 0.99.
E VALUATION OF DECODING PERFORMANCE
E XPERIMENTAL R ESULTS
When LSTM-RNN was used, it also showed lower decoding error with ZDCV than other neural representations (p <. 0.01). The average error of decoding the hand velocity (A) and reconstructing the hand position (B) from decoded velocity (from six different neural representations (i.e., ZE-FR, ZPCA, ZFA, ZLDS, ZLCV, and ZDCV) (see the text) for the descriptions of neural representations) using decoders (linear model (orange) and LSTM-RNN (purple)). When LKF was used, the post-hoc test showed lower decoding error with ZDCV than other neural representations (p < 0.01) except for ZLDS (bl 1).
When LKF was used, the post-hoc test showed lower error with ZDCV than neural representations (p < 0.01). When LSTM-RNN was used, it showed lower error with ZDCV than other neural representations except for ZLCV. When LKF was used, the post-hoc test showed lower error with ZDCV than other neural representations except for ZLDS (p = 0.06).
The post-hoc test with Bonferroni correction showed lower error using LSTM-RNN than using LKF for all neural representations (p < 0.01). For all decoders, the rate decoding error with ZDCV was smaller than with other neural representations (p < 0.01). For all decoders, the position error with ZDCV was smaller than with other neural representations (p < 0.01).
For all decoders, position error with ZDCV was smaller than all other neural representations ( p <0.05).
D ISCUSSION
When LSTM-RNN was used, it only showed lower decoding error with ZDCV than ZE-FR (p < 0.01), while it showed no difference between ZDCV and other representations (p > 0.05). Furthermore, I evaluated the possible interaction effects of neural representations and decoder types using a two-way Friedman test (Figs. 3-9). For all decoders, the decoding error of velocity with ZDCV was smaller than ZE-FR and ZPCA (p < 0.01).
Furthermore, I applied those canonical variables to decoding models to predict the arm movements of NHPs and compared the effect of neural representations in terms of decoding performance. Decoding arm movement information using DCCA-estimated canonical variables resulted in a better performance in both cases using LCCA-estimated canonical variables or using other neural representations regardless of decoder type. Mean decoding error of hand velocity and position (open squares) from the six different neural representations (i.e., ZE-FR, ZPCA, ZFA, ZLDS, ZLCV, and ZDCV) (see text for descriptions of the neural representations) by used decoders (linear model (red) and LSTM-RNN (blue)).
Meanwhile, a training error that directly reflects the learning quality of the decoding model was found to be superior to the other neural representations, along with r2. Besides better features of the canonical variables, there could be another reason why DCCA improved decoding using LKF, while the other neural representations did not. This could be a reasonable reason why DCCA yielded better model decoding performance than the neural representations via unsupervised learning methods.
Apparently, this analysis proved that the direct speed reconstruction through the maps constructed by CCA was poorer than those from the proposed decoding methods to predict speed from neural representations using LKF or LSTM-RNN.
ESTIMATION OF KINEMATIC-DRIVEN NEURAL REPRESENTATIONS FOR
- E XTRACTING K INEMATICS D EPENDENT L ATENT F ACTORS
- E STIMATING K INEMATICS D EPENDENT L ATENT F ACTORS FORM P OPULATION A CTIVITY
- O THER N EURAL R EPRESENTATIONS
- N EURAL T RAJECTORY V ISUALIZATION
- D ECODING AND E VALUATION
- E XPERIMENTAL R ESULTS
- Estimation of Kinematics Dependent Latent Factors
- Neural Dynamics
- Motor Decoding
- D ISCUSSION
This model captures dynamic properties of population activity and achieves significantly higher performance of motor decoding than non-dynamic neural representations. In this chapter, I propose a new kinematics-driven approach to find neural representations of neuronal population activity by estimating kinematics-dependent latent factors. I consider the output of this model (ie, KDLF) as neural representations specific to kinematics of limb movements.
For neural representations of FA, I simply used latent factors derived in the procedure above, that is, h in the previous section. I first compared the trajectories of the neural dynamics of all five neural representations (KLDF, SSC, FA, GPFA, and LFADS) through jPCA. Therefore, if neural representations reflect M1 control of movements consistently on each trial, movement initiation points would be well constrained in the output zero dimension.
I analyzed the dynamics of neural representations generated by each method (KLDF, SSC, FA, GPFA and LFADS). In particular, the neural trajectories of KDLF appeared to show more consistent and clearer rotation patterns than other neural representations. I reconstructed the position trajectory by integrating the predicted velocity and compared it with neural representations.
These characteristics of neural pathways associated with arm movements were most pronounced with KDLF compared to other neural representations.
CONCLUSIONS AND FUTURE WORK
Hochberg et al., “Neuronal ensemble control of prosthetic devices by a tetraplegic human,” Nature, vol. Carmena et al., “Learning to control a brain-machine interface for reaching and grasping by primates,” PLoS Biol., vol. Pandarinath et al., “Latent factors and dynamics in the motor cortex and their application to brain-machine interfaces,” J.
Perich et al., “Motor cortical dynamics are shaped by multiple distinct subspaces during naturalistic behavior,” bioRxiv, p. Perel et al., "Single-unit activity, threshold crossings, and local field potentials in motor cortex differentially encode reach kinematics," J. Butts et al., "Temporal precision in the neural code and the time scales of natural vision."
Kim et al., "A comparison of optimal MIMO linear and nonlinear models for brain-machine interfaces." J. Vaskov et al., “Cortical Decoding of Individual Finger Group Motions Using ReFIT Kalman Filter,” Front. Kim et al., “Divide-and-conquer approach to brain machine interfaces: Nonlinear blending of competing linear models,” Neural Networks , vol.
Rao et al., “Learning mappings in brain-machine interfaces with echo state networks,” in Proceedings.
![Figure 1-2 illustrates this series of firing rate estimation processes [67]. The spike trains for n neurons are represented as a dark raster, where the vertical line for each raster denotes the occurrence of a spike for that time point in a single trial](https://thumb-ap.123doks.com/thumbv2/123dokinfo/10495828.0/25.892.124.770.855.1102/figure-illustrates-estimation-processes-neurons-represented-vertical-occurrence.webp)
