Optimization of Performance Metrics of Charge
Trapping Synaptic Device for Neuromorphic Applications
Item Type Conference Paper
Authors Raza Ansari, Md. Hasan;Elatab, Nazek
Citation Raza Ansari, Md. H., & El-Atab, N. (2023). Optimization of Performance Metrics of Charge Trapping Synaptic Device for Neuromorphic Applications. 2023 7th IEEE Electron Devices Technology & Manufacturing Conference (EDTM). https://
doi.org/10.1109/edtm55494.2023.10103084 Eprint version Post-print
DOI
10.1109/edtm55494.2023.10103084Publisher IEEE
Rights This is an accepted manuscript version of a paper before final publisher editing and formatting. Archived with thanks to IEEE.
Download date 2024-01-13 17:33:21
Link to Item
http://hdl.handle.net/10754/6919512023 Electron Devices Technology and Manufacturing Conference (EDTM)
Optimization of Performance Metrics of Charge Trapping Synaptic Device for Neuromorphic Applications
Md. Hasan Raza Ansari and Nazek Al-Etab
SAMA Labs, Electrical and Computer Engineering, King Abdullah University of Science and Technology (KAUST), Thuwal 23955-6900, Saudi Arabia, email: [email protected]
Abstract
This work validates the synaptic behaviors (long-term potentiation (LTP) and depression (LTD)) of a junctionless transistor (JL) through the simulator. The synaptic transistor is an essential component for implementing artificial neural networks (ANN), which are called hardware neural networks (HNNs). This analysis shows optimization of nonlinearity and dynamic range of conductance values of LTP and LTD and is used for implementing the ANN with the MNIST dataset.
The device achieves linear conductance (0.1) value and a higher dynamic range (~105) by optimizing the gate voltage. These results indicate that the JL device achieves 88.1 % image recognition accuracy.
Keywords: Junctionless Transistor, Synaptic Transistor, STP, LTP, LTD, ANN, Neural Network
Introduction
Recent progress in artificial intelligence (AI) requires innovation in artificial neural networks (ANNs). An ANN is a mathematical algorithm that can be trained to mimic human behaviors, such as speech and image recognition [1]–[5]. This ANN can be constructed through artificial neurons and synapses, known as hardware neural networks (HNNs) [1]–[5]. Recent studies show that different artificial synaptic devices can mimic human behaviors such as long-term potentiation (LTP) and depression (LTD) with maximum pulse states, better linearity, and high dynamic range (on-off ratio) [1], [4], [6]. These are the essential parameters and are crucial for hardware neural networks. Although two terminal devices are showing great interest in future artificial neural networks, but these devices are facing the issue of optimizing these parameters. Also, these devices are facing the problem of sneak path and variability. To overcome these issues, we have proposed a double gate junctionless charge-trapping memory for an artificial synapse in this work. A junctionless device can be operated at a lower drain voltage, which consumes low power and energy, and mitigates the trade-off of synaptic parameters.
Results and Discussion
Fig. 1(a) shows the simulated n-type double gate junctionless transistor through the Silvaco Atlas simulator with a gate length (Lg) of 100 nm and metal gate workfunction for the top and back gate is 5.25 eV (p-poly) [7]. The silicon film thickness (TSi) is 10 nm, and the oxide/nitride/oxide (O/N/O) thickness is 2/2/2 nm with a back gate oxide thickness of 2 nm to deplete the carriers from the silicon and make a virtual p-region to store the hole in the silicon body [7]. The front gate with
thicker gate oxide is used for charge trapping and the back gate with thinner is used for modulating the conductivity of the channel. A junctionless transistor with the same doping type throughout the semiconductor helps to trigger the impact ionization and band-to-band tunneling at a lower drain bias than an inversion mode transistor [8], [9].
Operations of the JL device as synapse are based on the generation and recombination in the silicon body and trapping and de-trapping of carriers in the nitride layer. Impact ionization (II) and non-local band-to-band tunneling (BTBT) models are incorporated to capture charge generation. We have also used the macro DYNASONOS model to capture the trapping and de-trapping phenomenon in the nitride layer. The operation of double gate charge trapping memory as synapse is well explained in our previous papers [10], [11]. Fig. 1(b) shows the variation of conduction values during LTP and LTD with pulse number (Total pulse number is 32, which consists of 16 pulses for LTP and 16 for LTD). Then, these values are estimated from the JL device and utilized for HNN. Fig. 1(c) demonstrates the artificial neural network for image classification with one hidden layer.
0 8 16 24 32
0 50 100 150
Conductance [mS/mm]
Pulse number
n
+n
++n
++Tunneling Oxide (SiO2) Nitride Layer (Si3N4) Blocking Oxide (SiO2)
Oxide (SiO2) Front Gate
Back Gate TSi
Ttox
Tnox
Tbox
Tox
S D
Human Brain
(a) (b) (c)
Neural Networks
Fig. 1 (a) Schematic representation of double gate junctionless transistor for an artificial synaptic transistor. (b) Variation of conduction values during LTP and LTD with pulse number. (c) Schematic representation of two layers of neural networks for image classification.
Fig. 2(a) shows the flowchart of this work to calculate the conductance values of the device during LTP and LTD, and then it can be utilized for hardware neural networks for image classification.
Potentiation operation is performed by applying repetitive pulse with front gate voltage (VGS1_P) = -1.0 V, drain voltage (VDS_P) = 0.8 V, and back gate voltage (VGS2_P) = -0.6, -0.5, and -0.4 V. The depression operation is performed VGS1_D of 4 V and VDS_D of 0.8 V. The potentiation and depression pulse times are 500 ns and 100 ns, respectively, with an interval time of 500 ns. Figs. 2(a) and 2(b) show the trapped charges in the nitride layer during potentiation and depression, respectively, with different VGS2_P. It is evident from Fig. 2(a) that increase in VGS2_P (in negative amplitude) increases the tunneling near the back channel and drain junction, which generates more electron-hole pairs
2023 Electron Devices Technology and Manufacturing Conference (EDTM)
in the silicon body and modulate the starting pulse of LTP. VGS2_P not only affects the starting pulse of LTP, it also affects the de-trapping of charges from the nitride layer during depression operation. VGS2_P
= -0.6 V traps more charges in the nitride layer than VGS2_P = -0.4 V and -0.5 V. Therefore, it de-traps fewer charges from the nitride layer with the same depression voltage. The inference operation of synaptic operation is performed at a lower drain voltage with different back gate voltage to optimize the synaptic performance metrics.
Start Device Selection
Performance Metrics Evaluation
Yes
No
Bias/Device Optimization
Balancing Trade- offs: NL and DR
End Conductance Values
for NN Yes
No Synapse Behaviors
Optimized device for synapse
(a)
0 5 10 15 20
0 1 2 3 4 5 6
Trapped Charges [pC]
Pulse number
VGS2_P = -0.6 V VGS2_P = -0.5 V VGS2_P = -0.4 V
LTP @ 3rd LTP @ 6th LTP @ 13th
(b)
Potentiation
0 5 10 15 20
0 2 4 6 8 10 12 14 16
Trapped Charges [pC]
Pulse number
VGS2_P = -0.6 V VGS2_P = -0.5 V VGS2_P = -0.4 V
(c)
Depression
Fig. 2. (a) Flowchart for assessing synapse behavior for the neural networks. Variation in trapped charges in the nitride layer during synapse operations of (b) potentiation and (c) depression with different VGS2_P.
10-9 10-6 10-3 100 103 10-11
10-10 10-9 10-8 10-7 10-6 10-5 10-4
Drain Current [A/mm]
Transient Time [s]
Inference :
LTP
Potentiation
(a)
Initial VDS_I = 0.1 V, VGS2_I= -0.5 V
STP
5th
6th 70th
10-9 10-6 10-3 100 103 10-8
10-7 10-6 10-5 10-4
Drain Current [A/mm]
Transient Time [s]
1stto 64thpulse Depression
Inference :
VDS_I = 0.1 V, VGS2_I= -0.5 V
1st
64th
(b)
10-9 10-6 10-3 100 103 10-11
10-10 10-9 10-8 10-7 10-6 10-5 10-4
Drain Current [A/mm]
Transient Time [s]
~ 4.5 103 DR Potentiation
VDS_I = 0.1 V, VGS2_I= -0.3 V Initial 5th
6th 70th
Inference :
(c) 10-9 10-6 10-3 100 103
10-8 10-7 10-6 10-5 10-4
Drain Current [A/mm]
Transient Time [s]
~ 3.8 10DR 1
Depression 1st
64th
VDS_I = 0.1 V, VGS2_I= -0.3 V Inference :
(d)
Fig. 3. Variation in transient drain current during inference operation at VDS_I = 0.1 V for each pulse of (a) potentiation and (b) depression when VGS2_I = -0.5 V, and each pulse of (c) potentiation and (d) depression when VGS2_I = -0.3 V. DR indicates the dynamic range (Gmax/Gmin).
Figs. 3(a) and 3(b) show the synaptic behavior in terms of the retention time of each pulse of STP, LTP, and LTD with inference voltage of VGS2_I = -0.5 V, VGS1_I = 0.0 V, and VDS_I = 0.1 V. Figs. 3(c) and 3(d) show the synaptic behavior of STP and LTP
inference voltage of VGS2_I = -0.3 V, VGS1_I = 0.0 V, and VDS_I = 0.1 V. It can be observed from Figs. 3(a) and (c) that when the device is in STP state (red color), the drain current of each pulse decays and reaches to initial state (Green color) of the device.
As shown in Fig. 2(a) for VGS_P = -0.5 V, charge trapping started from the 6th pulse. Thus, the 6th pulse confirms that the drain current is higher than the STP state and conveys that the device is in LTP.
During LTP states (blue color), the current increases at each pulse due to trapped charges in the nitride layer and is retained for a longer duration (> 103 s).
Figs. 3(b) and (d) show that the drain currents decrease with an increase in pulse number for depression operation due to de-trapping of charges from the nitride layer. Conductance linearity, symmetry, dynamic range, weight precision, and reliability are essential synapse performance metrics for neural networks [1]–[5].
0 32 64 96 128
0 150 300 450 600
Conductance [mS/mm]
Pulse number
VGS2_P = - 0.6 V VGS2_P = - 0.5 V VGS2_P = -0.4 V Potentiation
Voltage
(a)
Inference:
VDS_I= 0.1 V VGS1_I= 0.0 V VGS2_I= 0.0 V
0 32 64 96 128
10-3 10-2 10-1 100 101 102 103
Conductance [mS/mm]
Pulse number
LTP
VDS= 0.1 V
(b)
VGS2_I= - 0.5 V VGS2_I= 0.1 V
-0.5 -0.4 -0.3 -0.2 -0.1 0.0 0.1 100
101 102 103 104 105 106
Dynamic Range
Dynamic Range (DR)
VGS2_I [V]
0.0 0.2 0.4 0.6 0.8 1.0 0.0
0.2 0.4 0.6 0.8 1.0
Normalized Conductance
Normalized Pulse #
Sim Data (LTP) Sim Data (LTD) Fit Data (LTP) Fit Data (LTD)
LTP
LTD
64 pulse for LTP and LTD NL = -0.1
NL = -2.7
(c)
VGS2_I= 0.0 V
InputLayer
100 Hidden Neurons
Output Layer 10Neurons 28 28 Cropped
Handwritten Digit
(d)
-5 -4 -3 -2 -1 0 1
-7 -6 -5 -4 -3
LTD nonlinearity
LTP nonlinearity
83.74 84.61 85.48 86.36 87.23
Accuracy
(e) 0 32 64 96 128
0 150 300 450 600
Conductance [mS/mm]
Pulse number
64 pulses 32 pulses 16 pulses 8 pulses
Inference:
VDS_I= 0.1 V VGS1_I= 0.0 V VGS2_I= 0.0 V
(f)
Fig. 4. Effect of (a) VGS2_P and (b) VGS2_I on the conductance values and dynamic range of the device. (c) Variation in normalized conductance values of the device with normalized pulse number to estimate the nonlinearity of LTP and LTD conductance. (d) Two layers neural network for MNIST image classification. (e) Dependence of image recognition accuracy on nonlinearity of LTP and LTD. (f) Variation in conductance values of the device with pulse number for different bit precisions (number of pulses).
Fig. 4(a) shows the effect of VGS2_P on LTP and LTD conductance values. As shown in Fig. 2(a), VGS2_P = -0.6 V traps more charges in the nitride layer than lower VGS2_P, which results in higher
2023 Electron Devices Technology and Manufacturing Conference (EDTM)
conductance values for LTP in Fig. 4(a). Higher back gate voltage achieves maximum conductance values during LTP. Also, it has maximum conductance values during LTD due to less de-trapping of positive charges from the nitride layer.
Fig. 4(b) shows the effect of inference voltage on the conductance values of LTP and LTD to observe the nonlinearity (NL) and dynamic range (DR = Gmax/Gmin). The nonlinearity of LTP and LTD’s conductance values are estimated through the fitting equations, as explained in [12]. The open-access MATLAB code has fitting parameters that estimate NL [13]. The minimum nonlinearity (NL = -0.1) is achieved for VGS2_I = 0.1 V as shown in Fig. 4(c), and the maximum DR (Gmax/Gmin = ~ 1.5 × 105) is achieved with VGS2_I = -0.5 V.
In order to observe the effect of NL and DR on the image classification accuracy, the neural network simulation is performed with the MLP simulator Neurosim ver. 3.0 [13]. In this simulation, we have chosen a two-layer perceptron artificial neural network with 784 as an input layer, 100 neurons for the hidden layer, and 10 for the output layer as shown in Fig. 4(d). For image classification, the MNIST dataset is used. 784 neurons for the input layer corresponds to the 28×28 MNIST image. The image is converted into black and white to reduce the complexity of input encoding. The algorithm training is performed with stochastic gradient descent (SGD) optimizer, and the learning rate for the first and second layers is 0.4 and 0.2, respectively. After the training process of patterns, the two-layer ANN is used to perform a classification task for 10,000 separate testing images.
The recognition rate was calculated for every 40,000 images during the training process. Fig. 4(e) shows the image recognition accuracy. Though the better NL is achieved with VGS2_I = 0.1 V, and maximum DR is achieved with VGS2_I = -0.5 V, the maximum accuracy (88.10 %) is achieved with optimized VGS2_I = -0.3 V. Thus, the results confirm that the device requires the optimized value of NL and DR to achieve higher accuracy. Fig. 4(f) shows the variation of conductance values with pulse numbers for different bit precision (64, 32, 16, and 8 pulses for LTP and LTD). The nonlinearity can be improved for lower precision value, but the accuracy of ANN algorithm depends on the precision value [13]. The maximum accuracy with synaptic weight is achieved for 6-bit precision.
Fig. 5 shows the effect of temperature on the charge-trapping memory-based artificial synapse performance metrics. An increase in temperature increases the generation of electron-hole pairs in the device due to thermal generation. Therefore, an increase in temperature requires less number of pulses to achieve an LTP state, as shown in Fig. 5(a).
LTP is achieved at 6th for 27 C and reduced to 3rd pulse for 105 C. Although the higher temperature requires less pulse to achieve LTP, at the same temperature also increases the recombination in the
device and reduces mobility. Due to this effect, fewer holes are trapped in the nitride layer, thus, resulting in lower conductance values, as shown in Fig. 5(b).
0 1 2 3 4 5 6 7 8 9 10 10-10
10-7 10-4 10-1 102
ID [mA/mm]
Transient Time [ms]
27 C 45 C 65 C 85 C 105 C Potentiation
(a) 00 10 20 30 40 50 60 70
50 100 150 200 250 300
Conductance [mS/mm]
Pulse number
27 C 45 C 65 C 85 C 105 C
Inference:
VDS_I= 0.1 V
(b)
Fig. 5. Variation in (a) transient drain current and (b) conductance values of LTP and LTD with different temperatures.
Conclusion
In this work, we have successfully validated the synapse behavior of junctionless transistors at a lower drain voltage. For implementing hardware neural networks, the JL device achieved better NL, symmetry, and a DR of conductance value. The JL device achieves 88.1% with optimized NL and DR values. These results make the device suitable for low-power consumption and next-generation neuromorphic computing applications.
Acknowledgments
This work was supported by the King Abdullah University of Science and Technology baseline fund.
References
[1] S. Yu, “Neuro-Inspired Computing with Emerging Nonvolatile Memorys,” Proc. IEEE, vol. 106, no. 2, 2018.
[2] G. Zhou et al., “Volatile and Nonvolatile Memristive Devices for Neuromorphic Computing,” Adv. Electron. Mater., vol. 8, no. 7, 2022.
[3] D. Marković et al., “Physics for neuromorphic computing,” Nat.
Rev. Phys., vol. 2, no. 9, 2020.
[4] G. Indiveri and S.-C. Liu, “Memory and Information Processing in Neuromorphic Systems,” Proc. IEEE, vol. 103, no. 8, 2015.
[5] C.-H. Kim et al., “Emerging memory technologies for neuromorphic computing,” Nanotechnology, vol. 30, no. 3, 2019.
[6] D. Kwon et al., “On-Chip Training Spiking Neural Networks Using Approximated Backpropagation With Analog Synaptic Devices,”
Front. Neurosci., vol. 14, no, 2020.
[7] “Atlas User’s Manual.,” St. Clara, CA, USA Silvaco.
[8] S. Sahay et al.,, “Physical Insights into the Nature of Gate-Induced Drain Leakage in Ultrashort Channel Nanowire FETs,” IEEE Trans.
Electron Devices, vol. 64, no. 6, 2017.
[9] M. S. Parihar et al., “Bipolar effects in unipolar junctionless transistors,” Appl. Phys. Lett., vol. 101, no. 9, 2012.
[10] M. H. R.Ansari et al., “Core-Shell Dual-Gate Nanowire Synaptic Transistor with Short/Long-Term Plasticity,” in 2021 5th IEEE Electron Devices Technology & Manufacturing Conference (EDTM), 2021.
[11] M. H. R. Ansari et al., “Core-Shell Dual-Gate Nanowire Charge-Trap Memory for Synaptic Operations for Neuromorphic Applications,” Nanomaterials, vol. 11, no. 7, 2021.
[12] X. Sun et al.,, “Impact of Non-Ideal Characteristics of Resistive Synaptic Devices on Implementing Convolutional Neural Networks,” IEEE J. Emerg. Sel. Top. Circuits Syst., vol. 9, no. 3, 2019.
[13] P.-Y. Chen et al., “NeuroSim+: An integrated device-to-algorithm framework for benchmarking synaptic devices and array architectures,” in 2017 IEEE International Electron Devices Meeting (IEDM), 2017.
