International Journal of Distributed Sensor Networks
2019, Vol. 15(9) ÓThe Author(s) 2019 DOI: 10.1177/1550147719872452 journals.sagepub.com/home/dsn
A fuzzy recurrent neural network for driver fatigue detection based on
steering-wheel angle sensor data
Zuojin Li1 , Qing Yang1, Shengfu Chen1, Wei Zhou1, Liukui Chen1 and Lei Song2
Abstract
The study of the robust fatigue feature learning method for the driver’s operational behavior is of great significance for improving the performance of the real-time detection system for driver’s fatigue state. Aiming at how to extract more abstract and deep features in the driver’s direction operation data in the robust feature learning, this article constructs a fuzzy recurrent neural network model, which includes input layer, fuzzy layer, hidden layer, and output layer. The steering-wheel direction sensing time series sends the time series to the input layer through a fixed time window. After the fuzzification process, it is sent to the hidden layer to share the weight of the hidden layer, realize the memorization of the fatigue feature, and improve the feature depth capability of the steering wheel angle time sequence. The experi- mental results show that the proposed model achieves an average recognition rate of 87.30% in the fatigue sample data- base of real vehicle conditions, which indicates that the model has strong robustness to different subjects under real driving conditions. The model proposed in this article has important theoretical and engineering significance for studying the prediction of fatigue driving under real driving conditions.
Keywords
Fatigue driving features, fuzzy recurrent neural network, steering wheel angle, robust learning
Date received: 30 June 2019; accepted: 1 August 2019 Handling Editor: Xing Chen
Introduction
Road traffic accidents are one of the main causes of human casualties. According to the World Health Organization (WHO),1 traffic accidents will jump to the fifth place in the main cause of death in 2030, far higher than infectious diseases such as AIDS and tuber- culosis. As the primary cause of traffic accidents, fati- gue driving has attracted wide attention from all walks of life in recent years. In 2014, NHTSA’s FARS data- base had 846 driving-related fatal accidents, which was basically consistent with the death data of the past decade; between 2005 and 2009, there were an average of 83,000 accidents related to fatigue driving each year, and 886 among them were fatal accidents (accounting for 2.5% of all fatal accidents).2Therefore, research on
identification of driver fatigue state and early warning technology under real vehicle conditions is conducive to reducing road traffic accidents caused by fatigue driving.
Fatigue driving usually refers to the driver’s mental and physiological dysfunction after a long-time driving,
1College of Electrical and Information Engineering, Chongqing University of Science and Technology, Chongqing, China
2School of Computing and Information Technology, Unitec Institute of Technology, Auckland, New Zealand
Corresponding author:
Zuojin Li, College of Electrical and Information Engineering, Chongqing University of Science and Technology, Chongqing 401331, China.
Email: [email protected]
Creative Commons CC BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License (http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages
(https://us.sagepub.com/en-us/nam/open-access-at-sage).
which naturally leads to a decrease in the driver’s driv- ing control ability. According to the sensor data source of the fatigue monitoring systems, the existing driver fatigue monitoring systems mainly fall into two types:
intrusive and non-intrusive. The intrusive fatigue moni- toring system mainly analyzes the physiological charac- teristics of the driving process by collecting the physiological data of the driver and realizes the moni- toring of the driver’s fatigue state. The data used in such systems mainly include electroencephalography (EEG) signals, electro-optical (EO) signals and electro- cardiography (ECG) signals.3–6The results of other lit- erature studies show that the main disadvantage of the contact fatigue monitoring system is that the system needs to install a data acquisition sensor in a certain part of the driver’s body, which may easily lead to dis- traction or discomfort of the driver, and thus its appli- cation in real conditions is restricted. Non-intrusive systems primarily monitor driver fatigue by extracting driver’s facial features and operational behavior char- acteristics.7–11During the monitoring process, the driv- er’s normal driving will not be disturbed, and the data acquisition and fatigue monitoring can objectively reflect the driver’s actual operating behavior or state.11 At present, the non-intrusive method has become pop- ular in the field of driver fatigue monitoring.12
A fatigue monitoring system based on driver’s steer- ing wheel angle (SWA) information is a type of non- intrusive method. Studies have shown that after driving for a long time, the driver will suffer from lack of energy or fatigue. At this time, his ability to operate or control the vehicle will be greatly reduced, resulting in decline in steering wheel rotation accuracy and fre- quency.13,14Therefore, by collecting the real-time oper- ation data of the driver’s steering wheel, the fatigue characteristics of the direction angle data change are mined, and the driver fatigue level identification model is constructed to realize the effective detection of the driver’s fatigue level.15–17
Eskandarian and Mortazavi18established a monitor- ing model and method for the driver’s fatigue level by analyzing the amplitude variation characteristics of the driver’s direction operation data, and the correct recog- nition rate of the fatigue level is 85%. Berglund19used simulator data for fatigue driving experiments on 22 subjects, which collected state variables of 10 vehicles including SWA, steering wheel angular velocity, steer- ing wheel torque, yaw rate, and vehicle lateral position.
With the extracted 17 fatigue indicators and a linear regression model, the detection accuracy of fatigue detection reached 87%. Fukuda et al.20 analyzed the statistical characteristics of driver’s SWA sequence and found that the periodicity of the time series has a cer- tain internal relationship with the fatigue state, realiz- ing the online identification of the driver’s fatigue level, and the correct recognition rate of the fatigue level
reached from 76% to 88%. In addition, based on the statistical analysis of the driver’s direction angle data, Qu et al.2and others extracted 11 characteristic indica- tors reflecting the driver’s fatigue level, and constructed a SVM-based fatigue classification recognition model.
The correct monitoring rate of the third-level fatigue hit 87.7%. However, most of the research data in the existing literature comes from the driving simulator environment, and the reliability of the fatigue monitor- ing system developed on this basis still lacks effective verification under real vehicle conditions.
The existing literature monitors a lot of work based on the fatigue of the driver’s direction angle, but these methods have extracted more statistical features of driv- ers’ operation time series, and shows little of their indi- vidual difference, which easily leads to the fact that the fatigue detection method is not robust enough under real vehicle conditions. Bittner et al.21 reported the effectiveness of the existing fatigue feature indicator under actual road conditions. It was found that angle standard deviation with good performance in the driv- ing simulation environment is not effective in the fati- gue detection under real vehicle conditions. This is because the steering characteristics of the real vehicles are not only related to the fatigue level of the driver, but also related to the external environment such as the operator’s driving habits, steering ability, speed, and actual road conditions; Meanwhile, due to the random jitter on the road, the probability of mixing noise and drift in the SWA data is greatly increased, which leads to more complicated analysis of the SWA characteris- tics, feature selection, and performance optimization under real vehicle conditions.
In addition to the feedforward connection structure of the traditional network, the recurrent neural net- work realizes the recurrent connection between the neu- rons in the network, that is, the hidden layer units are not independent of each other, and the hidden layers are not only related to each other, but also the time series input before the time of the hidden layer unit is accepted, so that, in this structure, drivers’ operational characteristics and fatigue features can be transmitted in the direction of the time series. This advantage enables it to make full use of historical information and achieve a series of remarkable results in solving problems such as time-series nonlinear system model- ing.22–25This article makes use of the unique advantage of the recurrent neural network in processing time series data, designs the feature extraction model for the driver’s direction angle time series under real driving conditions, and deeply exploits the driver’s operational characteristics in the time direction. A robust driver fatigue feature learning model is constructed to solve the individual differences and random complexity faced by the driving fatigue detection system.
Background
Usually, the driving process can be considered as a typ- ical nonlinear dynamic system. Mining the nonlinear characteristics of the time series data of the operational state is beneficial to the driver’s fatigue state analysis and identification. Under the real driving conditions, in order to ensure safe driving, the driver must constantly judge the driving state of the vehicle. Once the vehicle state is out of the estimation, it should be corrected in real time. When the driver is in a state of fatigue, the driver’s ability to perceive the environment, the ability to judge the situation, and the ability to control the vehicle will decrease, resulting in greater error tolerance to the vehicle and reduced control accuracy, and, con- sequently, the vehicle control variables and state vari- ables will have a different amplitude or frequency with those in the awake state. Existing studies, Krajewski et al.26and Zhang et al.,27have shown that fatigued driv- ers’ behaviors in operating the steering wheel become obviously abnormal compared with those in a non- fatigued state. For instance, the correction time interval of the steering wheel becomes longer and the instanta- neous correction range becomes larger. Therefore, the driver’s steering wheel operating characteristics can reflect the fatigue state of the driver, and the nonlinear characteristics, irregularities, and individual differences implied by the steering wheel operation time series data also vary under different fatigue states.
Traditional artificial neural networks, also known as feedforward neural networks (FNNs), consist of a series of simple neurons. A simple FNN usually con- sists of an input layer, a hidden layer, and an output layer. The neuron response in the FNN is usually derived from the different degrees of excitation of all neurons in the upper layer, that is, mathematically can be expressed as the sum of the weights of all the neu- rons in the upper layer, but there is no connection between the neurons in the same layer of the FNN.
There is no loop on the network structure, and there is no feedback connection between the output of the net- work and the model itself. Data pass through the neural network layer by layer from the input layer until the final output layer. The observations of all the neu- ron cells in the same layer of the traditional artificial neural network are processed independently of each other. However, the data in many tasks are rich in con- text information, and there is a strong correlation between them in time, such as, audio, video, and text, so FNN still has a lot of limitations in many tasks, especially the time series features with strong correla- tion in mining time dimension and thus the perfor- mance of traditional artificial neural network is not promising. The biggest difference between recurrent neural networks and traditional artificial neural net- works is that the neurons between the hidden layers
have temporal storage and recursion. The output of neurons at a certain moment can be used as the input of the next layer of neurons. The network structure is well suited for time series data and can maintain depen- dencies between the data. The classical recurrent neural network structure is shown in Figure 1,28in which the H layer is an implicit layer, which is connected through the loops on the hidden layer, so that the network state at the previous moment can be transmitted to the cur- rent moment, and the current state can also be passed to the next moment. For the expanded recurrent neural network, a repetitive structure can be obtained and the parameters in the network structure are shared, greatly reducing the neural network parameters required for training. Therefore, the recurrent neural network has a good advantage in studying the behavioral characteris- tics of the driver and the learning of fatigue characteristics.
Methodology
Combining the self-organizing, adaptive learning and selective long-term memory ability of self-organizing competitive network, a new fuzzy recurrent neural net- work is constructed to extract fatigue characteristics, and the fatigue feature space of the driver is obtained.
In this article, a four-layer fuzzy recurrent neural net- work model is designed, and its structure is shown in Figure 2.
In Figure 2, the first layer is the input layer, and the input data are the spatio-temporal feature quantity of the SWA under real road conditions, and the input variable isXi,i=1,2,. . .,60. The direction angle time series is processed with double windowing, as shown in Figure 3.
In Figure 3, time windowti=1s,Xi is the average time window
Xi=1 f
XSWAnðn=1,2,. . .,fÞ ð1Þ
Figure 1. Traditional recurrent neural network model.
In equation (1), f is the sampling frequency of the direction angle time series, and f =100. SWAn is the steering-wheel Angle at a certain sampling point when the driver steering the wheel.
In Figure 2, the second fuzzified neuron is blurred by a smooth Gaussian function, and the fuzzification of the second layer is expressed by equation (2)
Fi= 1
1+exi ð2Þ
The third layer is the hidden layer of the recurrent neural network. The neurons are not only related to the excitation output of the previous layer but also real- ize the information transmission of the neurons in the same layer, making weight influence on the neighbors and depicting the precise relationship between state and time under the real vehicle conditions. The forward propagation of the hidden layer can be expressed by equation (3)
Hi=sðWFHFi+WHHHi1+bHÞ ð3Þ In equation (3), WFH is the weight matrix from the fuzzy layer to the hidden layer, WHH is the connection weight matrix between the hidden layer neurons, and bHis an offset vector. The recurrent neuron coefficients are trained by backpropagation through time (BPTT) algorithm. For gradient-based learning, there is usually a closed solution between the model parameter uand the loss function L. The loss function is minimized according to the error between the estimated value and the actual value. The calculated gradient information can be transmitted back to the model to modify the neural network parameters. Assume for the sequence
Z1,Z2,. . .,Zt, the state of the previous momentSt1 is
mapped to the state of the next moment St through St=Gu(St1,Zt). The gradient of the hidden layer neu- ron loss function is shown in equation (4)
ruLT=∂LT
∂u = X
tłT
∂LT
∂ST
∂ST
∂St
∂GuðSt1,ZtÞ
∂u ð4Þ
According to the chain rule, the Jacobian matrix ∂S∂ST is decomposed as shown in equation (4) t
∂ST
∂St
= ∂ST
∂ST1
∂ST1
∂ST2
∂St+1
∂St
ð5Þ
The fourth layer in Figure 2 is the competition layer, and the competition winner, output 1, is transmitted to the next neuron through the memory unit S and consti- tutes a sequence of fatigue characteristic indicators. The algorithm for finding winning neurons is as follows:
assume that the mode input through the input layer is X = (X1,X2,. . .,Xn), where Xi=½xi,xi 2I½0,1,
i=1,2,. . .,n, and all the neurons of the competition
layer correspond to the weight vector wj= (wj1,wj2,. . .,wjn)(j=1,2,. . .,m); the weight vec- tor most similar toX is judged as the winning neuron.
Finally, adjust the weight according to the Winner- Take-All learning rule. The adjustment rules are
wjðt+1Þ=wjð Þt +a f w j1,X
,f w j2,X
,. . .,f w jn,X
ð6Þ f w ji,X
=sgn x iwji
d w ji,X
ð7Þ In equation (6), a2 ½0,1 is the learning rate. After fully learning with this method, the weight vector of the winning neuron becomes a clustering center of the input mode.
Experiment and results
Experiment platform and sample data
The structure of the real vehicle and road data acquisi- tion platform is shown in Figure 4. Distributed sensors Figure 2. An improved recurrent neural network model.
Figure 3. Time window and sample window of SWA sensor data.
are mainly used for data acquisition of driver operating characteristics and vehicle status such as SWA, brake- froce, leftsteer, rightsteer, can_braking, can_thrott, and YawRate. The data collected by the sensor are pro- cessed by the VBOX system, to realize data storage, display and video monitoring, and so on.
With the data acquisition platform shown in Figure 4, the subjects’ facial video images are obtained.
This article uses the facial-video-based expert scoring method to construct the driver fatigue state sample database. The method scores the fatigue status of a driver based on the driver’s facial expression and head posture. Wierwille and Ellsworth29 first cited this method in the driver fatigue state assessment. The spe- cific operation steps are as follows: the driver’s facial video is divided into video clips; multiple experts score the video clips according to the random sequence according to the driver’s blinking, scratching, yawning, closing the eyes, adjusting the posture, and other fati- gue characterizations. The evaluation result is a contin- uous value of 0–100 points; the average of multiple expert scores is used as the fatigue score of the video.
In this article, the driver’s fatigue state is set to three levels, that is, awake, fatigue, and very fatigue level.
With the driver’s facial video as the evaluation basis, the driver’s operation characteristics and vehicle state data collected by VBOX are taken as experimental data. Considering the complexity of multidimensional spatio-temporal data, this article only takes the driver’s SWA data as the research object to test the feasibility of the designed model in this article. The time series
distribution of the SWA when the driver’s awake, fati- gued, and very fatigued is, respectively, shown in Figures 5–7. As can be seen from Figure 5, when the driver is in the awake state, the frequency of the wheel- steering operated by the driver is relatively high, and the frequency of the substantial direction operation is low. When the driver is getting tired, the driver’s direc- tion steering frequency is reduced, and the frequency in dramatic steering is increased as shown in Figure 6.
When the driver is very tired, with the decrease of driv- er’s control ability, the time of non-operating the steer- ing wheel increases, and at the same time, the driver Figure 4. Acquisition platform for driver operation and vehicle data.
Figure 5. Time series of SWA at awake level.
quickly and dramatically turns the steering wheel, and the time series distribution is as shown in Figure 7.
From the fatigue driving data of eight subjects, after data filtering processing, 790 fatigue driving samples were formed, and the fatigue state level distribution of each subject is shown in Figure 8. In the experiment of the article, the number of neural network layers is four layers, the input layer node, the fuzzy layer, the hidden layer, and the output layer node are all 60, and the learning rate is 0.42.
Experiment results
In this article, 60% of the samples are randomly selected from the 790 samples of eight subjects as train- ing samples, and the remaining 40% are test samples.
First, the ‘‘very fatigued’’ level is incorporated into the general ‘‘fatigued’’ level, that is, the fatigue level of the test sample is divided into two levels: awake and fati- gued, and all the sample test results of the fatigue level
‘‘0’’ and ‘‘1’’ of the subjects are shown in Table 1. The test samples in Table 1 totals 315, including 222 cases of fatigue level ‘‘0’’ and 93 cases of fatigue level ‘‘1.’’ It can be seen from Table 1 that the correct detection rate of fatigue level ‘‘0’’ is 90.54%, and misjudgment rate is 9.46%. For the fatigue level ‘‘1,’’ the correct detection rate is 80.65% and the false positive rate is 19.35%.
Second, the sample data are tested according to the three fatigue levels, that is, the fatigue states of the sample are ‘‘awake,’’‘‘fatigued,’’ and ‘‘very fatigued,’’
which are represented by ‘‘0,’’‘‘1,’’ and ‘‘2,’’ respectively.
The results are as shown in Table 2: there are 315 experimental test samples, including 222 cases of awake state, 76 cases of fatigued state, 17 cases of very fati- gued state, the correct recognition rate of samples in Figure 6. Time series of SWA at fatigue level.
Figure 7. Time series of SWA at very fatigue level.
Figure 8. Samples distribution of driver fatigue state.
awake state is 90.05%, and the false rate is 9.91%, all of which are falsely judged as fatigue level 1. The cor- rect recognition rate of the fatigue state sample is 80.26%, and the false rate is 19.74%. Among them, 10.21% fatigued samples are misjudged to awake, and 9.53% misjudged to very fatigued; the recognition rate of the very fatigued sample is 76.47%, and the false rate is 23.53%, among which 5.52% very fatigued sam- ples are misjudged to awake, and 18.01% are mis- judged to fatigued state. The average false positive rate is 9.68% and the average false negative rate is 13.99%
from the all samples. Based on all the test samples in Tables 1 and 2, the average correct recognition rate is 87.30%.
Discussion
The method proposed in this article uses the SWA data to detect the driver’s fatigue state based on the fact that the driver’s mental state can directly reflect its opera- tional behavior. The steering wheel rotation is the most frequent and sensitive operation of the driver. Using the SWA makes the system more robust and reliable.
The fatigue detection method based on the SWA can identify the three levels of fatigue status: ‘‘awake,’’‘‘fati- gued,’’ and ‘‘very fatigued.’’ The method proposed in this article takes the driving sample data of eight sub- jects as the experimental object. The correct rate for fatigue state detection reaches 87.02%. The reliability of the detection system is ensured by the fact that the direction angle data in obtained in the real road envi- ronment and more accurately reflects the mental state and operational behavior of the driver than the data in the laboratory simulation environment. At the same
time, the detection system has a very low rate of false alarms of ‘‘fatigued’’ state (9.46%) from Table 1, and the rate of false alarms of ‘‘very fatigued’’ state (9.53%) from Table 2. This detection system is also robust because the evaluation criteria of the system comes from the consistent evaluation results of multiple experts and multiple methods, which is based on facial video information, rather than the direction angle information. The universality of the standard evalua- tion method is resulting from its combination of the driver’s facial expression, head posture, and mental state conditions to comprehensively score the driver’s fatigue rating. The recursiveness of the evaluation cri- teria makes the detection system very robust.
Unfortunately, there’s little existing literature report- ing the direction-angle fatigue detection method based on real vehicle conditions. It is difficult to have an objective comparison of the superiority of the method proposed in this article. Compared with the author’s previous research, the correct recognition rate of the proposed method, 87.30%, is higher than the previous results, 82.07% and 78.01%.15,16 Qu et al.,2using the simulation environment sample data under the labora- tory platform, with the SWA data, has obtained a cor- rect recognition rate of 86.1%, which is lower than the correct recognition rate of this article, 87.30%.
Comparing the directional data characteristics of the real vehicle environment and the laboratory environ- ment, it is found that in actual vehicle conditions, there is a serious drift in the steering wheel corner sequence due to the randomness of the vehicle vibration. The irregular drift of the time series exists in the original data as the statistical features of an illusion, making the statistical feature extraction of the directional angles extremely difficult. Although the correct rate of the driver’s fatigue state detection in the direction angle is lower than that of the biomedical signal (The accu- racy of estimation is about 96.5%–99.5%),3consider- ing the actual driving environment, the method proposed in this article is more convenient and feasible.
Conclusion and future work
A fatigue driving characteristic learning model based on recurrent neural network structure is proposed. This Table 1. Confusion matrix of the SWA based for detection
drowsiness levels ‘‘0’’ and ‘‘l.’’
Detection results
‘‘Awake’’
(level 0)
‘‘Fatigued’’
(level 1) Expert
classification
‘‘Awake’’ (level 0) 90.54% 19.35%
‘‘Fatigued’’ (level 1) 9.46% 80.65%
Samples 201 93
Table 2. Confusion matrix of the SWA based for detection fatigue levels ‘‘0,’’ ‘‘l,’’ and ‘‘2.’’
Detection results
‘‘Awake’’ (level 0) ‘‘Fatigued’’ (level 1) ‘‘Very fatigued’’ (level 2)
Expert classification ‘‘Awake’’ (level 0) 90.09% 9.21% 5.88%
‘‘Fatigued’’ (level 1) 9.91% 80.26% 17.65%
‘‘Very fatigued’’ (level 2) 0.00% 10.53% 76.47%
Samples 200 76 17
SWA: steering wheel angle.
model has solved the problem that the fatigue feature extraction of steering wheel operation data is difficult under real vehicle conditions. Considering the safety in real driving conditions, more researches were done in the past on the simulation angle data based on the laboratories and much less on the data based on the actual driving conditions. This article uses multi- distributed sensors installed in the real vehicle system, collecting data on driver operating characteristics and vehicle status such as SWA, brakefroce, leftsteer, rightsteer, can_braking, can_thrott, YawRate, and facial video, combining the time series distribution characteristics of steering data, and innovatively pro- poses a fuzzy circulatory neural network model. Taking 557 samples as the object, this method has achieved the fatigue state identification rate of 87.30%, surpassing the fatigue state detection based on laboratory simula- tion data.
Previous work of Qu et al.2 has shown that the SWA, combined with vehicle yaw angle, vehicle lateral position, and other vehicle status data, has a higher fatigue state recognition rate in the laboratory simula- tion environment. Inspired by this idea, the integration of SWA, vehicle yaw angle, vehicle lateral position, and other vehicle status data under real vehicle conditions may improve the driver’s fatigue detection accuracy, which is a meaningful work for us.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial sup- port for the research, authorship, and/or publication of this article: This work was supported by the Natural Science Foundation of China under grant number 61873043 and, in part, by the Natural Science Foundation of Chongqing under grant number cstc2018jcyjAX0048.
ORCID iD
Zuojin Li https://orcid.org/0000-0002-8154-3968
References
1. World Health Organization (WHO).Global status report on road safety: time for action. Geneva: WHO, 2009, http://www.who.int/violence_injury_prevention/road_saf ety_status/2009/en/
2. Qu X, Cheng B, Lin Q, et al. Drowsy driving detection based on driver’s steering operation characteristics.Qiche Gongcheng/Automot Eng2013; 35(9): 803–831.
3. Zhang C, Wang H and Fu R. Automated detection of driver fatigue based on entropy and complexity mea- sures.IEEE T Intell Transp2013; 15(1): 168–177.
4. Picot A, Charbonnier S and Caplier A. On-line detection of drowsiness using brain and visual information.IEEE T Syst Man Cy A2012; 42(3): 764–775.
5. Jung SJ, Shin HS and Chung WY. Driver fatigue and drowsiness monitoring system with embedded electrocar- diogram sensor on steering wheel.IEEE T Intell Transp 2014; 8(1): 43–50.
6. Correa AG, Orosco L and Laciar E. Automatic detection of drowsiness in EEG records based on multimodal anal- ysis.Med Eng Phys2013; 36(2): 244–249.
7. Ma T and Chen B. Detection of driver’ drowsiness using facial expression features. J Automot Saf Energy 2010;
13(5): 200–204.
8. Wu M, Wu P and Zhang C. Driver fatigue detection algorithm based on the states of eyes and mouth.Com- puter Appl Softw2013; 30(3): 25–54.
9. Lin SD, Lin JJ, Chen CH, et al. Sleepy eye’s recognition for drowsiness detection. Int Symp Biom Secur Technol 2013; 14(7): 176–179.
10. Wierwille WW, Wreggit SS, Kirn CL, et al.Research on vehicle-based driver status/performance monitoring; devel- opment, validation, and refinement of algorithms for detec- tion of driver drowsiness. Final Report No. DOT HS 808 638, September 1996. Washington, DC: National High- way Traffic Safety Administration.
11. King DJ, Siegmund GP and Montgomery DT. Outfitting a freightliner tractor for measuring driver fatigue and vehicle kinematics during closed-track testing.J Commer Veh1994; 103(2): 611–621.
12. Wang JQ, Li SE, Zheng Y, et al. Longitudinal collision mitigation via coordinated braking of multiple vehicles using model predictive control. Integr Comput-Aid E 2015; 22(2): 171–185.
13. Wang J, Lu M and Li K. Characterization of longitudi- nal driving behavior by measurable parameters. Trans- port Res Rec: J Transp Res Board2010; 2185: 15–23.
14. Wang J, Zhang L, Zhang D, et al. An adaptive longitudi- nal driving assistance system based on driver characteris- tics.IEEE T Intell Transp2013; 14(1): 1–12.
15. Li Z, Li S, Li R, et al. Driver fatigue detection using approximate entropic of steering wheel angle from real driving data.Int J Robot Autom2017; 32(3): 291–298.
16. Li Z, Li S, Li R, et al. Online detection of driver fatigue using steering wheel angles for real driving conditions.
Sensors2017; 17(3): 495.
17. Li Z, Chen L, Peng J, et al. Automatic detection of driver fatigue using driving operation information for transpor- tation safety.Sensors2017; 17(6): 1212.
18. Eskandarian A and Mortazavi A. Evaluation of a smart algorithm for commercial vehicle driver drowsiness detec- tion. In:Proceedings of the 2007 IEEE intelligent vehicles symposium, Istanbul, 13–15 June 2007, pp.553–559. New York: IEEE.
19. Berglund J.In-vehicle prediction of truck driver sleepiness:
steering related variables. Institutionen Fo¨r Systemteknik, 2007, https://pdfs.semanticscholar.org/67b9/48606bd8fbd 2bc85dbb87b57bd3d0e7d7049.pdf?_ga=2.75121175.4407 16504.1565588886-1383787297.1561959603
20. Fukuda J, Akutsu E and Aoki K. Estimation of driver’s drowsiness level using interval of steering adjustment for lane keeping.JSAE Rev1995; 16(2): 197–199.
21. Bittner R, Smrcka P, Vysoky´ P, et al. Detecting of fati- gue states of a car driver. In:International symposium on medical data analysis, Frankfurt, 29–30 September 2000, pp.260–273. Berlin; Heidelberg: Springer-Verlag.
22. Palangi H, Deng L, Shen Y, et al. Deep sentence embed- ding using long short-term memory networks: analysis and application to information retrieval. IEEE/ACM IEEE T Audio Speech2016; 24(4): 694–707.
23. Nguyen VQ, Anh TN and Yang H-J. Real-time event detection using recurrent neural network in social sen- sors.Int J Distrib Sens N2019; 15(6): 1–15.
24. Lee J. Deep learning-assisted real-time container corner casting recognition. Int J Distrib Sens N 2019; 15(1):
1–10.
25. Shi L, Wang L and Wang Z. The modeling and the sensor fault diagnosis of a continuous stirred tank reactor with a
Takagi-Sugeno recurrent fuzzy neural network.Int J Dis- trib Sens N2009; 5(1): 37.
26. Krajewski J, Golz M and Sommer D. Detecting sleepy drivers by pattern recognition based analysis of steering wheel behaviour. In:Proceedings of the fifth international driving symposium on human factors in driver assessment, training and vehicle design, Big Sky, MT, USA, 22–25 June 2009, pp.118–224..
27. Zhang X, Cheng B and Feng R. Real-time detection of driver drowsiness based on steering performance.Tsing Hua Univ (Sci & Tech)2010; 50(7): 1072–1081.
28. Elman JL. Finding structure in time.Cognitive Sci1990;
14(2): 179–211.
29. Wierwille WW and Ellsworth LA. Evaluation of driver drowsiness by trained observers. Accident Anal Prev 1994; 26(5): 571–581.
