Advanced structural health monitoring in carbon fiber-reinforced plastic using real-time self-sensing data and convolutional neural network architectures

(1)

Advanced structural health monitoring in carbon fiber-reinforced plastic using real-time self-sensing data and convolutional neural network architectures

In Yong Lee

¹

, Juhyeong Jang

¹

, Young-Bin Park

^⇑

Department of Mechanical Engineering, Ulsan National Institute of Science and Technology, UNIST-gil 50, Ulju-gun, Ulsan 44919, Republic of Korea

h i g h l i g h t s

Advanced real-time structural health monitoring method was proposed using electromechanical behavior data images and deep learning tools.

Proposed methodology overcome the limitation of self-sensing in previous research such as lots of electrodes embedding and small damage detectable area.

Convolutional neural network architectures were designed and optimized for damage localization and severity identification with real- time self-sensing data.

Proposed methods were applied to various types of carbon fiber reinforced plastic. And it showed wide range of applicability to CFRP structures in industries.

g r a p h i c a l a b s t r a c t

a r t i c l e i n f o

Article history:

Received 27 April 2022 Revised 4 October 2022 Accepted 4 November 2022 Available online 18 November 2022 Keywords:

A. Polymer-matrix composites A. Smart materials

D. Non-destructive testing

a b s t r a c t

In this study, advanced structural health monitoring (SHM) using a non-destructive self-sensing methodology was proposed for large-sized carbon fiber-reinforced plastic (CFRP). Cyclic point bending tests were performed on three types of CFRPs. The damage severity identification and localization were classified and investigated using four different convolutional neural network (CNN) architectures. Electrical resistance images were used to train each CNN architecture for damage analysis. An optimized CNN architecture for the damage analysis of CFRPs using electrical resistance data was proposed and compared with traditional damage analysis CNN architectures. The applicability of the proposed SHM methodology was verified by analyzing unseen damage in the CFRPs. This study addresses the limitations of previous self- sensing methods by reducing the number of electrodes, which reduces data complexity and increases the sensible area of CFRPs. Thus, this study successfully designed an efficient SHM methodology with a high accuracy of over 90 % for analyzing CFRP damage, including the severity and location, regardless of the type of carbon fiber and stacking sequence of composite structures that showed high applicability.

Ó2022 Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://

creativecommons.org/licenses/by-nc-nd/4.0/).

https://doi.org/10.1016/j.matdes.2022.111348 0264-1275/Ó2022 Published by Elsevier Ltd.

This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

⇑Corresponding author.

E-mail address:ypark@unist.ac.kr(Y.-B. Park).

1These authors contributed equally to this article.

Contents lists available atScienceDirect

Materials & Design

j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / m a t d e s

(2)

1. Introduction

Carbon fiber-reinforced plastic (CFRP), which has a high specific strength, has gained immense popularity in various industrial fields such as automobiles and aircraft that require stiff, strong, and lightweight structures. However, numerous instances of failure of composite structures also have been reported. Therefore, structural health monitoring (SHM) of these composite structures is crucial. Many non-destructive evaluation (NDE) methodologies, such as lead zirconate titanate (PZT) sensors [1–3], fiber Bragg grating[4–6], and acoustic emission[7–9]have been studied for SHM. Self-sensing that uses the piezoresistive effect, wherein a change in the electrical resistivity occurs upon subjecting the material to mechanical strain, have been reported [10–14].

Although many studies have investigated the electromechanical behavior of CFRPs, those for the large size of CFRPs have presented a significant limitation. In previous studies, the optimized electrode distance, which is within 100 mm for damage localization, has been reported[15–18]. It means that highly accurate damage analysis using the electromechanical behavior requires many electrodes to be embedded in large size of CFRPs. Real-time SHM of a large area of CFRPs with a minimum number of electrodes should be studied for efficient SHM system and reducing data complexity.

As deep learning has gained immense popularity owing to the advent of the fourth industry, many types of convolutional neural networks (CNNs) have been developed. CNNs have been recently studied in many research fields and have been widely used for the optimization of material design, material property prediction[19–22], classification [23–25], and SHM in various research fields [26–30].

Furthermore, a CNN combined with other types of neural networks, such as recurrent neural networks and long short-term memory, have been studied for material property prediction[31–33]. In self-sensing research, the optimized electrode distance is within 100 mm. Within this distance, monitoring sensitivity on damage up to tenths place range is shown to be appropriate for analyzing damages based on the change in gradients of electrical resistance. However, over this electrode distance, sensing sensitivity drastically decreases, which may hinder damage analysis. For large composite structures, many electrodes are required for self-sensing-based damage identification, which makes real-time health state monitoring system impractical due to data complexity. Therefore, the used of CNN, which is widely applied for image analysis and classification, was proposed to identify the deformation condition of CFRPs using self-sensing images.

In this study, an advanced SHM methodology that uses electrical resistance images and a CNN was proposed. The SHM of various types of CFRPs was investigated using the piezoresistive effect. This self-sensing method, which requires no additional sensors, is suitable for real-time SHM. The limitation of the electrode distance for damage analysis was overcome by using the CNN architecture. The health states of three types of large CFRPs were monitored in real time. The damage severity identification and damage localization were investigated using the proposed CNN architecture. In addition, an CNN architecture was designed and optimized for damage analysis using electrical resistance images and compared with the traditional CNN architecture for damage localization. The applicability of the SHM methodology was verified by analyzing the unseen damage in the CFRPs. Furthermore, the proposed methodology widens the applicability of self-sensing by increasing the sensing area.

2. Experimental 2.1. Materials

Unidirectional (UD) carbon fiber (type T700SC, 2 K, Mitsubishi, Japan) with a density of 299 g/ and thickness of 0.35 mm was pur-

chased from Keun Young Industry (Seoul, Korea). A 3 K plain- woven carbon fiber sheet (Mitsubishi, Japan), as shown inFig. 1 (a), was purchased from Jet Korea Corp. (Changwon, Korea). The density of the sheet was 305 g/m² and its ply thickness was 0.2 mm. Vinlyester (45 % styrene and 55 % epoxy acrylate) (RF- 1001MV, Epovia) and a methyl ethyl ketone peroxide curing agent (Arkema, USA) were mixed with 1.0 wt% of the polymer.

An electrode (30 AWG copper wire) was embedded in the UD carbon fiber, and silver paste (P-100, Elcoat, USA) was applied at the conjunction to minimize the contact electrical resistance between the carbon fiber and wire.

2.2. Sample preparation

For real-time damage identification and localization, three types of CFRP were manufactured using carbon fiber, as shown in Fig. 1. The thickness of all specimens was 2.5 mm with 12 plies of woven fabric carbon fiber and 8 plies of UD carbon fiber. For the UD CFRP specimen, two different stacking sequence samples, and , were manufactured. All the composite samples were fabri- cated using vacuum-assisted resin transfer molding (VARTM). Four electrodes were installed in CFRPs of size 500 mm500 mm2.

5 mm, and silver paste was used to minimize the contact resistance at the electrodes for electrical resistance measurement. Each electrode was embedded with gaps of 250 mm. The locations of embedded electrodes are shown inFig. 1.

2.3. Experimental setup

Cyclic bending tests with three different damage levels were performed to investigate the electromechanical behavior of the samples using applied weight on CFRPs (Fig. 2(a) and (b)). Four square spacers were used for the cyclic bending tests. Each spacer is 3 cm thick and is larger than the maximum bending deflection.

The spacers were placed at the four edges of CFRP. The three damage levels were achieved by using the three different weights on the CFRP, as shown inTable. 1. Each CFRP sample was divided into 16 large sections, and in each of these large sections, five sub- sections were created, as shown inFig. 2(c). Therefore, a total of 80 sections were divided in each of the three CFRP samples. The cyclic bending test was repeated 10 times for all 80 sections in each CFRPs. For instance, in section 7 shown as Fig. 2(c), five sub-sections were divided (7–1, 7–2, 7–3, 7–4 and 7–5). Then, 10 times cyclic bending testing was conducted at center of each sub-section. Each bending loading was exerted by putting the weight on the CFRPs for 2 s. Those weights were removed after bending. That procedure of bending experiment was repeated 10 times for all 80 sections. The number 800 of bending datasets for the 80 sections in the CFRP samples with three different levels is listed inTable. 2.

Electrode numbers (1–4) were assigned as shown inFig. 2(c).

The six pairs consisted of different combinations of two electrode numbers (pair 1–2, 2–3, 3–4, 4–1, 1–3, 2–4). Six different electrical resistance values were measured under bending deformation in six pairs. Initial electrical resistance was within 1–2 in and woven- shaped CFRPs. In shaped CFRP, the same electrical resistance range was measured along the carbon fiber direction. However, along the direction, around 10 of electrical resistance was measured in - shaped CFRP. The direct current electrical resistance was obtained by multichannel electrical resistance measurement during the cyclic bending test using a digital multimeter (Keithley 2002, USA) and a switching module (Keithley 7001, USA) which is used for simultaneous switching of the measuring pairs. The sensitivity of bending deformation is lower than 0.01 %, which is too small to identify the bending severity and location based on gradient changes of electrical resistance. This motivates the use of CNN

(3)

for damage identification in large areas of CFRPs. Multiple electrode sets are essential for damage localization in a large size of composites. A four-probe measurement method cannot be utilized

for real-time multichannel electrical resistance measurement because each electrode set requires an electrical ground and volt- age source. Thus, a two-probe measurement method is suitable for damage identification using multiple channels with the aim of real-time damage identification and localization.

2.4. Deep learning architecture

Machine learning is a subset of artificial intelligence. It can automatically learn with minimal human interference. Deep learn- Fig. 1.(a) Woven fabric CFRP and (b) size of unidirectional CFRP and unidirectional CFRP samples.

Fig. 2.Experimental setup of cyclic bending tests in (a) woven fabric CFRP and (b) unidirectional CFRP and unidirectional CFRP, and (c) schematic of damage localization section division.

Table 1

Cyclic bending conditions on CFRP.

Bending condition Deformation level 1 Deformation level 2 Deformation level 3

Woven 3 kg 4 kg 5 kg

3 kg 4 kg 5 kg

1.5 kg 2 kg 2.5 kg

Table 2

Number of electrical resistance change datasets collected in cyclic bending testing.

Deformation level 1 2 3

Woven 800 800 800

800 800 800

(4)

ing, which is a subset of machine learning, mimics the training process of the human brain using artificial neural networks[39–41].

Among deep learning algorithms, CNN is widely used for image recognition and analysis and is specifically designed to process pixel image data. Image datasets were used to train the CNN and prognosticate the possible labels to be assigned in the future.

CNN can automatically capture the important features in images without any additional human supervision. Furthermore, it is com- putationally efficient because of the convolutional and pooling layer in CNN.

The CNN architecture consists of multifunctional layers, such as convolutional, ReLU, batch normalization, max pooling, softmax, and regression layers. The convolution layer is utilized to extract data from the input using filters. The convolution kernel moves across the input matrix with a specific stride size, and the multi- plied values between the input and kernel are summed to form the output plus a bias. The weights and biases are optimized during deep learning. Meanwhile, ReLU is an activation layer that provides nonlinearity for the convolutional outputs. This activation layer has been reported to exhibit a higher performance than other activation functions because the gradient vanishing problem in other activation functions is totally removed. Thus, the accuracy of output prediction and algorithm efficiency can be maximized. Further- more, training speed is fast compared with other activation functions. Therefore, this activation layer has been widely used in several CNN architectures[26–30,38,39]. Moreover, batch normalization is a tool used to normalize the activation in intermedi-

ate layers in a CNN. It normalizes the output of the previous layers.

This step settles the learning process and drastically decreases the training epoch numbers which is required to train neural networks. Batch normalization is also used for regularization to avoid overfitting of the model. The pooling layer generates an output of reduced size. The max pooling layer chooses the maximum for each patch of the feature map using a pooling filter that is smaller than the feature maps. Therefore, it reduces the spatial size of the input images in abstracted form by extracting most important features. Furthermore, it can minimize the computational cost by reducing the number of parameters to learn, and it can remove the invariances such as shift, rotation, and scaling. Thus, max pooling layer is essential to CNN architectures. Moreover, the softmax function layer scales the input values to within 0 and 1, which can be expressed as probabilities. It is a generalization of the logis- tic function that can be utilized for multi-class classification. Then, a regression layer computes the half-mean-squared-error loss for regression tasks. Finally, the response of a trained regression network is predicted. Therefore, it is generally used for obtaining the exact prediction value. In this study, a softmax layer was used for section classification. A regression layer was used to calculate the coordinates of the damaged area.

All CNN architectures were performed with 0.01 learning weight, 100 epochs, and a training-to-validation ratio of 8:2.

Hyperparameters in CNN architectures were determined based on deep learning-based SHM literatures [26–30]. Example for pre-processing input image data was shown inFig. 3. Electrical

Fig. 3.Pre-processing of self-sensing input image data.

Fig. 4.Flowchart of proposed SHM methodology.

(5)

resistance was measured during cyclic bending testing. Therefore, time series comprising 10 times cyclic bending signals were pre- sent in one dataset. That data were divided into 10 images which were contained each loading state. Six different electrical resistance change ratio values were measured in six pairs, and the values were plotted within 0.006 to 0.006 range along the time axis.

The y-axis range is the minimum that can cover the maximum change of electrical resistance under the largest bending deformation. The x axial ranges from 40 to 43 is one time serial bending signal range among 10 times cyclic bending loading. Then, the labels in and y axis were removed. Lastly, black background was set in the graph images for efficient CNN architecture training.

Electrical resistance change images of cyclic bending tests at specific sections were used to train the damage level identification and damage localization process. The images of the unseen electrical resistance changes, which contain information regarding the unknown bending location, were used to predict the damage

severity and localization based on the trained CNN. The overall research flowchart is shown inFig. 4.

Four different shapes of CNN architecture were used for damage analysis.Fig. 5(a) shows CNN architecture that classify-three different damage levels as the outputs.Fig. 5(b), (c), and (d) show a CNN architecture with 80 classes, a hierarchical CNN, and a regression CNN, respectively, for damage localization. Many studies have localized damage using a regression CNN, which decides and pre- dicts the coordinates of the damage location with the architecture shown inFig. 5(d)[34–37]. The CNN with 3 classes was designed for severity identification of bending deformation. Therefore, this architecture classifies the input into three deformation levels. For the CNN with 80 classes, 80 locations of bending location were classified to localize the deformation. Meanwhile, in the hierarchical CNN architecture, 16 large CFRP sections were classified first, and then 5 sub-sections were classified within each large section to further localize the bending deformation. These three CNN

Fig. 5.CNN architecture for (a) damage level identification, (b) damage localization with 80 classes, (c) damage localization with hierarchical CNN architecture, and (d) damage localization with regression.

(6)

architectures utilized the softmax classifier to identify the bending severity and bending location. Additionally, the CNN with two coordinates directly calculated the, y coordinates of the bending location in CFRPs using the regression functional layer. The first three CNN architectures were compared with each other in terms of localization error. The best CNN architecture shape was proposed for an advanced SHM methodology with self-sensing data.

Table 3 summarizes the detailed architectures of the proposed and traditional regression CNNs.

3. Results and discussion

3.1. Damage localization and identification using CNN classification architecture

Electromechanical behavior image data were used for training to identify damage severity according to the damage level. A total of 2400 data points (three sets of 800 data points at each damage level) were used to classify the damage level in each CFRP specimen. In total, 7200 data points were used for damage severity identification in the three CFRPs. The confusion matrix for damage severity identification is shown in Fig. 6. The vertical axis shows the actual damage severity, and the horizontal axis shows the

labels assigned by the CNN architecture. The prediction was more than 88 % accurate for three different types of CFRPs. In other words, the damage level can be successfully investigated with high-accuracy classification. In addition, the damage level can be determined regardless of the damage location. The CFRP shape was manufactured and tested to compare deformation behavior with woven-shaped CFRP specimen. Unidirectional carbon fiber was stacked along the and direction to make similar carbon fiber alignment with woven CFRPs samples. Furthermore, the weights of the masses used for varying the deformation level were also identically set to compare each sample. However, has a slightly higher bending stiffness than woven-shaped CFRPs. Therefore, showed smaller deformation under the same stress conditions compared to woven-shaped CFRPs. It causes small changes in electrical resistance which may hinder severity classification. Thus, the bending severity classification showed a slightly lower accuracy of 88 %. However, this accuracy in is high enough for damage identification[26–30].

After determining the damage severity, the damage was localized on the CFRPs. A CNN with 80 classes and a hierarchical CNN were used for damage localization in woven CFRP using 2400 electrical resistance images. The confusion matrix results for the classification of damage localization according to damage level are

Table 3

Types of CNN architecture configurations applied in this study.

Type of CNN architecture Layer name Layer description

Input 3527 electrical resistance change ratio image

Convolutional layer Convolutional filter 33, strides 1

ReLU, Batch normalization

Pooling layer Max pooling filter 22, strides 1

CNN with 3 classes (Severity identification) CNN with 80 classes (Bending localization)

Convolutional 1, Pooling 1 Number of filters = 8

Convolutional 3 Number of filters = 32

Fully connected Softmax

Output = Three level of bending Bending location in 80 sections Hierarchical CNN

(Bending localization)

Fully connected Softmax

Output = bending location in 16

large sections bending location in 5 sub-sections CNN with two coordinates

(Bending localization)

Fully connected Regression layer

Output = x, y coordinates of bending location

Fig. 6.Confusion matrix for bending severity classification in (a) woven fabric CFRP, (b) unidirectional CFRP, and (c) unidirectional CFRP.

(7)

shown inFigs. 7–9. In addition, accuracy of classification for bending localization using two CNN architectures are shown inTable 4.

The vertical axis shows the actual damage location, and the hori-

zontal axis shows the labels assigned by each of the CNN architectures in the damage location. The CNN with 80 classes showed high classification accuracy in the three different cases of CFRPs. Fur-

Fig. 7.Confusion matrix for damage localization classification on woven fabric CFRP using a CNN with 80 classes under (a) bending deformation level 1, (b) level 2, and (c) level 3, and using a hierarchical CNN under (d) bending deformation level 1, (e) level 2, and (f) level 3.

Fig. 8.Confusion matrix for damage localization classification on unidirectional CFRP using a CNN with 80 classes under (a) bending deformation level 1, (b) level 2, and (c) level 3, and using a hierarchical CNN under (d) bending deformation level 1, (e) level 2, and (f) level 3.

(8)

thermore, damage location was correctly classified with an average classification accuracy of more than 80 %. However, the hierarchical CNN classified the damage location in the three different cased of CFRPs with around 50 % accuracy. CNN with 80 classes architecture showed better performance than hierarchical CNN architecture in bending deformation localization.

The representative historical curves of training for bending severity identification and localization along with accuracy and loss are shown inFig. 10. Loss of validation value converged after 30 epochs while that of training value also converged. It means that CNN architecture was in plateau, steady state without overfitting[42–46]. The accuracy converged at around 30 epochs, which is also shown by the decreasing loss results.

The damage severity and localization of the CFRPs were investigated using self-sensing data and various types of CNN architectures. The level and location of damage could be analyzed with high classification accuracy, especially using a CNN with 80 classes.

In other words, SHM was conducted with a CNN using self-sensing data.

3.2. Comparison of SHM result using different CNN architectures

The distance error was calculated during the damage localization process to optimize the CNN architecture for damage analysis in CFRPs. The localization error was measured as the distance

between the actual location of bending and the estimated location using the CNN architecture, as shown inFig. 11(a). The traditional regression CNN localization methodology that has been reported in many previous studies for damage localization[34–37], which cal- culates the Cartesian coordinates of the damage location using a regression CNN, was tested to compare the quantitative localization accuracy with the proposed CNN architectures (CNN with 80 classes and hierarchical CNN) in this study.

The two types of proposed CNN architectures showed smaller distance errors than the traditional CNN damage localization methodology, as shown inFig. 11(b), 11(c), and 11(d). In addition, the CNN architecture with 80 classes exhibited the highest damage localization performance among the three CNN architectures for all CFRPs. Furthermore, the damage was successfully localized at the three different damage levels. To compare the CNN architecture performance in greater detail, the average distance error of the damage localization was calculated, as shown inFig. 12. The damage location in CFRP was correctly estimated with a localization distance error of less than 50 mm using the CNN architecture with 80 classes that showed the best performance for damage localization.

2-coordinate CNN showed lowest damage location accuracy error because the output of 2-coordinate CNN regression is given by two coordinates of bending location. Therefore, the output can be any rational number. However, in the other two cases of CNN Fig. 9.Confusion matrix for damage localization classification on unidirectional CFRP using a CNN with 80 classes under (a) bending deformation level 1, (b) level 2, and (c) level 3, and using a hierarchical CNN under (d) bending deformation level 1, (e) level 2, and (f) level 3.

Table 4

Accuracy of classification for bending localization using two CNN architectures.

Woven CNN with 80 classes Hierarchical CNN

0.7122 0.7456 0.7598 0.4562 0.4125 0.4688

0.7758 0.8561 0.8641 0.5750 0.5250 0.5375

0.7243 0.8434 0.8176 0.5438 0.5625 0.6375

(9)

architectures, outputs were 80 classes which indicate the regions of loading location. Thus, 2-coordinate CNN regression has wider range of output compared with other two CNN architectures. Fur- thermore, electrical resistance change ratio images are not per- fectly identical even though the bending loading was exerted on same spot. It may more negatively affect the prediction accuracy

in case of wide range of outputs. The hierarchical CNN showed a lower damage location accuracy owing to the training procedure.

First, it classified the 16 sections in CFRPs and classified the five- sub sections in one selected large section among the 16 sections.

It means that if some data are misclassified in a large section, it causes large distance error. Therefore, this methodology has higher Fig. 10.Representative historical curves of training in (a) bending deformation severity identification, (b) bending localization by CNN with 80 classes.

(10)

risk for bending localization than the CNN with 80 classes. This resulted in a lower localization accuracy in the damage localization and confusion matrix in CFRPs, as shown inFigs. 11and6–8.

A CNN with 80 classes was determined to be the best architecture for damage localization using electrical resistance change data.

3.3. SHM in CFRPs using a CNN

The damage location was visualized using the CNN with 80 classes, which showed the highest localization performance on actual photos of CFRPs, to verify the proposed SHM system.

A representative visualization of the damage location images is shown inFig. 13. In these images, the damage was successfully localized in CFRP samples. In previous studies on self-sensing methodologies[15–17], damage localization outside the electrode set area has not been reported. However, when failure occurs outside the area of the electrodes in the CFRPs, it can be monitored in real time using the proposed method. Thus, it overcomes the limitation of electrode distance for SHM.

An unseen electrical resistance input image was used to determine the damage localization procedure using an untrained bending location. The damage location was predicted using the trained Fig. 11.(a) Schematic of distance error calculation, and localization error according to damage level in (b) woven CFRP, (c) CFRP, and (d) CFRP.

Fig. 12.Average localization distance error according to CNN architecture in (a) woven CFRP, (b) CFRP, and (c) CFRP.

(11)

CNN architectures. Damage level 2, which is 2 kg of weight, was applied to the new area in CFRPs, and three trained CNN architectures were used to predict the damage level and area. The damage level was predicted to be level 2 based on the unseen input data.

The CNN with 80 classes predicted the damage points near the actual bending points. When the diameter of the weights (pressed zone) was considered, the predicted damage location was in the actual damaged area as shown inFig. 14(a). Therefore, a CNN with 80 classes was successfully used to predict the untrained failure in real time. However, the other two CNN methodologies inFig. 14(b) and (c) (hierarchical CNN and CNN with two coordinates) showed poor damage localization accuracy for untrained electrical resistance input image data. All bending deformation prediction was conducted within few seconds.

An efficient real-time SHM methodology for CFRPs was proposed and validated as having high performance. Electrical resistance in case of damage is changed similarly or more than that under bending loading. Damage in composite structures usually occurred with plastic deformation which may result in electrical network break- age. It causes slightly larger gradient of electrical resistance change in case of damage than that of bending loading. Therefore, this advanced SHM technique is also applicable in damage conditions in composites with no monitoring sensitivity issues. The damage severity was identified, and damage localization was investigated using CNN architectures. The proposed CNN for SHM analysis overcomes the electrode distance limitation of previous self-sensing NDE methods. The decreased number of electrodes reduces the data complexity for real-time SHM, while maintaining high accuracy in Fig. 13.(a)–(h) Representative image of damage localization in actual CFRP photos.

Fig. 14.Prediction result of bending localization for bending level 2 using (a) CNN with 80 classes, (b) hierarchical CNN, and (c) CNN with two coordinates.

(12)

damage localization. Furthermore, the proposed method makes it possible to monitor a large area of CFRPs, regardless of the carbon fiber type and stacking sequence. Therefore, it has high potential applicability to composite structures in various industrial fields.

4. Conclusions

Cyclic bending tests were performed at three damage levels in three different types of CFRPs. Real-time electrical resistance was used for SHM of the CFRPs. These images were used to train CNN architectures. Four types of CNNs were designed to identify damage.

The severity of bending was identified using a CNN with more than 85 % classification accuracy for all CFRPs. Confusion matrices were investigated to determine the localization accuracy according to the type of CNN architecture. The CNN architecture with 80 classes showed better performance for damage localization in CFRPs than the hierarchical CNN. The localization error was calculated to compare the damage localization performance of the traditional and proposed CNN architectures. The traditional CNN (two coordinates) exhibited errors of more than 150 mm for localization. However, the CNN with 80 classes, which showed the best performance for localization, localized the damage within a 50 mm error, which was within the pressed zone of the weights.

The damaged area was successfully identified using the CNN with 80 classes regardless of the type of carbon fiber in the CFRPs. Fur- thermore, the unseen damage was successfully monitored using the trained CNN architecture with 80 classes.

The proposed real-time structural monitoring system improves upon the existing self-sensing NDE methodologies. The data complexity owing to the large number of required electrode sets in the existing self-sensing was reduced owing to the requirement of only a minimum number of electrodes. In addition, the electrode distance for damage analysis is significantly increased in self- sensing techniques. Furthermore, the health state of a large area in CFRPs can be monitored efficiently in real time with high accuracy. Therefore, the advanced monitoring system proposed in this study is more practical than previous self-sensing systems. With this self-monitoring system, which is applicable to CFRP structures, such as wind blades, aircraft, and civil infrastructure, system safety can be ensured.

Data availability

The data used in this research are available from the corresponding author upon reasonable request.

Declaration of Competing Interest

The authors declare that they have no known competing finan- cial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This research was supported by the Basic Science Research Pro- gram (Mid-career Research Program, Grant No. NRF- 2021R1A2C2009726) through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT of Korea.

Appendix A. Supplementary material

Supplementary data to this article can be found online at https://doi.org/10.1016/j.matdes.2022.111348.

References

[1]M. Kobayashi, C. Jen, Piezoelectric thick bismuth titanate/lead zirconate titanate composite film transducers for smart NDE of metals, Smart Mater.

Struct. 13 (4) (2004) 951.

[2]X. Qing, X. Liu, J. Zhu, Y. Wang, In-situ monitoring of liquid composite molding process using piezoelectric sensor network, Struct. Health Monit.

1475921720958082 (2020).

[3]S.Y. Kim, T. Tanimoto, K. Uchino, C.H. Nam, S. Nam, W.I. Lee, Effects of PZT particle-enhanced ply interfaces on the vibration damping behavior of CFRP composites, Compos. A Appl. Sci. Manuf. 42 (10) (2011) 1477–1482.

[4]Y. Okabe, S. Yashiro, T. Kosaka, N. Takeda, Detection of transverse cracks in CFRP composites using embedded fiber Bragg grating sensors, Smart Mater.

Struct. 9 (6) (2000) 832.

[5]H. Tsuda, Ultrasound and damage detection in CFRP using fiber Bragg grating sensors, Compos. Sci. Technol. 66 (5) (2006) 676–683.

[6]S. Lu, M. Jiang, Q. Sui, Y. Sai, L. Jia, Damage identification system of CFRP using fiber Bragg grating sensors, Compos. Struct. 125 (2015) 400–406.

[7]C. Barile, Innovative mechanical characterization of CFRP by using acoustic emission technique, Eng. Fract. Mech. 210 (2019) 414–421.

[8]V. Munoz, B. Valès, M. Perrin, M.-L. Pastor, H. Welemane, A. Cantarel, M.

Karama, Damage detection in CFRP by coupling acoustic emission and infrared thermography, Compos. B Eng. 85 (2016) 68–75.

[9]R. Unnthorsson, T.P. Runarsson, M.T. Jonsson, Acoustic emission based fatigue failure criterion for CFRP, Int. J. Fatigue 30 (1) (2008) 11–20.

[10]S. Nauman, Piezoresistive sensing approaches for structural health monitoring of polymer composites—A review, Eng 2 (2) (2021) 197–226.

[11]H.D. Roh, H. Lee, Y.-B. Park, Structural health monitoring of carbon-material- reinforced polymers using electrical resistance measurement, Int. J. Precis.

Eng. Manuf.-Green Technol. 3 (3) (2016) 311–321.

[12]Y. Wang, Y. Wang, B. Wan, B. Han, G. Cai, Z. Li, Properties and mechanisms of self-sensing carbon nanofibers/epoxy composites for structural health monitoring, Compos. Struct. 200 (2018) 669–678.

[13]A. Martins, Z. Aboura, W. Harizi, A. Laksimi, K. Hamdi, Structural health monitoring by the piezoresistive response of tufted reinforcements in sandwich composite panels, Compos. Struct. 210 (2019) 109–117.

[14]A. Alsaadi, J. Meredith, T. Swait, J. Curiel-Sosa, Y. Jia, S. Hayes, Structural health monitoring for woven fabric CFRP laminates, Compos. B Eng. 174 (2019) 107048.

[15]I.Y. Lee, H.D. Roh, Y.-B. Park, Novel structural health monitoring method for CFRPs using electrical resistance based probabilistic sensing cloud, Compos.

Sci. Technol. 213 (2021) 108812.

[16]H.D. Roh, S.Y. Oh, Y.-B. Park, Self-sensing impact damage in and non- destructive evaluation of carbon fiber-reinforced polymers using electrical resistance and the corresponding electrical route models, Sens. Actuators, A 332 (2021) 112762.

[17]D. Zhang, B. Yang, J. Tan, Y. Jin, B. Xiao, G. Xian, X. Xue, Y. Li, Impact damage localization and mode identification of CFRPs panels using an electric resistance change method, Compos. Struct. 276 (2021) 114587.

[18]L. Groo, J. Nasser, D. Inman, H. Sodano, Damage localization in fiberglass- reinforced composites using laser induced graphene, Smart Mater. Struct. 30 (3) (2021) 035006.

[19]C. Yang, Y. Kim, S. Ryu, G.X. Gu, Prediction of composite microstructure stress- strain curves using convolutional neural networks, Mater. Des. 189 (2020) 108509.

[20]S. Ye, B. Li, Q. Li, H.-P. Zhao, X.-Q. Feng, Deep neural network method for predicting the mechanical properties of composites, Appl. Phys. Lett. 115 (16) (2019) 161901.

[21]C. Rao, Y. Liu, Three-dimensional convolutional neural network (3D-CNN) for heterogeneous material homogenization, Comput. Mater. Sci 184 (2020) 109850.

[22]C.W. Park, C. Wolverton, Developing an improved crystal graph convolutional neural network framework for accelerated materials discovery, Phys. Rev.

Mater. 4 (6) (2020) 063801.

[23]P. Davis, F. Aziz, M.T. Newaz, W. Sher, L. Simon, The classification of construction waste material using a deep convolutional neural network, Autom. Constr. 122 (2021) 103481.

[24]F. Sultana, A. Sufian, P. Dutta, Advancements in image classification using convolutional neural network, in: 2018 Fourth international Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), IEEE, 2018, pp. 122–129.

[25]N. Sharma, V. Jain, A. Mishra, An analysis of convolutional neural networks for image classification, Proc. Comput. Sci. 132 (2018) 377–384.

[26]C. Barile, C. Casavola, G. Pappalettera, V.P. Kannan, Damage monitoring of carbon fibre reinforced polymer composites using acoustic emission technique and deep learning, Compos. Struct. 292 (2022) 115629.

[27]S. Sikdar, D. Liu, A. Kundu, Acoustic emission data based deep learning approach for classification and detection of damage-sources in a composite panel, Compos. Part B: Eng. 228 (2022) 109450.

[28]M. Lin, S. Guo, S. He, W. Li, D. Yang, Structure health monitoring of a composite wing based on flight load and strain data using deep learning method, Compos.

Struct. 286 (2022) 115305.

[29]A. Khan, D.-K. Ko, S.C. Lim, H.S. Kim, Structural vibration-based classification and prediction of delamination in smart composite laminates using deep learning neural network, Compos. Part B: Eng. 161 (2019) 586–594.

(13)

[30] A. Khan, J.K. Shin, W.C. Lim, N.Y. Kim, H.S. Kim, A deep learning framework for vibration-based assessment of delamination in smart composite laminates, Sensors 20 (8) (2020) 2335.

[31]T.-Y. Kim, S.-B. Cho, Predicting residential energy consumption using CNN- LSTM neural networks, Energy 182 (2019) 72–81.

[32]I.E. Livieris, E. Pintelas, P. Pintelas, A CNN–LSTM model for gold price time- series forecasting, Neural Comput. Appl. 32 (23) (2020) 17351–17360.

[33]A.U. Rehman, A.K. Malik, B. Raza, W. Ali, A hybrid CNN-LSTM model for improving accuracy of movie reviews sentiment analysis, Multimedia Tools Appl. 78 (18) (2019) 26597–26613.

[34]S. Cofre-Martel, P. Kobrich, E. Lopez Droguett, V. Meruane, Deep convolutional neural network-based structural damage localization and quantification using transmissibility data, Shock Vib. (2019).

[35]M. Rautela, S. Gopalakrishnan, Ultrasonic guided wave based structural damage detection and localization using model assisted convolutional and recurrent neural networks, Exp. Syst. Appl. 167 (2021) 114189.

[36]S. Zhang, C.M. Li, J. Yang, W. Ye, Effective combination of modeling and experimental data with deep metric learning for guided wave-based damage localization in plates, Mech. Syst. Signal Process. 172 (2022) 108979.

[37]S. Zhang, C.M. Li, W. Ye, Damage localization in plate-like structures using time-varying feature and one-dimensional convolutional neural network, Mech. Syst. Signal Process. 147 (2021) 107107.

[38] T. Szandała, Review and comparison of commonly used activation functions for deep neural networks, Bio-inspired neurocomputing, Springer 2021, pp.

203–224.

[39]H. Ide, T. Kurita, Improvement of learning for CNN with ReLU activation by sparse regularization, in: 2017 International Joint Conference on Neural Networks (IJCNN), 2017, pp. 2684–2691.

[40]B. Mahesh, Machine learning algorithms-a review, Int. J. Sci. Res. (IJSR).

[Internet] 9 (2020) 381–386.

[41]N. Sharma, R. Sharma, N. Jindal, Machine learning and deep learning applications-a vision, Glob. Trans. Proc. 2 (1) (2021) 24–28.

[42]A. Yamanaka, R. Kamijyo, K. Koenuma, I. Watanabe, T. Kuwabara, Deep neural network approach to estimate biaxial stress-strain curves of sheet metals, Mater. Des. 195 (2020) 108970.

[43]W. Tian, X. Cheng, Q. Liu, C. Yu, F. Gao, Y. Chi, Meso-structure segmentation of concrete CT image based on mask and regional convolution neural network, Mater. Des 208 (2021) 109919.

[44]A.P. Garland, B.C. White, S.C. Jensen, B.L. Boyce, Pragmatic generative optimization of novel structural lattice metamaterials with machine learning, Mater. Des 203 (2021) 109632.

[45]P. Hao, D. Liu, K. Zhang, Y. Yuan, B. Wang, G. Li, X. Zhang, Intelligent layout design of curvilinearly stiffened panels via deep learning-based method, Mater. Des 197 (2021) 109180.

[46]B. Shi, Z. Chen, A layer-wise multi-defect detection system for powder bed monitoring: lighting strategy for imaging, adaptive segmentation and classification, Mater. Des 210 (2021) 110035.