In Chapter 2, we proposed a deep learning-based hardness prediction model in laser surface hardening (heat treatment) of AISI H13 tool steel, from an input of cross-sectional temperature distribution (the first model of deep learning in laser hardening). In Chapter 3, we studied a cross-sectional weld bead image prediction in laser keyhole welding of AISI 1020 steel, using state-of-the-art deep learning algorithms (the first deep learning model in laser weld bead image prediction). We expect the presented deep learning framework to play a leading role in future advanced modeling of laser keyhole welding.
In Chapter 4, we presented a deep learning framework for predicting the shape of cross-sections in the self-drilling rivet (SPR) joining process (the first deep learning model in SPR).
Introduction
Introduction of Modern Materials Processing
- General Introduction of Modern Materials Processing
- Laser Materials Processing and Laser Heat Treatment
- Laser Keyhole Welding
- Self-Piercing Riveting (SPR)
In Figure 1.6(a), power densities with corresponding bead shapes are plotted according to typical welding heat sources (laser, electron beam, plasma, and arc) [17]. In Figure 1.7 (a), the top and bottom coaxially observed welding surfaces during the process are shown [18], and a 3-D schematic of laser keyhole welding is shown in Figure 1.7 (b) . A schematic of the SPR process is shown in Figure 1.9(a) [27]: 1) clamping; 2) upper punch sheet, driven by a punch; . 3) the top sheet complete piercing and turning the legs of the spikes on the bottom sheet; 4) release.
Figure 1.9(b) shows two types of SPR joints, namely carbon fiber reinforced plastic (CFRP) – galvanized steel (left) and steel alloy – aluminum alloy (right). b) SPR – CFRP/galvanized steel (left) and steel alloy/aluminum alloy (right) joints.
Introduction of Deep Learning
- General Introduction of Artificial Intelligence and Deep Learning
- Image Recognition
- Image Generation
Deep Learning Study on Laser Heat Treatment
- Introduction
- Data Preparation
- Deep Learning Model
- Results and Discussion
- Conclusion
Process conditions shown in red and blue squares were used as a training dataset and a test dataset, respectively. This result applies to data set h, and for brevity only the first channel (C = 1) is shown.
Deep Learning Study on Laser Keyhole Welding
- Introduction
- Data Preparation
- Deep Learning Model
- Results and Discussion
- Conclusion
The first and second columns are input normalized laser intensity and interaction time maps (c), respectively. The third and fourth columns are prediction results for the first generator (G1(c)) and their ground truths (x), respectively. The fifth and sixth columns present predicted OM images (G2(G1(c))) and ground truths (OM), respectively.
In the first generator, encoding progresses in the first row (from left to right) and decoding is performed in the second row (from right to left).
Deep Learning Study on Self-Piercing Riveting
Introduction
Self-piercing riveting (SPR) is one of the most promising techniques for joining sheet materials. To successfully perform the technique, the actual application of SPR must be preceded by process simulation because the cross-sectional shape behind the spike determines the quality of the joint [26]. 93] modeled a physical joint failure mechanism in the SPR process for aluminum alloy by solving the governing equations using FEM and experimentally validating their prediction results.
95] carried out numerical simulation of SPR of aluminum alloy and steel sheet based on FEM with different fracture criteria and experimentally verified the model in terms of deformed shape, force-displacement curve and residual stress profile. FEM simulation of SPR of aluminum alloys and three joint shapes were predicted and compared with experimental results. 99] proposed a rivet stretching model to predict the amount of rivet deflection from force-displacement curves and plate, die, and rivet geometries.
The simulation was based on a damage model to consider the deformation and failure behavior of the composites. In this study, a novel deep learning framework was presented to predict the cross-sectional shape of the SPR joint based on the scalar input impact force. Using the proposed deep learning architecture, we were able to obtain cross-sectional shapes, including the location and deformed shape of the rivet and plates, which determine the quality of the SPR connection.
Note that our segmentation model can segment the SPR OM images regardless of the material combination types (ie, one trained model can segment both CFRP-GA590DP and SPFC590DP-Al5052 combinations with the given accuracies). Similarly, a state-of-the-art deep learning model for image segmentation was introduced along with appropriate hyperparameters.
Data Preparation
The tensile strength, thickness, coating, and applied punching forces are listed in Table 4.1 with the rivet and workpiece holder properties, and the geometric shape and dimension of the rivet and die are shown in Figure 4.1(a) (CFRP-GA590DP) and Figure 4.1(b) (SPFC590DP- Al5052). As shown in both figures, the die (supplied by BÖ LLHOFF) had a basic flat bottom shape. SPR joints were produced by a hydraulic riveter (Rivset Gen2, BÖ LLHOFF) with a maximum setting force of 78 kN.
Note that the punching speed was not varied (it was almost 0.13 m/s) as the riveting machine was hydraulic type. To assess the prediction quality, three geometric indexes of the SPR joint were measured, namely the head height (distance between the head surface of the rivet and the top surface of the top plate), the interlock width (distance between the tip of the deformed rivet shank and the pierced point of the top plate) and the thickness of the bottom (remaining thickness of the bottom plate after riveting), as shown in Figure 4.1(d). All geometric indices were measured twice for each OM image (left and right) and then averaged.
Deep Learning Model
A schematic architecture of the model is presented in Figure 4.4; the figure legend shows the six types of control blocks. Presented segmentation map and spatial dimensions of the inner layers are for the material combination type CFRP-GA590DP. Four types of convolution blocks are shown in Figure 4.6, and the numbers under each block indicate the shape of the convoluted images (H, W, C; for brevity, only the case of the CFRP-GA590DP was shown).
In this study, based on the results of the previous studies that adding a conventional loss function to the GAN loss greatly improved the generation quality, a case study for the objective function was conducted, considering that data type is the segmentation map. Accordingly, the last activation function of the generator is also included in the case study and its results are shown in Figure 4.7. Further details of the GAN loss can be referred from [36, 52] and LSGAN papers [89] (not presented here for the sake of brevity).
However, when the generator trained with the L1 loss (second column in Figure 4.7), it created significantly better results, and the final activation of the softmax layer showed much better quality than that of tanh (sharper edges without blurring; mIOU = 92.7%). which is actually an obvious result since the data type was the segmentation map (0 or 1 is assigned in each material class). With the CEE (third column in Figure 4.7), the generation result with the softmax layer was significantly better (mIOU=92.8%) than with the tanh layer, which showed an underdeveloped generation. Note that the area encompassing the predicted values was separated into “positive” and “negative”; the area that included the correct predictions was marked as 'true' and the area that included the incorrect predictions was marked as 'false'.
In Figure 4.9, the mIOU-epoch curves for the training data set (black line) and the validation data set (blue line) were presented, for two consecutive training sessions (output span = 16 at epoch 0 ~99 and 8 in the era 100~ 199). Validation accuracy peaked at epoch, so the model parameters at this epoch were used for model testing (i.e. the model was validated at epoch 157).
Results and Discussion
As shown in the figures, material segmentation was successfully performed for all test sets. After ensuring material segmentation with 98–99% accuracy, the training data to be used in the predictive cross-section model was safely replaced with the segmentation map instead of the OM image. The cross-sectional predictive model can now be further trained in the future by simply feeding new OM images into the deep learning segmentation model.
Furthermore, if other combinations of SPR joint sheets are added in the future, the trained segmentation model can easily be further trained by recovering the parameters of the validated model. As can be seen from the predicted results in Figure 4.13, the higher the impact force, the more the rivet is pressed and the longer the locking length. Prediction (three columns on the left; three latent variables) and ground truth (three columns on the right; three replicates) results for CFRP-GA590DP are presented for each impact force.
Prediction (three columns on the left; three latent variables) and ground truth (two columns on the right; two replicates) results for the SPFC590DP-Al5052 are presented for each impact force. Calculated accuracy of head height, lock and bottom thickness against punching force for CFRP-GA590DP panels. Calculated accuracy of head height, lock and bottom thickness versus punching force for SPFC590DP-Al5052 plates.
The presented results show that the deep learning architecture can be very useful in predicting the deformed cross-section shape in the SPR process. These forces were used in the experiments and provided as input to the deep learning model as scalar values.
Conclusion
Summary and Future Perspectives
Summary
Catanzaro, “High-resolution image synthesis and semantic manipulation with conditional gans,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. Ledig et al., “Photorealistic single-image super-resolution using a generative adversarial network,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. Lempitsky, "Few-shot adversarial learning of realistic neural talking head models," in Proceedings of the IEEE International Conference on Computer Vision, 2019, pp.
Sun, "Deep residueel leren voor beeldherkenning", in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. Mu Lee, "Enhanced Deep Residual Networks for Single Image Super-Resolutie", in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp. Fu, "Residual density network for image super-resolutie", in Proceedings of the IEEE conference on computer vision and patroonherkenning, 2018, pp.
Zhang, “Second-order attention network for single-image super-resolution,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. Paul Smolley, “Least squares generative adversarial networks,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” in Proceedings of the European conference on computer vision (ECCV), 2018, pp.
Chollet, "Xception: Deep learning with depthwise separable convolutions", in Proceedings of the IEEE conference on computer vision and patroonherkenning, 2017, pp. Efros, "Context encoders: Feature learning by inpainting", in Proceedings of the IEEE conference on computer visie en patroonherkenning, 2016, pp.
