Application of machine-learning techniques for characteristic analysis of refractory materials
Item Type Conference Paper
Authors Ijaz, Sumbel;Noureen, Sadia;Rehman, Bacha;Zubair,
Muhammad;Mehmood, Muhammad Qasim;Massoud, Yehia Mahmoud
Citation Ijaz, S., Noureen, S., Rehman, B., Zubair, M., Mehmood, M. Q., &
Massoud, Y. (2022). Application of machine-learning techniques for characteristic analysis of refractory materials. Photonics for Energy II. https://doi.org/10.1117/12.2643946
Eprint version Post-print
DOI 10.1117/12.2643946
Publisher SPIE
Rights This is an accepted manuscript version of a paper before final publisher editing and formatting. Archived with thanks to SPIE.
Download date 2023-12-06 19:43:40
Link to Item http://hdl.handle.net/10754/686574
Application of machine-learning techniques for characteristic analysis of refractory materials
Sumbel Ijaz
a, Sadia Noureen
a, Bacha Rehman
b, Muhammad Zubair
a, Muhammad Qasim Mehmood
a*, Yehia Massoud
c*a
MicroNano Lab, Electrical Engineering Department, Information Technology University (ITU) of the Punjab, Ferozepur Road, 54600 Lahore, Pakistan;
bNamal University, Mianwali, Pakistan Mianwali, Pakistan;
cInnovative Technologies Laboratories King Abdullah University of Science
and Technology Saudia Arabia
ABSTRACT
Flat optics have become capable of achieving unprecedented functionalities through electromagnetic (EM) wave manipulation by employing the metasurfaces. The most crucial part in the design of metasurface is the selection the constitutive component i.e. the meta-atom’s material and structure so that it exhibits the precise operation as per the desired application. The unit-cell design calls for an iterative loop of simulations in order to explore the EM responses for intended operation. In this work, we have studied the absorption response of refractory materials under visible light radiations for their utilization in energy harvesting applications. The absorption response estimation using machine-learning techniques for the materials having very high melting-points, mechanical stabilities and inertness to the atmosphere has been carried out to investigate their performance in the broadband range. The presented regression models incorporate hybrid data format i.e. they simultaneously contend with 3-D and 1-D properties of various shapes of nano-resonators. The images’
feature extraction is carried out by employing Singular Value Decomposition. The trained models are potent enough to bypass the repetitive sequence of optimization involved in conventional EM solvers. Additionally, the models are capable of predicting the optimum shape along with structural dimensions of unit-cell. For forward model, the MSEs for training and testing are 1.302×10-2 and 3.269×10-2 while R2 scores are 0.9804 and 0.8764, respectively. The approach applied is so robust that, irrespective of complexity of unit-cell structure is, it serves the purpose of predicting the distinct structure with highest performance while bypassing the time- and computationally-intensive EM simulations.
Keywords: Metasurface, Refractory metals, Random-Forest Regression Algorithm, Lasso Regression, Mean Square Error, R2 score, Singular Value Decomposition (SVD)
1. INTRODUCTION
Metasurface based designs have shown the potential for shaping the optical wavefronts by featuring only compact footprint in contrast to the bulky geometric optics. The designs become capable of providing the control by engineering the geometry and size of meta-atom1–3. The design of meta-atoms is of key importance because it is a fundamental building blocks in the design of metasurfaces and has a significant role in achieving the desired optical characteristics. Their design methods are based on trial-and-error methods to accomplish the target electromagnetic responses. As there exists a countless possibilities of designs differing from each other in their physical and geometric parameters, the need of a massive computational resource arises to conclude on a desired response. Unlike conventional iterative optimization design methods including EM simulation software based on full-wave numerical calculations (e.g., Finite Element Method (FEM)
4, Method of Moments (MOM) 5, Finite Integration Technique (FIT) 6 and Finite-Difference Time-Domain (FDTD)) 7, the machine-learning (ML) based models are fast in figuring out the response. These methods are employed extensively in various disciplines of science these days 8 in order to provide an assistance to the existing traditional design methods. They intend to deal with the complex problems, because ML-based regression models have been becoming dominant in the regression of large sample data 9. It becomes time-friendly and memory conducive for computational equipment when data-driven approaches are brought in. Many of the machine learning methods such as decision tree regression, k nearest [email protected], [email protected]
neighbor, polynomial regression and support vector machines have been employed for predicting the EM spectrum to substitute numerical simulations based on finite difference time domain methods (FTDT) 10,11.
In this paper, machine learning-based metasurface/meta-atom modeling approaches have been demonstrated with a view to reduce the characterization time while maintaining the accuracy. Here, we have proposed regression techniques including Random Forest (RF) and LASSO ((Least Absolute Shrinkage and Selection Operator) Regression (LR) to mitigate difficulties faced during conventional design procedure of metasurfaces. RF is one of the machine learning algorithms which is powerful data driven approach. It is an ensemble-based regression method commonly employed in statistical analyses12. The relationship between features and the target is in the form of a tree structure as this method is a decision tree adaptation, in which a model produces predictions based on a succession of base models. RF works on combining several randomized decision trees and aggregates the predictions by averaging13. Furthermore, it is handy enough to be functional for large-scale problems and has shown a higher stability and robustness with varying training parameters14. Many of the regression methods, however, tend to over-fit and over-estimate in terms of the model variables.
LASSO regression model proves to be an attractive alternative to addresses both these matters15. It is designed to distinguish the variables and corresponding coefficients of the model with an aim to minimizing the prediction error. First, a restriction is put on the model parameters, forcing the regression coefficients to become zero and are excluded from the model, thereby making the sum of the absolute value of the regression coefficients lesser than a particular constant ‘λ’.
This regression model has outperformed standard methods in some scenarios. However, the demerit associated with this method is that the regression coefficients may no longer remain reliably interpretable in terms of risk factors leading to a lower accuracy15.
In this work, we have parameterized the metasurface structure (i.e., hdielectric, hpattern and period), and their images to obtain a sufficient simulation data (absorption spectrum) for training purpose using CST®. After obtaining the dataset having large variance the models are trained in forward configuration to predict the EM responses (broadband absorption) of specific meta-atom in matter of seconds. In the same way, the inverse problem is also solved to find the physical parameters of device given a desired electromagnetic response. The image data for meta-atoms is originally a high dimensional space comprising of three channels in 128 × 128 pixels format. Before using the data for training purpose, we have used Singular Value Decomposition (SVD) to reduce every channel to 20 pixels data to continue with only significant components carried in data. SVD is a data dimensionality reduction tool used when the data is exceptionally high dimensional, such as for those of high resolution images. SVD reduces the data to only key features needed to analyzing and describing. The technique tailors the coordinate system driven by source data after identifying the prevailing correlations. SVD decomposes original space ‘X’ into three simple transformations in a row i.e. an initial rotation V, a scaling along the coordinate axes, and a final rotation U to approximate ‘X’ as shown in Figure 1.
Figure 1. The SVD decomposition of original space using three transformations i.e., an initial rotation through V, a scaling along the coordinate axes, and a final rotation through U.
Meta-atom design
For designing a meta-absorber, the shape, size and the material of a meta-atoms plays a key role. The optical response of a material is characterized by dielectric permittivity ‘ϵ’and magnetic permeability ‘μ’16, whereas the magnetic interaction is not significant for the visible range wavelengths for natural materials. However, the electric permittivity bears importance in determining the response of a particular design and the imaginary part of dielectric function i.e. the extinction coefficient is responsible for the absorption. The materials employed in this work possess higher values of absorption coefficient and have exhibited high absorption in the desired range i.e. 250 – 1000 THz (300 to 1200 nm). In this paper, we have used five different subwavelength shapes including “circular ring”, “triangle”, “hexagon”, “T shaped” and “square supercell” meta-atoms made up of “Niobium (Nb)”, “Vanadium (V)” and “Rhenium (Re)” (Figure 2) for obtaining broadband absorption response. The goal of this work is to design a broadband absorber for Solar ThermoPhotoVoltaics (STPV) applications. The meta-atom assumes metal-insulator-metal structure where the insulator material used is SiO2. STPV technology is a modern concept of energy conversion which aids in the solar power applications and whose efficiency depends on absorber’s performance17,18. These applications employ materials having high melting points >
2000°C and high chemical stability19. Noble metals, characterized with their low melting point and softness, are incapable of forbearing such high temperatures for real-world applications. STPV technology is characterized by absorption of solar irradiation using a broadband absorber, which causes heat in the intermediate component and the subsequent emission of this thermal energy in a narrow spectrum is integral to its performance for efficient light absorption by the photovoltaic cell. Such devices can theoretically achieve energy conversion efficiencies up to 85%18. High-temperature stability is most desired feature for devising STPV systems. Refractory metals have been replacing noble metals as suitable replacements because of their good plasmonic behavior enabling the realization of exciting devices with even enhanced efficiencies19. Along with the desirable optical performance, these materials possess complementary metal-oxide semiconductor compatibility and chemical stability. They have proven to be the alternatives for most of the major limitations associated with noble metals, thereby advancing the existing STPV technology20. All of the proposed materials have been proven to be significantly good absorbers as can be seen from Figure 3 where the maximum absorption values for each of the shapes are found to be > 99%.
Figure 2. The meta-atom shapes and materials used for design of broadband absorber
Figure 3. The peak values of absorption for respective materials using different shapes in order for Nb, Re and V
For the proposed architectures, we have used the hybrid format of training dataset consisting of three structural parameters and 3D image data of shapes the meta-atoms. The input feature vector comprises of 63 points in case of forward modelling with 500 absorption values over the specified range. The input vector is a concatenation of three (3) structural values with sixty (60) pixel values obtained from 3D data i.e. the meta-atom structure images using Singular Value Decomposition (SVD). The geometrical parameters include heights of dielectric (hd) and that of the pattern of nano-resonator layers (hp) and the period (p) of the unit-cell. The other defining parameters of each shape are made dependent on the value of ‘p’ so that the pattern varies during the phase of data collection. The range of parameters p ϵ [200, 500 nm], hd ϵ [40, 80 nm] and hp ϵ [20, 60 nm] while the ground height has been kept as 150 nm in each case. The step size of 25 nm for “p”, and of 5 nm in case of both “hd” and “hp” has been used during the course of dataset collection. The total number of input vectors thus becoming 5×5×13 = 325 which are based on input geometry values for individual shape. For the five shapes used, the total number of vectors become 5 × 325 = 1625. In the original format, the shape of images is taken as a three channel input with vector size of (128, 128, 3). The shape of image against each channel is converted to 20 pixels by applying SVD for red, green, and blue, thereby generating a vector of size (1,60). Thus the problem becomes 63×500 dimensional for forward topology in case of each material. The output feature vector i.e. the absorption spectrum consists of 500 samples generated using CST® simulations for a single instance. The 80% of the dataset is used for training while rest of the 20%
has been left as test set to validate the training effect. The overall schematic of the proposed idea is given in Figure 4, where input feature vector for forward case, the ground truth and the predicted absorption for Random Forest and Lasso regression are included showing the best fit for RF model with a less accurate prediction for LR. In the same way, the output for inverse case is shown to be three geometrical parameters with 60 image pixels in the form of a vector.
Figure 4. The machine-learning based model diagram for spectrum prediction in forward and geometrical parameters and image pixel values prediction in inverse configurations. Random forest and Lasso regressor models based predicted curves versus simulated ones.
The dataset generation, though, is a time taking procedure but it is one-time and the trained regression model is fast enough for new structure parameters saving time as well as the computational cost of the design process. An added advantage of such trained models is their reusability. It has been shown that random forest has performed exceptionally well in terms accuracy while LR has responded very fast. The training time for random forest is as high as 5 minutes 57 secs in forward model with minimum MSE of 1.302 × 10-2 and maximum R2 score of 0.982 while the Lasso consumes minimum of 1 sec.
The devised methods are gainful in terms of computational power and thereby no longer remaining computer-intensive in nature. These methods perform ultrafast when compared with numerical simulations. The simulations take so much of time and memory because of their being the brute force approach which solves the Maxwell equations on case by case basis.
2. RESULTS AND DISCUSSIONS
Forward machine learning architecture
Before collecting dataset, the fact was established that the proposed meta-atom structures have a high amplitude of absorption for the materials under investigation, for the given range of parameters with different shape. The machine learning models i.e. a tree-based regression algorithm i.e. random forest (RF) model and Lasso regression model have been used. The predictions against simulated responses are included in Figure 5 (a-c) for RF model and (d-f) for LR model. It can be seen that RF model have been working relatively better than that of LR model. The results are included for hexagon of Nb with geometrical features as [40, 30, 300 nm], triangle of Re with [80, 60, 500 nm], triangle of V with [80, 60, 500 nm], circular-ring of Nb with [40, 20, 225 nm], T-shape of Re with [40, 30, 500 nm] and square super cell of V with [40, 20, 475 nm]. The performance of the regression models is measured using “mean squared error (MSE)” and R2 scores. For forward model minimum of training MSE and maximum of R2 score are 1.302×10-2 and 0.982 while for testing data, the MSE is 3.269×10-2 and R2 scores is 0.8764 in case of RF model by setting the number of splits as 10. The training MSE and R2 scores are 7.59×10-2 and 0.287 while that for testing data, the MSE is 7.68×10-2 and R2 score is of 0.300 in case of LR model by setting alpha as 0.1000. The summary of the performance of forward models for all three materials in included in Table 1.
Figure 5. Forward Trained models’ predicted response comparisons with simulated responses employing different networks (a-c) Random Forest Regressor (d-f) Lasso Regressor against different geometrical parameters: [hd, hp, period] = [40, 30, 300 nm], [80, 60, 500 nm], [ 80, 60, 500 nm], [40, 20, 225 nm], [40, 30, 500 nm], [40, 20, 475 nm] respectively with hexagon, triangle, circular ring, T-
shaped and square supercell meta-atom shapes.
Table 1. Summary of performance of Random forest and Lasso regressor for V, Re and Nb of forward models.
Model RF LR
Training Testing
Training time
Training Testing
Training time
Material MSE R2 MSE R2 MSE R2 MSE R2
V 1.36×10-2 0.982 3.65×10-2 0.869 5m 50s 9.08×10-2 0.186 8.97×10-2 0.201 2s Re 1.42×10-2 0.977 3.84×10-2 0.826 5m 34s 8.68×10-2 0.226 8.62×10-2 0.194 1s Nb 1.30×10-2 0.980 3.27×10-2 0.876 5m 57s 7.59×10-2 0.287 7.68×10-2 0.300 2s
Inverse machine learning architecture
The inverse design modelling poses challenge in training because many different designs can generate the same EM response leading to the conflicting training instances which results in difficulty for convergence. As the conventional, full- wave simulations based methods are not only slow but also inefficient in terms of memory, the prediction of geometric parameters combination of meta-atom for a given optical response becomes critical. The inverse model intends to predict the geometry and shape of the meta-atom. In this approach we swap the inputs and outputs, the model is given EM spectrum over the range 250 THz to 1000 THz as input to predict the feature vector of 63 points. The prediction in the inverse configurations are shown in Figure 6 with input and predicted values of geometrical features using Random forest (a-c) and Lasso regression (e-f). The predictions are included in order from Nb, Re and V in case of each model with shapes of square supercell, hexagon, circular ring, triangle and T-shapes, respectively. The values of predicted features are include with ground truth values and for image pixel prediction, the mean average difference between original and predicted values is found to be very close to zero and is included in Table 2. It can be seen that the geometrical parameters predicted in each case are within acceptable range. For inverse models, the minimum obtained training MSE and maximum R2 scores are 2.98 and 0.916, respectively while for testing the MSE and R2 scores are 7.99 and 0.463 in case of RF model. The values of the minimum training MSE and maximum R2 scores are recorded as of 9.56 and 0.112 while that of testing MSE 9.44
and R2 scores of 0.114 in case of LR model. The summary of performance of the inverse models is included in Table 3. It can be seen that inverse models are not as accurate as are the forward models. Also, the inverse training process is slow with minimum time needed in case of LR model to be 10 minutes 20 seconds.
Table 2. Summary of performance of Random forest and Lasso Regressor in inverse configuration for selective examples where predicted geometry is shown with the average difference of original and predicted pixel values of image
Material Random forest Lasso regression
Input Predicted MAE
for image pixels
Input Predicted MAE
for image pixels
hd hp p hd hp p hd hp p hd hp p
Nb 50 40 425 50.2 40.2 408.5 6.8×10-7 70 40 300 70.2 38.5 310.7 1.2×10-5 Re 70 30 425 66.7 36.4 389.8 2.2×10-5 70 40 300 66.7 42.7 341.2 2.2×10-5 V 40 20 200 44.2 23.5 256.5 1.1×10-7 60 50 325 62.7 40.6 373.2 1.7×10-5
Figure 6. Inverse trained models’ predicted response comparisons with simulated responses employing different networks (a-c) Random Forest Regressor (d-f) Lasso Regressor against different geometrical parameters: [hd, hp, period] = [50, 40, 425 nm], [70, 30, 425 nm],[ 70,40, 300 nm], [40, 20, 200 nm], [70,40, 300 nm],[60, 50, 350 nm] respectively with square supercell, hexagon, circular ring, triangle and T-shaped meta-atoms.
Table 3. Summary of performance of Random forest and Lasso regressor for V, Re and Nb of inverse model
Model RF LR
Training Testing
Training time
Training Testing
Training time
Material MSE R2 MSE R2 MSE R2 MSE R2
V 2.98 0.916 8.33 0.463 25m 5s 9.56 0.095 10.00 0.113 11m 7s Re 3.03 0.916 7.99 0.410 27m 19s 9.59 0.112 9.66 0.114 10m 20s Nb 3.27 0.917 8.12 0.408 26m 38s 9.38 0.091 9.44 0.091 11m 2s
CONCLUSION
An illustration of an intelligent design based on machine learning algorithms namely “Random Forest” and “Lasso Regression” techniques for photonics device i.e. the broadband EM absorber is presented. We have taken five different example structures including “circular ring”, “triangle”, “hexagon”, “T-shaped” and “square super cell” made of Vanadium (V), Niobium (Nb) and Rhenium (Re) refractory materials, one at a time. The input training parameters are a composite of geometrical features augmented with processed meta-atom images, where the 128 ×128 images are converted into 1,60 values by applying Singular Value Decomposition. The trained models predict the absorption spectrum with a very high efficiency for the case of random forest algorithm. The Lasso regression algorithm has also have performed significantly well with a very high speed with minimum of training time as 1 second. RF has shown advantages in terms of accuracy, while Lasso has in terms of training time. The minimum MSEs for forward and inverse models are 1.302 × 10-2 and 2.98, respectively while the maximum R2 scores are 0.982 and 0.916. The same dataset suffices for inverse modeling in predicting the geometrical features as well as the underlying image pixels. The importance of the carried out work lies in the fact that it can be extended to any other physics problem solution. The proposed method has offered remarkable benefit of speeding up the customary meta-device design methods by predicting the un-deterministic response of a meta-atom.
The absorption spectrum lies between 250 and 1000 THz, but is extendable to any region of EM spectrum based on objective.
REFERENCE
1. Sihvola, A. Metamaterials in electromagnetics. Metamaterials vol. 1 2–11 (2007).
2. Sun, Z., Xu, B., Jin, F., Zhou, G. & Lin, L. Machine Learning Approach for On-Demand Rapid Constructing Metasurface. IEEE Journal of Selected Topics in Quantum Electronics vol. 28 (2022).
3. Ijaz, S. et al. The Dawn of Metadevices: From Contemporary Designs to Exotic Applications. Advanced Devices
& Instrumentation vol. 2022 1–24 (2022).
4. Alagappan, G. & Png, C. E. Deep learning models for effective refractive indices in silicon nitride waveguides.
Journal of Optics (United Kingdom) vol. 21 (2019).
5. Ma, W., Cheng, F. & Liu, Y. Deep-Learning-Enabled On-Demand Design of Chiral Metamaterials. ACS Nano vol. 12 6326–6334 (2018).
6. Nadell, C. C., Huang, B., Malof, J. M. & Padilla, W. J. Deep learning for accelerated all-dielectric metasurface design. Optics Express vol. 27 27523 (2019).
7. Kiarashinejad, Y., Abdollahramezani, S. & Adibi, A. Deep learning approach based on dimensionality reduction for designing electromagnetic nanostructures. npj Computational Materials vol. 6 (2020).
8. So, S., Badloe, T., Noh, J., Rho, J. & Bravo-Abad, J. Deep learning enabled inverse design in nanophotonics.
Nanophotonics vol. 9 1041–1057 (2020).
9. Xiao, L. Y., Jin, F. L., Wang, B. Z., Liu, Q. H. & Shao, W. Efficient Inverse Extreme Learning Machine for Parametric Design of Metasurfaces. IEEE Antennas and Wireless Propagation Letters vol. 19 992–996 (2020).
10. Xia, S.-X., Zhai, X., Wang, L.-L. & Wen, S.-C. Plasmonically induced transparency in double-layered graphene nanoribbons. Photonics Research vol. 6 692 (2018).
11. Zhang, T. et al. Machine learning and evolutionary algorithm studies of graphene metamaterials for optimized plasmon-induced transparency. Optics Express vol. 28 18899 (2020).
12. Ranstam, J. & Cook, J. A. Statistical models: an overview. Br. J. Surg. 103, 1047 (2016).
13. Liaw, Andy & Wiener, Matthew. Classification and Regression by randomForest. R News vol. 2 18–22 (2002).
14. Rodriguez-Galiano, V., Sanchez-Castillo, M., Chica-Olmo, M. & Chica-Rivas, M. Machine learning predictive models for mineral prospectivity: An evaluation of neural networks, random forest, regression trees and support vector machines. Ore Geol. Rev. 71, 804–818 (2015).
15. Tibshirani, R. Regression Shrinkage and Selection Via the Lasso. Journal of the Royal Statistical Society: Series B (Methodological) vol. 58 267–288 (1996).
16. Naik, G. V., Shalaev, V. M. & Boltasseva, A. Alternative plasmonic materials: Beyond gold and silver.
Advanced Materials vol. 25 3264–3294 (2013).
17. Ijaz, S. et al. Exploiting zirconium nitride for an efficient heat-resistant absorber and emitter pair for solar thermophotovoltaic systems. Opt. Express 29, 31537–31548 (2021).
18. Fan, S. An alternative ‘Sun’ for solar cells. Nat. Nanotechnol. 9, 92–93 (2014).
19. Guler, U., Boltasseva, A. & Shalaev, V. M. Refractory plasmonics. Science vol. 344 263–264 (2014).
20. Chihhui, W. et al. Metamaterial-based integrated plasmonic absorber/emitter for solar thermo-photovoltaic systems. Journal of Optics vol. 14 24005 (2012).
