III. Machine Learning-Assisted Development of Organic Photovoltaics via High-Throughput
3.3 Results and Discussion
3.3.2 Machine Learning-Assisted Optimization and Prediction
A first step to develop ML model is designing input data structure, so-called feature extraction. Any training data, including images, need to be converted to fixed-length arrays. We introduced the DD as a universal feature of OPV. The feature covers thicknesses of device and ratio between components.
D:A ratio is a commonly used feature when devices are optimized, however, it would not be useful when a new component is added. In contrast, DD is an absolute amount and will be reusable for different material combinations or more complicated systems such as quaternary OPVs. Additives or co-solvents can also be described in DD. In the case of 4-dimensional or higher-dimensional space, it will be impossible to visualize without dimensionality reduction. That was another reason to design the dataset with only three axises in this study.
ML is typically used for classification, clustering and regression. For our purpose, i.e. analyzing the performance trend in the 3D parameter space, regression needs to be used. Regression is classified as supervised ML, meaning each training data set is paired with an output. The input parameters can be any size (dimension), and one variable was selected as the output for one model. In this work, PCE was an obvious choice for the output.
Table 3. 3. Summary of averaged model performance metrics at best trial obtained by different ML algorithms. 10 times trial hold-out validation method (80% and 20% of 2218 dataset were used for training and testing data, respectively) was used to select an algorithm for further study.
ML Model Data Type MAPE RMSE R2
Random forest Training 2.86 0.160 0.996
Testing 6.69 0.375 0.977
Gradient boosting Training 19.4 0.692 0.918
Testing 21.9 0.728 0.907
AdaBoost Training 86.0 1.08 0.799
Testing 75.2 1.08 0.795
Bagging Training 3.25 0.188 0.994
Testing 6.54 0.387 0.975
Extra trees Training 0.044 0.035 1.00
Testing 5.95 0.370 0.976 Histogram-based
gradient boosting
Training 11.4 0.381 0.975 Testing 12.4 0.496 0.959
We used the open-source Scikit-learn library and explored available algorithms: RF, gradient boosting, AdaBoost, bagging, ET, and histogram-based gradient boosting regression. The algorithms are ensemble methods, which combine weak estimators to generate a strong learner in order to improve
generalizability and robustness against outliers163. We also tried some non-ensemble methods and found the predicted values were far from the experimental result. Therefore, only ensemble methods were investigated further by comparing the MAPE, RMSE and R2 of the algorithms. Each algorithm was trained by using randomly selected 80% of 2218 datasets. Then the trained model was used to predict the PCE of the training dataset. MAPE, RMSE and R² were calculated from experimental PCE of the training data and predicted PCEs. The rest (20% of unused datasets, called a testing set) was used to predict PCE and then the three performance metrics were also calculated. The process was repeated 10 times, so-called 10 times trial hold-out validation, and their best predictive powers are summarized in Table 3. 3. Among the six algorithms, RF, ET and bagging algorithms showed good prediction accuracy with similar performance metrics. It is because ET is a modification of RF algorithm.164 We found that RF is an enhancement of bagging that can improve variable selection165 and has been widely used due to its robustness to outliers and high performance with non-linear high-dimensional data.166 It was successfully applied in various fields such as lithium-ion battery capacity estimation,167 wheat biomass estimation168 and magnetic resonance image synthesis166 etc. Therefore, we chose the RF algorithm to analyze the OPV data.
Figure 3. 15. Linear correlation between the measured and predicted efficiency of (a) training and (b) testing data in the RF model. Red line is represented as a reference for fitting the results. Various model performance metrics (MAPE, RMSE and R2) are summarized in green box. Inset graph shows the population distribution of absolute percentage errors for both training and testing data, respectively.
Figure 3. 15a shows a training data set and its highest performance metrics of the RF model (Table 3. 4). Experimentally measured PCEs are used as the X value and predicted PCEs are used as the Y value. For the perfect model, all green dots would be located on the redline, i.e. X = Y. The figure shows that most data points are in line with the ideal line and the performance metrics are exceptional
a b
0 2 4 6 8 10
0 2 4 6 8 10
0 1 2 3 4 5 6 7 8 9 10
0 200 400 600 800
# of Data
Absolute Percentage Error (%)
MAPETraining = 2.86 RMSETraining = 0.160 R2Training = 0.996
Measured PCE (%)
Predicted PCE (%)
0 2 4 6 8 10
0 2 4 6 8 10
0 2 4 6 810 12 14 16 18 20 0
20 40 60 80 100 120 140
# of Data
Absolute Percentage Error (%)
MAPETesting = 6.69 RMSETesting = 0.375 R2Testing = 0.977
Measured PCE (%)
Predicted PCE (%)
100
compared to previous reports.169, 170 However, the performance metrics of the training set can be overestimated due to overfitting. Therefore, the validation results of the testing data set are more important since the performance metrics were obtained from unknown data. The results are shown in Figure 3. 15b. As expected, the data points are more scattered than those of the training set, and the performance metrics are inferior. However, the performance metrics are still exceptional, and most of the data are within 6 % error range. The reason would be the consistently collected dataset. Previous works used data collected from multiple reports in the literature.
Table 3. 4. Model performance metrics of random forest algorithm. All the metrics were obtained by 10 times trial hold-out validation. The highest metric is trial #1.
Trial Data Type MAPE RMSE R2
1 Training 2.86 0.160 0.996
Testing 6.69 0.375 0.977
2 Training 2.91 0.165 0.995
Testing 7.10 0.416 0.971
3 Training 2.79 0.160 0.996
Testing 8.18 0.503 0.955
4 Training 2.88 0.162 0.996
Testing 6.65 0.424 0.966
5 Training 2.89 0.163 0.995
Testing 9.30 0.470 0.962
6 Training 3.21 0.159 0.996
Testing 8.25 0.396 0.971
7 Training 2.98 0.162 0.995
Testing 8.02 0.406 0.972
8 Training 2.84 0.157 0.996
Testing 7.06 0.499 0.956
9 Training 2.89 0.162 0.995
Testing 6.12 0.384 0.973
10 Training 2.63 0.162 0.995
Testing 9.70 0.420 0.969
Average Training 2.89 0.161 0.996
Testing 7.71 0.429 0.967
101
Figure 3. 16. Predicted PCEs of 125,000 deposition parameters (up to 50 g cm-2 DD of each material, 1 g cm-2 resolution) created by ML (represented as a top left illustration) and PCE variations in thin (TDD as ~ 20 g cm-2), middle (~ 40 g cm-2) and thick (~ 60 g cm-2) films depending on the composition. PM6 fraction represents DD of PM6/TDD.
After validation of the algorithm, a RF-based ML model was trained by using the 2218 datasets.
Using the model, predicted PCEs of whole 3D space of PM6, Y6 and IT-4F (0 to 50 μg cm-2 range, 1 μg cm-2 interval, total 125,000 datasets) could be created as shown in Figure 3. 16. The as-generated 3D graph is not useful to analyze performance trend as is. Therefore, meaningful datasets need to be filtered out and visualized in a more human-friendly way. As an example, composition parameters correspond to TDDs of ~20, ~40 and ~60 μg cm-2, referred as thin, middle and thick films, respectively, were extracted from the 3D graph as shown in Figure 3. 17a–c. The data planes in the 3D graph were converted to 2D graphs as shown in Figure 3. 16 by simple dimensionality reduction. The 2D graphs clearly show different optimum compositions for different thickness. The optimum composition at each thickness range can easily be found in this process. Such graphs would be useful for the applications requiring specific thickness. For example, semi-transparent application of OPV would need a thin film, and industrial printing would need thick film for reliable production.
0.0 0.2 0.4 0.6 0.8 1.0
0.0 0.2 0.4 0.6 0.8 1.0
Y6 fraction
PM6 fraction
0.001.00 2.003.004.00 5.006.007.00 8.009.00 10.00
Mid film PCE (%)
0.0 0.2 0.4 0.6 0.8 1.0
0.0 0.2 0.4 0.6 0.8 1.0
Y6 fraction
PM6 fraction
0.001.00 2.003.004.00 5.006.007.00 8.009.00 10.00
Thin film PCE (%)
0.0 0.2 0.4 0.6 0.8
0.0 0.2 0.4 0.6 0.8 1.0
Y6 fraction
PM6 fraction
0.00 1.002.003.00 4.005.006.00 7.008.009.00 10.00
Thick film PCE (%)
102
Figure 3. 17. Predicted PCE variations at TDD of (a) ~20, (b) ~40 and (c) ~60 μg cm-2, which correspond to thin, middle and thick film region. TDD was obtained from sum of DD for PM6, Y6 and IT-4F, respectively.
c b a
103
Figure 3. 18. Procedure for filtering and converting axes >8% PCE data. (a) Deposition parameters predicted to be > 8 % PCE and (b) corresponding parameters with converted axises to PM6 and Y6 fraction in a 3D parameter space.
Figure 3. 19. Procedure rounding filtered data to generate overlapping coordinates and counting the number of data at each PM6 and Y6 fraction coordinates. It is shown as a 3D surface plot of the number of data with PCE > 8% depending on PM6 and Y6 fraction.
We took the approach further to find a printing-friendly composition. As discussed earlier, manufacturing of OPVs by industrial printing methods demands a robust formulation that shows
a b
PM6 Fraction
Y6
Fraction Corresponding PCE 0.32 0.921569 8.0183 0.31579 0.903846 8.0274 0.31579 0.923077 8.0183 0.311688 0.90566 8.0274 0.311688 0.924528 8.0183 0.307692 0.907407 8.0274 0.307692 0.925926 8.0183
PM6 Fraction
Y6
Fraction Corresponding PCE
0.32 0.92 8.0183
0.32 0.90 8.0274
0.32 0.92 8.0183
0.31 0.91 8.0274
0.31 0.92 8.0183
0.31 0.91 8.0274
0.31 0.93 8.0183
Rounding to 2 decimal places
104
consistently high performance regardless of thickness variation, preferably in the range of 200 nm or thicker. To find such a formulation, another set of predicted PCEs in the whole 3D space was firstly created with higher resolution (200 data per axis in 0.25 μg cm-2 interval, total 8,000,000 datasets). We then filtered compositions with high PCE (> 8% was chosen in this case) from the dataset (8,000,000 compositions) obtained from ML (Figure 3. 18a). The absolute DD values were converted to relative compositions. The relative composition was used as the xy-plane, and TDD was used as the z-axis which became the thickness of the film. (Figure 3. 18b).
Figure 3. 20. Summary of predicted (a) top 5 BPFs and (b) top 10 BEFs with corresponding optimized TDD.
The data points with the same x-y coordinates (rounded to two decimal places) were then counted and the result is shown in Figure 3. 19. The x-y coordinate (PM6 fraction in total solid as x, Y6 fraction in acceptors as y) with the highest count was selected as a peak, in this case, it is located at PM6 fraction of 0.42 and Y6 fraction of 0.88 (PM6:Y6:IT-4F = 1:1.22:0.17). The formulation is called BPF. Finding BEF was simple. The 8,000,000 datasets were sorted by PCE and then the first dataset was selected.
BEF was PM6:Y6:IT-4F = 1:1.08:0.27 at 28.25 μg cm-2 TDD. Other top formulations can be seen in Figure 3. 20.
* Predicted Best Printable Formulation PM6:Y6:IT-4F (w/w)
= 0.42:(1-0.42)*0.88:(1-0.42)*0.12
= 0.42:0.5104:0.0696 1:1.22:0.17
* Predicted Best Efficiency Formulation PM6:Y6:IT-4F (w/w)
= 12:13:3.25 1:1.08:0.27
* Optimized Total Deposition Density by Machine Learning
= (12+13+3.25) g cm-2
= 28.25 g cm-2
a
b
105