Estimation of in-situ rock strength from borehole geophysical logs in Australian coal mine sites
Zizhuo Xiang a, Zexin Yu a, Won-Hee Kang b, Guangyao Si a, Joung Oh a,*, Ismet Canbulat a
a School of Minerals and Energy Resources Engineering, University of New South Wales, Sydney, NSW 2052, Australia
b School of Engineering, Design & Built Environment, Western Sydney University, Penrith, NSW 2751, Australia
Abstract
This study aims to improve the conventional empirical rock strength estimation method that is widely adopted in the Australian coal mining industry and provide more accurate rock uniaxial compression strength (UCS) predictions based on artificial neural network (ANN) models. 274 UCS data were collected from two longwall coal mines in Australia, along with the four geophysical logs (sonic, gamma, neutron, and porosity logs) and rock density of the tested core specimens. A two-layer ANN model was developed from these data based on the Levenberg–
Marquardt algorithm as a base model. Compared to conventional sonic-UCS fitting equations, in the proposed ANN model, the mean percentage error, root mean squared error and max absolute error was reduced from 34.27%, 15.23 MPa and 70.66 MPa to 20.67%, 11.02 MPa and 47.66 MPa, respectively. Further investigations were conducted by splitting the dataset based on rock lithology and mine locations, and separate ANN models were developed based on the divided subsets. Both lithology-specific and site-specific models exhibited improved estimation accuracy in comparison to the base model, indicating lithology and local geological conditions could considerably affect the estimation accuracy. Among the three types of models, the lithology-specific models yielded the most accurate predictions over the 274 data (MPE: 17.55%; RMSE: 8.84 MPa), followed by the site-specific models and the base model in descending order. Overall, all three models significantly outperformed the current empirical methods adopted in the studied mine sites. The outcomes of this study could provide more accurate UCS predictions for further geotechnical
* Corresponding author: Tel: +61 2 9385 5002; Fax: +61 2 9313 7269; E-mail: [email protected]
Preprint not peer reviewed
stresses based on borehole breakout.
Keywords: Uniaxial compression strength; Borehole geophysical logging; Artificial neural network; Comparative analysis
Preprint not peer reviewed
1 Introduction
Uniaxial compressive strength (UCS) is the most widely adopted rock property to represent in-situ rock strength, which can be either used as a direct measure or an indicator of rock mass strength.
Accurate UCS estimation of the rock in the borehole wall is essential for a wide range of engineering applications, including the design and construction of dams and tunnels (Sopacı and Akgün, 2008; Gokceoglu et al., 2016), slope stability assessment (Fischer et al., 2010), in-situ stress estimation from borehole breakout (Zoback et al. 1985; Barton et al. 1988; Lin et al. 2020), rock mass classification (Brown 1981), and wellbore stability analysis (Zoback 2010; Gholami et al. 2014; Najibi et al. 2017). The most traditional and reliable approach for estimating the in-situ rock strength is to conduct standard laboratory UCS experiments on in-situ samples (Hawkins 1998). Although this approach can directly measure the rock mechanical properties, only limited samples would be tested as it is expensive and time-consuming. In addition, the samples could be damaged during the coring and transport process. It is also worth noting that the measured UCS themselves have 10-20% deviations from the average values due to sample variations (McNally 1987).
Borehole geophysical logs have been extensively utilised in various geological and geotechnical applications, such as joint detection (Kneuker et al., 2017; Marzan et al., 2021), coal quality assessment (Webber et al., 2013; Maxwell et al., 2019), and absorbed gas estimation (O’Neill et al., 2021). Attempts have also been made to derive UCS from the borehole geophysical log data to reduce the need for laboratory tests and cost-efficiently estimate the rock strength. Early studies such as Carroll (1966, 1969) utilised compressional wave velocity obtained from sonic logs to estimate dynamic moduli of volcanic rocks through simple linear fittings. McNally (1987) compared the performance between the models developed from sonic and neutron logs based on data obtained from Australian coal mines. The results indicated that high resolution sonic log is more useful in estimating rock UCS. Over the years, various log data have been adopted for rock and coal strength estimation, including density, gamma ray, porosity, neutron and resistivity logs as well as other drilling-related parameters (e.g., rate of penetration) (Onyia 1988; Raaen et al.
1996; Hatherly et al. 2002; MacGregor 2003; Chang et al. 2006; Khandelwal and Singh, 2009;
Preprint not peer reviewed
Crawford et al. 2011). These methods are primarily based on conventional curve-fitting approaches and have demonstrated varying degrees of practice significance.
With the advancement of computer-aided programs, machine learning techniques have shown increasing applications in the regression and classification tasks due to their advantages in capturing nonlinear relations between input variables and targets. The significance of machine learning approaches in rock engineering was discussed in detail by Elmo and Stead (2020). Various machine learning models, including artificial neural network (ANN), support vector machine (SVM), adaptive neuro-fuzzy inference system (ANFIS), and classification and regression tree (CART), have been widely adopted in mining, petroleum and civil industries (Grima et al. 2000;
Al-Anazi and Gates 2010; Mahdevari et al. 2014; Mia and Dhar 2016; Singh et al. 2021; Lin et al.
2022). For instance, ANN and ANFIS models have shown better prediction results on estimating UCS and Young’s modulus of gypsum in comparison to conventional regression models (Yilmaz and Yuksek 2009), where the mean accuracy improved from 84.3%-85.1% to 88.61%-95.8%.
Similar conclusions were also obtained in subsequent research on this field (Sharma et al. 2010;
Yagiz et al. 2012; Majdi and Rezaei 2013). Given the flexibility and capability of the model on complex regression problems, ANN is one of the most commonly used machine learning approaches in the abovementioned research. Most recently, Miah et al. (2020) implemented ANN and SVM techniques and accurately estimated the UCS of clastic sedimentary rocks using sonic, gamma, density, porosity and resistivity logs. However, despite the reliability and accuracy improvement exhibited by machine learning approaches, they were primarily developed for petroleum industries based on petroleum wellbore logging systems. Due to the difference in the logging equipment and borehole conditions between the mining and petroleum industries (Gan et al. 2016), these methods have not yet been widely tested in mining applications.
In Australian coal mining industries, the most common practice is still using empirical fittings to estimate rock UCS from sonic logs (McNally 1990; Oyler et al. 2010; Butel et al. 2014). The equations are generally in the form of Eq. (1).
𝑈𝐶𝑆=𝑎×𝑒𝑏×𝑣 (1)
where a and b are fitting coefficients, and v is the sonic velocity obtained from sonic logs.
Preprint not peer reviewed
This empirical approach was first introduced by McNally (1987), with a generalised equation derived based on around 350 data from 24 boreholes. With the model validation at 18 mine sites (McNally 1990), the equation was widely adopted in Australian coal mining industry. Today, mine sites are still developing their own site-specific equations based on this approach as a fast and cost- effective way to estimate UCS (Oyler et al. 2008; Sabine et al. 2012; Hatherly 2013). However, the accuracies of these models vary significantly between mine sites (e.g. the coefficient of determination between sonic velocity and UCS ranged from 0.15 to 0.73 in mine sites studied by Butel et al. (2014)), and the effectiveness of such models was questioned by Medhurst et al. (2010) and Barton (2007). Alternative methods like the geophysical strata rating (GSR) (Hatherly et al.
2009) have been developed but not yet been widely utilised due to the amount of data and processing involved. In addition, the potential of borehole logs other than sonic logs on rock strength estimation has not been fully explored in Australian coal mining industry.
This paper aims to form a machine learning based relation between rock strength and borehole logs and improve the UCS estimation accuracy for Australian mines. The study is conducted based on data from two longwall coal mine sites located in New South Wales and Queensland, Australia with low UCS-sonic correlations, and an ANN model is developed based on borehole logs. The impacts of lithology and regional geology (mine locations) are also assessed. The proposed models aim to provide new insights into the effective utilisation of borehole logs for in-situ rock strength estimation.
Preprint not peer reviewed
2 Data collection
This study is conducted based on borehole geophysical logging data collected from two longwall coal mines (Mine A and Mine B) in Australia as follows: Mine A is located in Central Coast, New South Wales, and Mine B is in Bowen Basin, Queensland. In both mine sites, UCS tests were conducted on core samples with a dimension of around 60 mm (diameter) × 150 mm (height). 143 laboratory UCS data were collected from 20 boreholes in Mine A. The depths of the collected core samples ranged from 324 m to 442 m. In Mine B, there were 131 data obtained from 15 boreholes with depths ranging from 295 m to 585 m. The distribution of UCS of these two datasets is shown in Fig. 1. Most of the UCS values in Mine A range from 20 to 75 MPa, while the values in Mine B are concentrated around 15−45 MPa with a few exceptions with the values below 10 MPa and above 80 MPa.
Fig. 1. UCS distribution in the studied mine sites
The following four geophysical data were extracted from the borehole log: sonic, gamma, neutron, and porosity logs. The sonic log is one of the most widely adopted logs in Australian coal mining industry for geotechnical analysis (Oyler et al. 2010). The logging tool emits ultrasonic waves travelling from the device into the rock formation and returning back to the source, and the transmit
Preprint not peer reviewed
time between receiver pairs is measured as the sonic logs. The logging tool used in the studied mine sites was the multi-channel P-wave compensated series, which has four receivers spaced at 0.2 m intervals (Fig. 2). The short-spaced channel between the first and the second receivers was used in this analysis, as it is most commonly used for geotechnical interpretation by mine sites (Butel et al. 2014). The collected sonic logs are in the form of sonic velocity (VL2F) with the unit of m/s. Gamma logs indicate the extent of the natural radiation in borehole walls. Previous research suggested that it does not correlate with any geomechanical parameter and is typically used as a lithological indicator (McNally 1990; Firth and Elkington 1999). However, Miah et al. (2020) found that gammy ray showed high relative significance in estimating rock strength. The gamma ray recorded from the density probe (GRDE) is used in subsequent analysis (Hatherly et al. 2002).
Neutron logs represent the variations in hydrogen ion content in the borehole wall. McNally (1987) indicated that a linear correlation exists between rock UCS and neutron logs. The studied mine sites separately recorded short-spaced (SSN) and long-spaced (LSN) neutron measurements. LSN is used in this study as suggested by the mine geotechnical engineers. Porosity logs are secondary log products and not directly measured through logging instruments. There are a number of ways to derive porosity from density, sonic and neutron logs. This study utilises the neutron porosity, which is calculated using a cubic function of the ratio of the LSN to SSN logs (Hatherly et al.
2004).
Preprint not peer reviewed
Fig. 2. A schematic diagram of a sonic logging instrument (after Butel et al. (2014))
The logging data were extracted and averaged at the core sample locations over the same depth interval. In addition to the downhole geophysical logs, laboratory measured density values for the core samples were also collected. Data quality control, such as checking the depth accuracy of core samples, diameter-to-height ratio of test specimens, loading curves, and failure mechanism, have been carried out to ensure data reliability and consistency. However, it should be noted that the mineral logging systems are generally not as sophisticated as those used in petroleum industries.
Data quality issues, such as calibration, repeatability and contractor equivalence, could present and affect estimation accuracy. Detailed discussion on the data issues and log quality control for mineral logging systems can be found in Hatherly et al. (2002). Removing the logging errors is difficult and time-consuming. Implementing standardised logging procedures like those mentioned in Butel et al. (2014) could eliminate this issue. However, it is challenging and cannot be achieved in a short period of time. Nonetheless, compared with using a single log, the chance of the model results being affected by logging errors would be reduced if multiple logs were used. In addition, the errors of the model in the subsequent should be interpreted considering the data quality issues.
Preprint not peer reviewed
3 Data analysis and model development
3.1 Empirical model accuracy
Both mine sites use the site-specific empirical correlations between UCS and sonic velocity to estimate in-situ rock strength. The model of each mine was developed based on their own UCS data of the core samples following the exponential form suggested by McNally (1990). The equations for Mine A and Mine B are presented as Eq (2) and Eq (3), respectively.
(2) 𝑈𝐶𝑆= 6.313 ×𝑒0.0005664 ×𝑉𝐿2𝐹
(3) 𝑈𝐶𝑆= 1352 ×𝑒‒
12500 𝑉𝐿2𝐹
In this study, to evaluate the model performance of these equations, three performance measures are used as follows: mean percentage error (MPE), root mean squared error (RMSE) and maximum absolute error (MAE). The mean percentage error (MPE) and RMSE represent the average discrepancies between the predictions and targets, while the MAE provides information regards to the worst performance of the model. Smaller MPE, MAE and RMSE values indicate more reliable models. These performance measures are defined as follows:
(4) 𝑀𝑃𝐸= 100 ×1𝑛∑𝑛
𝑖= 1
(
(𝑈𝐶𝑆𝑈𝐶𝑆𝑚𝑖‒ 𝑈𝐶𝑆𝑚𝑖 𝑝𝑖))
(5) 𝑅𝑀𝑆𝐸= 1𝑛∑𝑛
𝑖= 1(𝑈𝐶𝑆𝑚𝑖‒ 𝑈𝐶𝑆𝑝𝑖)2
(6) 𝑀𝐴𝐸= 100 × max
𝑖
|
𝑈𝐶𝑆𝑈𝐶𝑆𝑚𝑖‒ 𝑈𝐶𝑆𝑚𝑖 𝑝𝑖|
where n is the total number of samples, and UCSmi and UCSpi are measured and predicted UCS values, respectively.
Table 1 summarises the predictive performance of the two equations. In general, it is shown that both site-specific models are not very accurate in UCS predictions with over 30 % MPE. The MPE, RMSE and MAE of Mine B are about 3.2%, 6.7 MPa and 37.5 MPa higher than those of Mine A,
Preprint not peer reviewed
respectively. The errors could be attributed to the limitations of using single log or low-quality log data, or a combination of both. Lower UCS estimation accuracies mean that a higher factor of safety due to the higher prediction uncertainties would be required for further geotechnical analysis (e.g., roof support design), and thus have negative effects in achieving the cost-safety optimum in underground operations. More accurate models would obviously benefit the studied mine sites.
Table 1
Predictive performance of sonic-UCS equations on the studied mine sites
The number of data MPE (%) RMSE (MPa) MAE (MPa)
Mine A 143 35.80 17.73 70.66
Mine B 131 32.61 11.98 33.15
Overall 274 34.27 15.23 70.66
3.2 ANN development and performance
As mentioned in Section 1, ANN is selected as the machine learning algorithm for this study. To first assess the statistical importance of each input parameter, the correlations between inputs and output data are plotted in Fig. 3, and Pearson correlation coefficients between the inputs and output are given in Table 2. The coefficients represent the linear dependence between two variables (Eq (7)) and range between -1 and 1. A larger positive value means a greater positive correlation, whereas a stronger negative correlation could be obtained when the value is closer to -1.
(7) 𝜌(𝐴,𝐵) =𝑛 ‒11∑𝑛
𝑖= 1
(
A𝑖𝜎‒ 𝜇A 𝐴) (
B𝑖𝜎‒ 𝜇𝐵𝐵)
where μA and σA are the mean and standard deviation of variable A, respectively, and μB and σB are the mean and standard deviation of variable B, respectively.
Among all the inputs, sonic velocity has the highest linear correlation with the UCS (0.52).
However, it is still generally lower than the values presented in related literature, such as in German Creek Formation (0.91), Glennies Creek project (0.94) and Liddell Colliery (0.94) (McNally
Preprint not peer reviewed
1987). The UCS also exhibits positive relationships with the rock density (Fig. 3b) and neutron logs (Fig. 3d) while negatively correlating with the porosity (Fig. 3e). These correlations are considerably weaker than sonic velocity as evident by the values listed in Table 2. On the other hand, the linear relationship between the gamma log and rock strength remains unclear (Fig. 3c).
Despite the low correlation coefficients of some input variables, they are included in this analysis because their linear correlation is not negligible and additional nonlinear correlation can still exist between these inputs and the output.
(a) VL2F (b) Density (c) GRDE
(d) LSN (e) Porosity
Fig. 3. Visualisation of rock UCS against predictor data Table 2
Correlation coefficients between rock UCS and borehole logs
UCS VL2F Density GRDE LSN Porosity
UCS 1.000 0.522 0.230 0.169 0.198 -0.224
Preprint not peer reviewed
VL2F 0.522 1.000 0.549 -0.336 0.733 -0.691
Density 0.230 0.549 1.000 -0.323 0.377 -0.349
GRDE 0.169 -0.336 -0.323 1.000 -0.643 0.374
LSN 0.198 0.733 0.377 -0.643 1.000 -0.849
Porosity -0.224 -0.691 -0.349 0.374 -0.849 1.000 The input dataset is randomly divided into training (70%), validation (15%) and test (15%) datasets. The ANN model is constructed based on the Miah et al. (2020) and some trials to get the optimised results as follows: the model consists of one input layer, two sigmoid hidden layers and one output layer, with mean squared error (MSE) as the loss function. The Levenberg–Marquardt (LM) feedforward algorithm is used as the training function as it is generally more reliable and converged faster compared to other methods in the backpropagation system (Ceryan et al. 2013).
The number of neurons for each layer is tested from 1-15 per hidden layer and is determined to optimise the model performance while avoiding overfitting by checking if the performance measure values are consistent across training, validation and test datasets. The network which resulted in the minimum error in all three datasets is chosen as the final model. The flow chart of the ANN development procedure is given in Fig. 4. The model with six neurons in the first layer and ten neurons in the second layer yield the most accurate results, with an MPE of 20.67 % and RMSE of 11.02 MPa across all dataset. Fig. 5 presents the comparison between predictions and targets on various datasets on this optimised model. The red lines represent the perfect linear fit to provide insights into the overestimation/underestimation of the model. Overall, the errors on all three datasets are consistent, indicating the model is not overfitted. When the UCS is under 60 MPa, the data is reasonably distributed around the best fit line, whereas the model tends to underestimate the rock strength when UCS exceeds 70 MPa. This is due to the lower number of UCS values exceeding 70 MPa in the dataset (Fig. 1). Furthermore, such high UCS values are not common at the geological conditions and depths of the studied mine sites.
Preprint not peer reviewed
Fig. 4. A flow chart of the ANN model development procedure
Compared with site-specific sonic-UCS models in Section 3.1 (Table 1), considerable improvements can be observed on both sites, with a 13.6% improvement in MPE and a 4 MPa (27.6%) reduction in RMSE. The MAE also reduced by 29.7% from 70.66 MPa to 49.66 MPa. It is apparent that considering only sonic velocity is insufficient as it does not fully capture the variation of the UCS and other input parameters need to be included regardless of their relatively lower linear correlation levels. Utilising a nonlinear form of the model (ANN) involving multiple
Preprint not peer reviewed
geophysical parameters could more effectively reflect the variation of rock strength under different geological conditions and mitigate the effects of data quality issues compared to using a single log.
(a) Training dataset (b) Validation dataset
(c) Test dataset (d) All dataset
Fig. 5. Accuracy of the optimised ANN model on different datasets
Preprint not peer reviewed
Fig. 6. Comparison between the predictions of the ANN model and site-specific empirical fittings
Preprint not peer reviewed
4 Discussion and further model improvement
To further improve the stress estimation accuracy of the ANN model developed in Section 3.2 (referred to as the ‘base model’ in later sections), attempts have been made to derive different models based on data characteristics. In Section 4.1, three lithology-specific models were developed for sandstone, siltstone and interbedded sandstone and siltstone, and the impact of lithology on the machine learning model performance is assessed. Subsequently, in Section 4.2, the whole dataset was divided into two subsets based on mine locations, and two site-specific models were created to evaluate the effects of the regional geological and geotechnical conditions.
Finally, the comparison between the three types of models is given in Section 4.3, and a guideline on estimating in-situ rock strength based on the borehole logs for Australian mines is proposed.
4.1 Lithology-specific models
There are two most prominent lithologies among the collected data as follows: sandstone (SS) and siltstone (ST), which are two of the most common rock types in Bowen Basin and Sydney Basin (Wliwa et al. 2003). Within 274 data, there are 88 SS data, 79 ST data, and 84 data which are interbedded sandstone and siltstone (SS+ST). The proportion of each lithology in Mine A and Mine B is shown in Fig. 7. The other lithologies in Mine A (Fig. 7a) include mudstone, conglomerate and calcite. Previous research indicated that the correlation between rock UCS and sonic logs could vary depending on the lithology (McNally 1990; Butel et al. 2014; Rahman and Sarkar 2021). Therefore, the impact of lithologies (SS, ST and SS+ST) on the model performance are analysed in this section.
Preprint not peer reviewed
(a) (b) Fig. 7. The proportion of each lithology in data from (a) Mine A and (b) Mine B
The whole dataset is divided into three subsets (SS, ST and SS+ST). No ANN models could be developed for other lithologies due to the limited amount of data, and thus, they were not included in this analysis. The correlation coefficients between the input and output for different lithologies are listed in Table 3. It can be obtained that the correlations of the geophysical data to the rock UCS vary significantly by lithology, and all of them are different compared with the overall dataset (Table 2). For SS data, all inputs exhibit improved linear correlation to the output, except for the gamma log. Similarly, for the ST subset, only the correlation coefficient for density is reduced relative to the overall dataset, whereas the values for other logs show various degrees of improvement. The improvements in the correlation coefficients indicate that the uncertainties and nonlinear correlations in the data are reduced in these subsets. On the other hand, the rock strength in the SS+ST subset exhibits weaker linear correlations to all inputs compared to other lithologies and the overall dataset.
Table 3
Correlation coefficients for each lithology subset
VL2F Density GRDE LSN Porosity
UCS_SS 0.7652 0.4798 -0.0698 0.3712 -0.3987
Preprint not peer reviewed
UCS_ST 0.6802 0.2193 0.2159 0.3461 -0.3450
UCS_SS+ST 0.2966 0.0118 0.3898 -0.0256 -0.0858
The model development methodology follows the same procedure as presented in Section 3.2. The features are the same as the base model, and three lithology-specific models are developed based on three subsets. The model structures and the number of neurons are determined based on trials.
Similar to the base model, all models contain one input layer, two hidden layers and one output layer. The number of neurons in the first and second hidden layers for SS, ST and SS+ST models are (7,6), (3,3) and (11,4), respectively. The predictive performance of each model on the training, validation and test subset is given in Fig. 8. The overall estimation accuracy is listed in Table 4, along with the comparison to the base model performance on the corresponding dataset.
As indicated in the table, the base model exhibited similar accuracy on SS and ST subsets, and the errors are slightly lower than the values on the overall dataset. On the other hand, the base model does not yield similar performance on SS+ST data, which is around 5% worse in MPE and 5 MPa higher in RMSE compared to single-lithology data. In terms of lithology-specific models, various degrees of accuracy improvement can be seen across all three subsets. The performance of both SS and ST models is similar, with around 2 – 3.7% improvements in MPE and 1.6 – 3 MPa reductions in RMSE compared to the base model. Within the three subsets, the SS+ST model exhibits the most performance increase. The MPE reduces from 23.71% to 18.92%, and the RMSE reduces by over 3 MPa. However, despite the accuracy improvement, this model still performs worse than either SS or ST model. The primary reason can be rock heterogeneity and anisotropy.
Since the grain size for SS and ST is different, the rock would perform more heterogeneously and anisotropic compared with the single-lithology core samples, and the ratio of SS over ST within the sample could considerably affect its mechanical behaviours. This could also be the reason that the correlation coefficients for SS+ST data are lower than other lithologies as well as the overall dataset.
Preprint not peer reviewed
(a) SS
(b) ST
(c) SS+ST
Fig. 8. Correlation between predictions and actual values of each lithology-specific model on the corresponding subset
Table 4
Estimation accuracy of each lithology-specific model and its comparison to the base model
Preprint not peer reviewed
Subset SS ST SS+ST
MPE, % (base model) 19.00 18.33 23.71
RMSE, MPa (base model) 8.31 8.97 13.82
MAE, MPa (base model) 25.17 40.92 32.96
MPE % (lithology-specific model) 15.27 16.47 18.92
RMSE, MPa (lithology-specific model) 6.66 6.08 10.80
MAE, MPa (lithology-specific model) 14.48 16.03 38.88
In order to examine if the accuracy improvement is affected by the small number of training data, a validation process is carried out by developing a model based on 80 data, which are randomly sampled from the overall dataset. This random-selection process is repeated ten times, and the performance of all the trials is shown in Fig. 9. The dots represent the errors on each trial; the black dashed lines are the average results of the ten models; and the red dashed lines are the values of the base model. The MPE of each trial deviates approximately ± 6 % from the result of the base model, and the RMSE varies around ± 1.5 MPa. In general, the average MPE of all trials is slightly worse (2.7%) than the base model, while the average RMSE is identical to the based model. The comparison indicates that the impact of the small dataset (80 data) on the model performance is negligible.
Based on the above analysis, it is apparent that lithology has considerable impacts on the relationship between geophysical logs and rock strength, and lithology-specific could improve the estimation accuracy.
Preprint not peer reviewed
(a) (b)
Fig. 9. Performance measures of the models developed from randomly selected 80 data: (a) MPE and (b) RMSE
4.2 Site-specific models
Apart from the lithology, regional geology and mine site location may have impacts on the estimation accuracy (Butel et al. 2014). As mentioned in Section 2.2, Mine A is located in Central Coast, while Mine B is in Bowen Basin, and the regional geological and geotechnical conditions between the two mine sites vary significantly (Rajabi et al. 2017). Therefore, the overall data are separated into two subsets based on the locations of the data (Mine A and Mine B), and the performance of the site-specific models is investigated in this section.
Table 5 presents the correlation coefficients of the input variables for two mine sites. It can be obtained that the linear correlation between inputs and outputs in Mine B is significantly stronger than that of Mine A, as evident by the values listed in the table. Similar to Section 4.2, two site- specific models ANN are developed for two mine sites. For Mine A, the model contains 9 neurons in the first hidden layer and 13 neurons in the second hidden layer, whereas for Mine B, the numbers are 3 and 7, respectively. The model errors and their comparisons with the base model are given in Table 6. The values in parentheses in the ‘Overall’ rows are the predictive performance of the base model on the corresponding subset. It can be seen that the UCS estimation accuracy varies significantly by mine sites. The MPE, RMSE and MAE of the base model for Mine A data are approximately 9.4%, 7.4 MPa and 20.9 MPa better than for Mine B data, respectively. The Mine A model performs similarly to the base model, with minor accuracy improvements across
Preprint not peer reviewed
the subset. The MPEs for both general and site-specific models exceed 20%, and the RMSE values are over 13 MPa. On the other hand, the improvements for Mine B data are slightly greater than Mine A, and the site-specific model yields significantly more accurate estimations in this subset.
The variations in the accuracy between Mine A and Mine B models could be attributed to different geological environments or log quality inconsistencies. Nevertheless, It can be concluded that the performance of the ANN models is affected by the mining horizon, which is similar to the empirical sonic-UCS equations.
Table 5
Correlation coefficients for Mine A and Mine B data
VL2F Density GRDE LSN Porosity
UCS_Mine A 0.5205 0.3730 -0.0838 0.3425 -0.3236
UCS_Mine B 0.8108 0.5492 -0.4794 0.5952 -0.5249
Table 6
Predictive performance of site-specific models
Subsets MPE (%) RMSE (MPa) MAE (MPa)
Mine A Training dataset 21.35 12.96 47.09
Validation dataset 33.34 13.25 25.58
Test dataset 25.52 13.98 33.49
Overall 23.72 (25.14) 13.16 (13.92) 47.09 (49.66)
Mine B Training dataset 12.57 4.72 14.23
Validation dataset 16.81 5.08 12.56
Test dataset 18.29 4.22 7.61
Preprint not peer reviewed
Overall 14.09 (15.79) 4.71 (6.56) 14.23 (28.78)
4.3 Recommendation on UCS estimation for Australian mine applications
In this study, two types of models (lithology-specific and site-specific models) are developed based on different data characteristics in addition to the base model. These models could yield more accurate predictions on the subsets than the base model. Fig. 10 shows the performance of all three types of models across the whole dataset. Since the data with rock types other than SS, ST and SS+ST (23 data) are not included in the lithology-specific models, the predictions of these data in the ‘Lithology-specific model’ category in Figs. 10 and 11 utilise the values of the base model. All three types of models perform well in the range of 20 – 70 MPa (20 – 69 data per bin), with percentage errors below 20% and absolute errors ranging from 7 – 10 MPa. It can also be observed that both lithology-specific and site-specific models mostly outperform the base model within this range. Relatively larger errors can be seen for data below 10 MPa (3 data) or above 80 MPa (11 data) due to the small amount of data. Overall, among the three models, the lithology-specific models yield the most accurate predictions across the dataset, with MPE of 17.55% and RMSE of 8.84 MPa, followed by the site-specific models (MPE: 19.12%; RMSE: 11.02 MPa) and the base model (MPE: 20.67%; RMSE: 10.05 MPa) in descending order.
(a) (b)
Fig. 10. (a) Mean percentage error and (b) mean absolute error of three models based on 10 MPa interval
Preprint not peer reviewed
Fig. 11 summarises the MPE and RMSE of different models on Mine A and Mine B subsets. As discussed in Section 4.2, the models perform differently between the two mine sites. For Mine A, lithology-specific models exhibit the most accurate predictions with 19.11% MPE and 10.84 MPa RMSE. The site-specific model performs similarly to the base model with minor accuracy improvements. Therefore, it is apparent that lithology-specific models are more appropriate for estimating rock strength in Mine A, while for data without lithology information or with lithologies other than SS, ST and SS+ST, the site-specific model should be used. On the other hand, for Mine B, all three models show significant improvements over the empirical equation (Eq. 3) and could relatively accurately estimate the rock UCS. More specifically, the site-specific model yields the most accurate predictions, whereas the lithology-specific models have similar predictive performance as the base model with the same MPE (15.8%) and slightly lower RMSE (5.9 MPa).
As a result, the site-specific model is recommended for Mine B.
Fig. 11. Comparison of the accuracy of each model on Mine A and Mine B data
Based on the above discussion, the proposed models based on multiple borehole geophysical logs perform significantly better than the conventional sonic-UCS equations. The base model derived from the large dataset (274 data) is more reliable for applying to other unknown conditions, but its accuracy may not be satisfactory for quantitative geotechnical analysis. As for the lithology- specific models, the predictions are more accurate than the base model as the correlation between rock strength and geophysical logs would vary based on rock types. Current models only include three common rock types (SS, ST and SS+ST), and with more data collected and more lithology-
Preprint not peer reviewed
specific models developed, the accuracy is expected to improve further. However, it should be noted that there are limitations when utilising lithology-specific models. First of all, the model may not yield as accurate results for interbedded lithologies as single lithology. It is also difficult to collect sufficient data for each lithology to generate reliable lithology-specific models.
Furthermore, practical usability may be constrained as using these models would require additional features (lithology) to distinguish the data, which would add complexity to implementing these models. Developing site-specific models would also enhance the estimation accuracy, although the degree of improvement may not be greater than the lithology-specific models. They could be used for the data not included in the lithology-specific models or the lithology information in the area of interest is unclear. As for the model implementation on other mine sites, it is suggested to use the lithology-specific models if available, and the base model could be used when the lithology information is absent or the data is not covered by the lithology-specific models. In addition, based on the proposed ANN developments methodology and framework, site-specific models can be created based on site-specific UCS data and utilised as a replacement for the abovementioned recommendations.
Preprint not peer reviewed
5 Conclusion
Given the importance of the UCS in geological engineering applications and the difficulties of performing standard UCS tests on in-situ samples, the correlation between borehole logs and rock UCS was evaluated based on 274 data obtained from two Australian longwall coal mines. The empirical site-specific sonic-UCS equations could not yield accurate UCS predictions with MPE, RMSE and MAE of 34.27%, 15.23 MPa and 70.66 MPa, respectively, which would increase the uncertainties when applying the predictions for further geotechnical analysis. To improve the prediction accuracy, ANN models were constructed based on four geophysical borehole logs (sonic, gamma, neutron, and porosity logs) and rock density. The structure of the ANN models was determined based on prior literature and trials. The base model developed from the 274 data exhibited an improved accuracy of 20.67% (MPE), 11.02 MPa (RMSE) and 49.66 MPa (MAE), respectively.
The data were then divided into various subsets based on lithologies and site locations to investigate the impacts of rock types and regional geological conditions on the model predictive performance. Three lithology-specific models for SS, ST and SS+ST were developed, and various degrees of accuracy improvements were obtained in comparison with the base model. The models developed from single lithology (SS and ST) were found to perform better than the model based on interbedded lithologies (SS+ST), which is speculated to be caused by the heterogeneity and anisotropy introduced by the mixing rock types. Similarly, the site-specific models also exhibited better performance than the base model, and Mine B model yielded significantly more accurate predictions than Mine A model.
Overall, the lithology-specific models yielded the most accurate predictions over the entire dataset (MPE: 17.55%; RMSE: 8.84 MPa), followed by the site-specific models (MPE: 19.12%; RMSE:
11.02 MPa) and the base model (MPE: 20.67%; RMSE: 10.05 MPa) in descending order. Given the log quality issues and 10-20% inherent variation of the measured UCS values, the models are considered adequate as a replacement for existing sonic-UCS equations. For Mine A, it is recommended to use a combination of lithology-specific and site-specific models to estimate in- situ rock UCS, whereas in Mine B, the site-specific model should be used. In terms of further model deployment on other mine sites, the lithology-specific models are recommended to use first,
Preprint not peer reviewed
and the base model can be applied in the absence of the lithology-specific models. Site-specific models could also be further developed based on the proposed methodology and framework.
Acknowledgements
The work reported here is funded by the Australian Coal Industry’s Research Program (ACARP), Grant no. C26063.
Preprint not peer reviewed
6 References
Al-Anazi, A., Gates, I.D., 2010. A support vector machine algorithm to classify lithofacies and model permeability in heterogeneous reservoirs. Eng. Geol. 114, 267–277.
Barton, C.A., Zoback, M.D., Burns, K.L., 1988. In‐situ stress orientation and magnitude at the Fenton Geothermal Site, New Mexico, determined from wellbore breakouts. Geophys.
Res. Lett. 15, 467–470.
Barton, N., 2007. Rock Quality, Seismic Velocity, Attenuation and Anisotropy. London: Taylor
& Francis.
Butel, N., Hossack, A., Kizil, M.S., 2014. Prediction of in situ rock strength using sonic velocity.
Paper presented at the Coal Operators’ Conference. University of Wollongong. The Australasian Institute of Mining and Metallurgy & Mine Managers Association of Australia. pp. 89-102.
Carroll, R.D., 1969. The determination of the acoustic parameters of volcanic rocks from compressional velocity measurements. Int. J. Rock Mech. Min. Sci. Geomech. Abstr. 6, 557–579.
Carroll, R.D., 1966. Rock Properties Interpreted from Sonic Velocity Logs. J. Soil Mech. Found.
Div. 92, 43–51.
Ceryan, N., Okkan, U., Kesimal, A., 2013. Prediction of unconfined compressive strength of carbonate rocks using artificial neural networks. Environ. Earth Sci. 68, 807–819.
Chang, C., Zoback, M.D., Khaksar, A., 2006. Empirical relations between rock strength and physical properties in sedimentary rocks. J. Pet. Sci. Eng. 51, 223–237.
Crawford, B., Alramahi, B., Gaillot, P., Sanz, P., DeDontney, N., 2011. Mechanical Rock Properties Prediction: Deriving Rock Strength and Compressibility From Petrophysical Properties. Presented at the 12th ISRM Congress, Paper presented at the 12th ISRM Congress, Beijing, China.
Elmo, D., Stead, D., 2020. Disrupting rock engineering concepts: is there such a thing as a rock mass digital twin and are machines capable of learning rock mechanics? Presented at the Proceedings of the 2020 International Symposium on Slope Stability in Open Pit Mining and Civil Engineering, Australian Centre for Geomechanics, pp. 565–576.
Firth, D., Elkington, P., 1999. Log Analysis for Mining Applications. Reeves Wireline Services.
Fischer, L., Amann, F., Moore, J.R., Huggel, C., 2010. Assessment of periglacial slope stability for the 1988 Tschierva rock avalanche (Piz Morteratsch, Switzerland). Eng. Geol. 116, 32–
Preprint not peer reviewed
43.Gan, T., Balmain, B., Sigbatullin, A., 2016. Formation evaluation logoff results comparing new generation mining-style logging tools to conventional oil and gas logging tools for application in coalbed methane (CBM) field development. J. Nat. Gas Sci. Eng. 34, 1237–
1250.
Gholami, R., Moradzadeh, A., Rasouli, V., Hanachi, J., 2014. Practical application of failure criteria in determining safe mud weight windows in drilling operations. J. Rock Mech.
Geotech. Eng. 6, 13–25.
Grima, M.A., Bruines, P., Verhoef, P., 2000. Modeling tunnel boring machine performance by neuro-fuzzy methods. Tunn. Undergr. Space Technol. 15, 259–269.
Gokceoglu, C., Turer, A., Nefeslioglu, H.A., Turer, D., Meral, C., 2016. Safety assessment of limestone-based engineering structures to be partially flooded by dam water: A case study from northeastern Turkey. Eng. Geol. 209, 44–55.
Hatherly, P., 2013. Overview on the application of geophysics in coal mining. Int. J. Coal Geol.
114, 74–84.
Hatherly, P., Medhurst, T., Ye, G., Payne, D., 2009. Geotechnical evaluation of roof conditions at Crinum Mine based on geophysical log interpretation. Paper presented at the 9th Underground Coal Operators’ Conference, The Australasian Institute of Mining and Metallurgy, Melbourne.
Hatherly, P., Medhurst, T., Zhou, B., Guo, H., 2002. Geotechnical evaluation for mining - assessing rock mass conditions using geophysical logging. ACARP Project C8022b.
Hatherly, P., Sliwa, R., Turner, R., Medhurst, T., 2004. Qantitative Geophysical Log Interpretation for Rock Mass Characterisation. ACARP Project C11037.
Hawkins, A.B., 1998. Aspects of rock strength. Bull. Eng. Geol. Environ. 57, 17–30.
Kneuker, T., Hammer, J., Shao, H., Schuster, K., Furche, M., Zulauf, G., 2017. Microstructure and composition of brittle faults in claystones of the Mont Terri rock laboratory (Switzerland):
New data from petrographic studies, geophysical borehole logging and permeability tests.
Eng. Geol. 231, 139–156.
Lin, H., Kumar Singh, S., Xiang, Z., Hee Kang, W., Raval, S., Oh, J., Canbulat, I., 2022. An investigation of machine learning techniques to estimate minimum horizontal stress magnitude from borehole breakout. Int. J. Min. Sci. Technol.
Lin, H., Singh, S., Oh, J., Canbulat, I., Kang, W.H., Hebblewhite, B., Stacey, T.R., 2020. A combined approach for estimating horizontal principal stress magnitudes from borehole breakout data via artificial neural network and rock failure criterion. Int. J. Rock Mech.
Min. Sci. 136, 104539.
MacGregor, S., 2003. Maximising in-situ stress measurement data from borehole breakout using acoustic scanner and wireline tools. ACARP Project C10009.
Preprint not peer reviewed
Mahdevari, S., Shahriar, K., Yagiz, S., Shirazi, M.A., 2014. A support vector regression model for predicting tunnel boring machine penetration rates. Int. J. Rock Mech. Min. Sci. 72, 214–
229.
Majdi, A., Rezaei, M., 2013. Prediction of unconfined compressive strength of rock surrounding a roadway using artificial neural network. Neural Comput. Appl. 23, 381–389.
Marzan, I., Martí, D., Lobo, A., Alcalde, J., Ruiz, M., Alvarez-Marron, J., Carbonell, R., 2021.
Joint interpretation of geophysical data: Applying machine learning to the modeling of an evaporitic sequence in Villar de Cañas (Spain). Eng. Geol. 288, 106126.
Maxwell, K., Rajabi, M., Esterle, J., 2019. Automated classification of metamorphosed coal from geophysical log data using supervised machine learning techniques. Int. J. Coal Geol. 214, 103284.
McNally, G.H., 1990. The Prediction of Geotechnical Rock Properties from Sonic and Neutron Logs. Explor. Geophys. 21, 65–71.
McNally, G.H., 1987. Estimation of coal measures rock strength using sonic and neutron logs.
Geophys. Contrib. Aust. Coal Mine Plan. 24, 381–395.
Medhurst, T., Hatherly, P., Zhou, B., 2010. 3D geotechnical models for coal and clastic rocks based on the GSR. Paper presented at the 10th underground coal operator’s conference.
University of Wollongong & the Australasian Insitute of Mining and Metallurgy, pp. 40–
49.
Mia, M., Dhar, N.R., 2016. Prediction of surface roughness in hard turning under high pressure coolant using Artificial Neural Network. Measurement 92, 464–474.
Miah, M.I., Ahmed, S., Zendehboudi, S., Butt, S., 2020. Machine Learning Approach to Model Rock Strength: Prediction and Variable Selection with Aid of Log Data. Rock Mech. Rock Eng. 53, 4691–4715.
Najibi, A.R., Ghafoori, M., Lashkaripour, G.R., Asef, M.R., 2017. Reservoir geomechanical modeling: In-situ stress, pore pressure, and mud design. J. Pet. Sci. Eng. 151, 31–39.
O’Neill, K.T., Birt, B., Hopper, T., 2021. Borehole measurements of adsorbed gas content in coals using stimulated diffusion nuclear magnetic resonance. Int. J. Coal Geol. 247, 103845.
Onyia, E.C., 1988. Relationships Between Formation Strength, Drilling Strength, and Electric Log Properties. Presented at the SPE Annual Technical Conference and Exhibition, Paper presented at the SPE Annual Technical Conference and Exhibition, Houston, Texas.
Oyler, D.C., Mark, C., Molinda, G.M., 2010. In situ estimation of roof rock strength using sonic logging. Int. J. Coal Geol. 83, 484–490.
Preprint not peer reviewed
Oyler, D.C., Mark, C., Molinda, G.M., 2008. Correlation of sonic travel time to the uniaxial compressive strength of US coal measure rocks. Paper presented at the 27th International Conference on Ground Control in Mining. Morgantown, WV, pp. 338–346.
Raaen, A.M., Hovem, K.A., Joranson, H., Fjaer, E., 1996. FORMEL: A Step Forward in Strength Logging. Presented at the SPE Annual Technical Conference and Exhibition, Paper presented at the SPE Annual Technical Conference and Exhibition, Denver, Colorado.
Rahman, T., Sarkar, K., 2021. Lithological Control on the Estimation of Uniaxial Compressive Strength by the P-Wave Velocity Using Supervised and Unsupervised Learning. Rock Mech. Rock Eng. 54, 3175–3191.
Rajabi, M., Tingay, M., Heidbach, O., Hillis, R., Reynolds, S., 2017. The present-day stress field of Australia. Earth-Sci. Rev. 168, 165–189.
Sabine, S., Guy, G., Gordon, N., 2012. Back analysis of roof classification and roof classification and roof support systems at Kestrel North. Paper presented at the 12th Coal Operators’
Conference, University of Wollongong & the Australasian Institute of Mining and Metallurgy, pp. 42–51.
Sharma, M.S.R., O’Regan, M., Baxter, C.D.P., Moran, K., Vaziri, H., Narayanasamy, R., 2010.
Empirical relationship between strength and geophysical properties for weakly cemented formations. J. Pet. Sci. Eng. 72, 134–142.
Singh, S.K., Raval, S., Banerjee, B., 2021. Roof bolt identification in underground coal mines from 3D point cloud data using local point descriptors and artificial neural network. Int. J.
Remote Sens. 42, 367–377.
Sopacı, E., Akgün, H., 2008. Engineering geological investigations and the preliminary support design for the proposed Ordu Peripheral Highway Tunnel, Ordu, Turkey. Eng. Geol. 96, 43–61.
Webber, T., Costa, J.F.C.L., Salvadoretti, P., 2013. Using borehole geophysical data as soft information in indicator kriging for coal quality estimation. Int. J. Coal Geol., Special issue on geostatistical and spatiotemporal modeling of coal resources 112, 67–75.
Wliwa, R., Fraser, S.J., Dickson, B.L., 2003. Application of self-organising maps to the recognition of specific lithologies from borehole geophysics. Paper presented at the 35th Sydney Basin Symposium on “Advances in the study of the Sydney Basin”, University of Wollongong, NSW.
Yagiz, S., Sezer, E.A., Gokceoglu, C., 2012. Artificial neural networks and nonlinear regression techniques to assess the influence of slake durability cycles on the prediction of uniaxial compressive strength and modulus of elasticity for carbonate rocks. Int. J. Numer. Anal.
Methods Geomech. 36, 1636–1650.
Yilmaz, I., Yuksek, G., 2009. Prediction of the strength and elasticity modulus of gypsum using multiple regression, ANN, and ANFIS models. Int. J. Rock Mech. Min. Sci. 46, 803–810.
Preprint not peer reviewed
Zoback, M.D., 2010. Reservoir geomechanics. Cambridge university press, Cambridge, UK.
Zoback, M.D., Moos, D., Mastin, L., Anderson, R.N., 1985. Well bore breakouts and in situ stress.
J. Geophys. Res. Solid Earth 90, 5523–5530.
Preprint not peer reviewed
