Enhanced Data-Driven Machine Learning Models for Predicting Total Organic Carbon in Marine–Continental Transitional Shale Reservoirs

Natural gas, as a sustainable and cleaner energy source, still holds a crucial position in the energy transition stage. In shale gas exploration, total organic carbon (TOC) content plays a crucial role, with log data proving beneficial in predicting total organic carbon content in shale reservoirs. However, in complex coal-bearing layers like the marine–continental transitional Shanxi Formation, traditional prediction methods exhibit significant errors. Therefore, this study proposes an advanced, cost- and time-saving deep learning approach to predict TOC in marine–continental transitional shale. Five well log records from the study area were used to evaluate five machine learning models: K-Nearest Neighbors (KNNs), Random Forest (RF), Gradient Boosting Decision Tree (GBDT), Extreme Gradient Boosting (XGB), and Deep Neural Network (DNN). The predictive results were compared with conventional methods for accurate TOC predictions. Through K-fold cross-validation, the ML models showed superior accuracy over traditional models, with the DNN model displaying the lowest root mean square error (RMSE) and mean absolute error (MAE). To enhance prediction accuracy, δR was integrated as a new parameter into the ML models. Comparative analysis revealed that the improved DNN-R model reduced MAE and RMSE by 57.1% and 70.6%, respectively, on the training set, and by 59.5% and 72.5%, respectively, on the test set, compared to the original DNN model. The Williams plot and permutation importance confirmed the reliability and effectiveness of the enhanced DNN-R model. The results indicate the potential of machine learning technology as a valuable tool for predicting crucial parameters, especially in marine–continental transitional shale reservoirs lacking sufficient core samples and relying solely on basic well-logging data, signifying its importance for effective shale gas assessment and development.