Preventing overfitting when using tree-based methods for mapping hydrothermal favorability

Abstract

Ensemble tree-based algorithms are robust tools for estimating sparsely distributed resources with non-linear dependencies (e.g., hydrothermal systems). These algorithms naturally accommodate the threshold conditions necessary to enable and support hydrothermal systems (e.g., having sufficient heat and permeability) and are simpler than many other non-linear machine learning strategies (e.g., artificial neural networks), which is an advantage when working with few labeled examples from which to learn. In previous work, we used eXtreme Gradient Boosting (XGBoost) to produce regional prediction and uncertainty maps of hydrothermal favorability; however, recent studies suggest that, even when properly applied, XGBoost has some risk of overfitting when there are few labeled examples from which to learn. To evaluate overfitting when constructing hydrothermal favorability maps with tree-based methods, we compare XGBoost with Extremely Randomized Trees (ExtraTrees), another ensemble tree-based algorithm that has the potential to underfit when using few labeled examples. We hold all other modeling parameters constant, resulting in two contrasting favorability maps of conventional geothermal resources for the Great Basin. Our results indicate that ExtraTrees demonstrably reduces overfitting compared with XGBoost. After considering overall performance, we conclude that ExtraTrees provides a more suitable modeling approach than XGBoost for the purposes of conventional hydrothermal resource assessments.