Targeting predictors in random forest regression

Random forest (RF) regression is an extremely popular tool for analyzing high -dimen-sional data. Nonetheless, its benefits may be lessened in sparse settings due to weak predictors, and a pre-estimation dimension reduction (targeting) step is required. We show that proper targeting controls the probability of placing splits along strong predictors, thus providing an important complement to RF's feature sampling. This is supported by simulations using finite representative samples. Moreover, we quantify the immediate gain from targeting in terms of the increased strength of individual trees. Macroeconomic and financial applications show that the bias-variance trade-off implied by targeting, due to increased correlation among trees in the forest, is balanced at a medium degree of targeting, selecting the best 5%-30% of commonly applied predictors. Improvements in the predictive accuracy of targeted RF relative to ordinary RF are considerable, up to 21%, occurring both in recessions and expansions, particularly at long horizons.(c) 2022 International Institute of Forecasters. Published by Elsevier B.V. All rights reserved.

Automated summary

Random forest (RF) regression is popular for high-dimensional data analysis, but its effectiveness in sparse settings can be weakened due to weak predictors, requiring a pre-estimation dimension reduction step. We demonstrate that proper targeting can control the placement of splits on strong predictors, complementing RF's feature sampling. Simulations using representative samples support this finding. Additionally, we quantitatively measure the immediate gain from targeting in terms of the increased strength of individual trees. Macroeconomic and financial applications show that the bias-variance trade-off involved in targeting is balanced with moderate targeting, selecting the top 5%-30% commonly applied predictors. Targeted RF significantly improves predictive accuracy compared to ordinary RF, with improvements of up to 21% observed in both recessions and expansions, particularly at long horizons.