Open Access
August 2015 Consistency of random forests
Erwan Scornet, Gérard Biau, Jean-Philippe Vert
Ann. Statist. 43(4): 1716-1741 (August 2015). DOI: 10.1214/15-AOS1321

Abstract

Random forests are a learning algorithm proposed by Breiman [ Mach. Learn. 45 (2001) 5–32] that combines several randomized decision trees and aggregates their predictions by averaging. Despite its wide usage and outstanding practical performance, little is known about the mathematical properties of the procedure. This disparity between theory and practice originates in the difficulty to simultaneously analyze both the randomization process and the highly data-dependent tree structure. In the present paper, we take a step forward in forest exploration by proving a consistency result for Breiman’s [ Mach. Learn. 45 (2001) 5–32] original algorithm in the context of additive regression models. Our analysis also sheds an interesting light on how random forests can nicely adapt to sparsity.

Citation

Download Citation

Erwan Scornet. Gérard Biau. Jean-Philippe Vert. "Consistency of random forests." Ann. Statist. 43 (4) 1716 - 1741, August 2015. https://doi.org/10.1214/15-AOS1321

Information

Received: 1 May 2014; Revised: 1 February 2015; Published: August 2015
First available in Project Euclid: 17 June 2015

zbMATH: 1317.62028
MathSciNet: MR3357876
Digital Object Identifier: 10.1214/15-AOS1321

Subjects:
Primary: 62G05
Secondary: 62G20

Keywords: Additive model , consistency , Dimension reduction , random forests , Randomization , Sparsity

Rights: Copyright © 2015 Institute of Mathematical Statistics

Vol.43 • No. 4 • August 2015
Back to Top