





















Abstract:Oblique decision trees combine the transparency of trees with the power of multivariate decision boundaries, but learning high-quality oblique splits is NP-hard, and practical methods still rely on slow search or theory-free heuristics. We present the Hinge Regression Tree (HRT), which reframes each split as a non-linear least-squares problem over two linear predictors whose max/min envelope induces ReLU-like expressive power. The resulting alternating fitting procedure is exactly equivalent to a damped Newton (Gauss-Newton) method within fixed partitions. We analyze this node-level optimization and, for a backtracking line-search variant, prove that the local objective decreases monotonically and converges; in practice, both fixed and adaptive damping yield fast, stable convergence and can be combined with optional ridge regularization. We further prove that HRT's model class is a universal approximator with an explicit $O(\delta^2)$ approximation rate, and show on synthetic and real-world benchmarks that it matches or outperforms single-tree baselines with more compact structures.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2602.05371 [cs.LG] |
| (or arXiv:2602.05371v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.05371 arXiv-issued DOI via DataCite |
From: Jun Xu [view email]
[v1]
Thu, 5 Feb 2026 06:49:01 UTC (441 KB)
[v2]
Mon, 9 Mar 2026 16:55:02 UTC (441 KB)
[v3]
Thu, 30 Apr 2026 17:23:26 UTC (442 KB)
此内容由惯性聚合(RSS阅读器)自动聚合整理,仅供阅读参考。 原文来自 — 版权归原作者所有。