IDEAS home Printed from https://ideas.repec.org/a/eee/ejores/v324y2025i2p567-579.html
   My bibliography  Save this article

k-Tree: Crossing sharp boundaries in regression trees to find neighbors

Author

Listed:
  • Tian, Xuecheng
  • Wang, Shuaian
  • Zhen, Lu
  • Shen, Zuo-Jun (Max)

Abstract

Traditional classification and regression trees (CARTs) utilize a top-down, greedy approach to split the feature space into sharply defined, axis-aligned sub-regions (leaves). Each leaf treats all of the samples therein uniformly during the prediction process, leading to a constant predictor. Although this approach is well known for its interpretability and efficiency, it overlooks the complex local distributions within and across leaves. As the number of features increases, this limitation becomes more pronounced, often resulting in a concentration of samples near the boundaries of the leaves. Such clustering suggests that there is potential in identifying closer neighbors in adjacent leaves, a phenomenon that is unexplored in the literature. Our study addresses this gap by introducing the k-Tree methodology, a novel method that extends the search for nearest neighbors beyond a single leaf to include adjacent leaves. This approach has two key innovations: (1) establishing an adjacency relationship between leaves across the tree space and (2) designing novel intra-leaf and inter-leaf distance metrics through an optimization lens, which are tailored to local data distributions within the tree. We explore three implementations of the k-Tree methodology: (1) the Post-hoc k-Tree (Pk-Tree), which integrates the k-Tree methodology into constructed decision trees, (2) the Advanced k-Tree, which seamlessly incorporates the k-Tree methodology during the tree construction process, and (3) the Pk-random forest, which integrates the Pk-Tree principles with the random forest framework. The results of empirical evaluations conducted on a variety of real-world and synthetic datasets demonstrate that the k-Tree methods have greater prediction accuracy over the traditional models. These results highlight the potential of the k-Tree methodology in enhancing predictive analytics by providing a deeper insight into the relationships between samples within the tree space.

Suggested Citation

  • Tian, Xuecheng & Wang, Shuaian & Zhen, Lu & Shen, Zuo-Jun (Max), 2025. "k-Tree: Crossing sharp boundaries in regression trees to find neighbors," European Journal of Operational Research, Elsevier, vol. 324(2), pages 567-579.
  • Handle: RePEc:eee:ejores:v:324:y:2025:i:2:p:567-579
    DOI: 10.1016/j.ejor.2025.02.031
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0377221725001547
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.ejor.2025.02.031?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ejores:v:324:y:2025:i:2:p:567-579. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/eor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.