TransForge: A Genetic Algorithm Framework for Cross-Category Evaluation of Endpoint Detection Robustness to Code Transformations

Endpoint protection systems increasingly rely on a combination of signature-based and behavioral detection mechanisms, yet their robustness under systematic code transformation remains insufficiently understood. This paper presents a multi-category evaluation of endpoint detection robustness under automated, semantic-preserving code transformations across diverse execution variants. We introduce TransForge, a generalized transformation framework designed to generate functionally equivalent execution variants for controlled robustness assessment across heterogeneous artifact categories and programming environments. Building on our prior work, ShellForge, which focused on a single artifact class, TransForge extends this approach to support multi-category analysis through a modular transformation pipeline and an evolutionary strategy that enables non-deterministic variant generation. Using a dataset of 75 base samples spanning six execution categories and four programming languages, we conduct controlled experiments to evaluate how endpoint detection systems respond to systematically generated variants under consistent conditions. Across the evaluated corpus, 53 of 75 evolved variants (70.7%) achieved zero VirusTotal detections across 76 engines, while 94% of variants preserved full functional correctness. Detection outcomes varied substantially by category: staged downloaders exhibited a mean of 8.75 alerts while credential access, surveillance, and cross-site execution variants produced zero alerts under identical evolutionary configuration, with all six categories converging uniformly at generation 4. Statistical analysis using ANOVA, Mann–Whitney U, and Wilcoxon signed-rank tests confirm that these differences are statistically significant ( p < 0.001 ) and practically meaningful (Cohen’s d > 0.8 ). The findings highlight coverage gaps in both signature-based and behavioral detection pipelines when faced with semantic-preserving transformations, motivating the development of robustness-aware evaluation frameworks and detection pipelines that leverage behavioral correlation and adaptive analysis beyond static signature matching.