Enhancing PE Malware Detection: A Comparative Study of Feature-Based and Image-Based Representations

Windows Portable Executable (PE) files are a common format for software applications and so they are also a primary target for malware authors. The increasing complexity of malicious software in the form of polymorphic and obfuscated malware, poses a challenge for traditional detection systems that are solely based on static or dynamic analysis. For enhancing the detection accuracy and resilience, this study explores a hybrid malware classification framework. It combines the conventional machine learning models trained on static features with image-based deep learning techniques which transform the metadata into visual representations from the features. Models such as Random Forest, CNN, and MLP are trained on PE file features that are JSON-extracted and a fine-tuned MobileNetV2 model is used for classifying malware and benign files based on their image equivalents. The proposed approach aims to use the strengths of both the structured and spatial representations to for improving the generalization and robustness in detection of malware.