Evaluating Prompt Engineering Strategies for Few-Shot Cyber Threat Intelligence Entity and Relation Extraction from Multi-Source Reports

The proliferation of multi-source cyber threat intelligence reports---spanning vulnerability databases, government advisories, vendor analyses, and open-source feeds---has outpaced the capacity of human analysts to extract structured knowledge about adversary tactics, techniques, and procedures. While large language models present a promising avenue for automating this extraction under low-resource conditions, no systematic empirical comparison of prompt engineering strategies exists for the cyber threat intelligence domain. This study evaluates six prompt engineering strategies---zero-shot, one-shot, three-shot, five-shot, retrieval-augmented five-shot, and chain-of-thought five-shot---across four publicly available cyber threat intelligence named entity recognition datasets (DNRTI, CyNER, AnnoCTR, APTNER) and one relation extraction corpus, using GPT-4, GPT-3.5-turbo, and Llama-3-70B. The retrieval-augmented five-shot strategy achieves the highest named entity recognition F1 of 0.753 on CyNER with GPT-4, narrowing the gap with the fine-tuned SecureBERT baseline to 2.8 percentage points. Chain-of-thought prompting yields the lowest expected calibration error (0.108), suggesting its value for uncertainty-aware intelligence triage. Cross-source extraction variance reaches 12.2 F1 points between the easiest and hardest corpora, underscoring the challenge of heterogeneous intelligence fusion. These findings offer actionable guidance for deploying prompt-based extraction in operational threat intelligence pipelines aligned with the NIST Cybersecurity Framework and national cyber defense priorities.