Evaluating configurational comparative methods for analysing complex causal relationships in implementation science

Background Implementation science often involves complex causal relationships shaped by multiple interacting conditions (e.g., barriers, enablers, and implementation strategies) and co-occurring outcomes. Traditional regression-based methods, which typically analyse quantitatively measured implementation constructs, may be limited in identifying such complexity. Configurational comparative methods (CCM), including Qualitative Comparative Analysis (QCA), Coincidence Analysis (CNA), and Combinational Regularity Analysis (CORA), offer alternative approaches, but their comparative performance in implementation science remains unclear. Methods This study compared QCA, CNA, and CORA using three published public health datasets and a large-scale simulation, covering a single-outcome dataset on HPV vaccination uptake, and two multi-outcome datasets, one examining diabetes and depression and the other examining traffic accidents and self-inflicted injuries. The simulation generated 1,000 causal structures across three sample sizes and five noise levels, resulting in 15,000 datasets to simulate different implementation scenarios. Method performance was evaluated using structural recovery metrics (error-freeness, correctness, completeness, no-model rate) and solution quality metrics (consistency and coverage). Results In the single-outcome analysis, all three methods produced identical solutions. In multi-outcome settings, CORA uniquely identified shared causal structures (shared combinations of conditions associated with more than one outcome), while QCA and CNA used separate analyses. In the simulation study, CNA achieved the highest error-freeness (up to 1.00) but frequently failed to produce solutions. QCA achieved perfect correctness and completeness (1.00) at n = 1000 under 0% noise. CORA maintained high correctness under low-noise conditions across sample sizes (0.83–0.91) but was limited by computational constraints under high-noise conditions. Consistency and coverage revealed different performance patterns across methods. Conclusions No single method is universally superior. Method selection should depend on sample size, data quality, and outcome multiplicity. CORA is advantageous for multi-outcome analysis, QCA is most productive for larger datasets, and CNA conservatively prioritises avoiding incorrect conclusions. These findings provide practical guidance for implementation scientists in selecting appropriate CCM methods based on sample size, data quality, and whether multiple outcomes are being analysed.