From Analytic Assistant to Scientific Replicator: Testing the Process Fidelity of Generative AI in Qualitative Tourism Research

This study assesses the capacity of Generative AI to comprehend and implement systematic methodologies for research replication, addressing the crucial need for reliable knowledge transfer in the hospitality and tourism sector. We assessed GAI’s fidelity in replicating the qualitative content analysis of Loo and Leung (2018), which categorized consumer reviews using the 7P marketing mix framework. To rigorously test GAI’s adherence to the scientific process, this study employed a structured Chain-of-Thought (CoT) prompting protocol derived from the original paper’s coding scheme. The comparative performance assessment revealed that while GAI achieved only a low-to-moderate rank correlation with the original output (low output fidelity), it failed to execute the prescribed CoT protocol (high process failure), defaulting instead to an opaque, internal approach. This study’s core contribution is the empirical demonstration that GAI’s inherent “black box” logic compromises the required methodological transparency, thereby fundamentally undermining the knowledge transformation phase of research. GAI is currently an effective analytical assistant but not a trustworthy scientific replicator, posing a significant, fidelity-based barrier to its adoption for high-stakes academic research.