(Human) Attention Is (Still) All You Need: Human Oversight Makes Ai-Assisted Social Science

Large language models (LLMs) are increasingly delegated tasks once reserved for trained researchers: generating hypotheses, choosing specifications, drafting conclusions. Whether this delegation produces trustworthy science is not solely a technical question about model capability; it also depends on how cognitive labour is structured between humans and machines. We study this problem by organising an AI-assisted research workflow around behavioural-science principles, pre-commitment, decision sequencing, accountability, and attention allocation, rather than by directly measuring human psychology at the gates. We propose that reliability in AI-assisted research is a property of decision architecture: the placement, sequencing, and binding force of the choices that humans and AI components are each permitted to make. In a pre-specified 2 × 4 factorial experiment (N = 280 complete research runs across four datasets), an unconstrained multi-agent baseline produced critical failures in 72% of runs; the same underlying model and the same agent decomposition, with identical prompts on the reasoning agents shared by both arms, failed in 16% once organised by three architectural commitments (LLMs restricted to reasoning, data and estimation executed deterministically, three human decision gates; Fisher’s exact p