Structured AI Socratic Homework in Higher Education: Implementation, Student Experience, and Evidence from a Pre-Registered Field Study

Background: Structured AI Socratic tutoring — in which a large language model guides students through multi-step problems using iterative questioning rather than direct answers — offers a scalable, zero-cost approach to formative homework in higher education. Existing literature on AI in education relies heavily on unstructured ChatGPT use and focuses on language disciplines, leaving structured Socratic designs in quantitative courses largely unexplored. Rigorous empirical evidence on design, implementation, and academic effectiveness remains scarce. Objectives: This pre-registered pilot study documents the design and iterative implementation of an AI Socratic homework intervention and evaluates its academic and experiential outcomes compared to a conventional homework platform. Methods: A quasi-experimental design with historical controls compared a Spring 2026 cohort using AI Socratic homework (n = 76) against a Fall 2025 cohort using a publisher platform (n = 84) in an undergraduate quantitative course, with identical exams, instructor, grader, and course materials across semesters. Mixed methods included Welch t-tests and equivalence testing for academic outcomes, pre-post surveys for student experience and AI literacy, and thematic analysis of open-ended responses. Results and Conclusions: Academic outcomes were statistically equivalent with a small positive effect favoring the AI group (d = 0.26), meeting a pre-registered practical significance threshold. Students preferred AI homework independent of cost (52%, p = .0002) and reported substantial AI comfort gains (71%). Eighty-nine percent used AI beyond required assignments, indicating spillover AI literacy development. The study's central contribution is a documented twelve-cycle prompt design and iteration framework constituting transferable implementation knowledge for instructors in any quantitative discipline.