Providing feedback without verification: A hidden risk of trust in generative AI

Generative AI (GenAI) can produce responses that appear credible yet are inaccurate or fabricated, creating risks of flawed judgment and misinformation diffusion. Addressing these risks depends not only on system performance, but also on how users respond when GenAI outputs may be wrong. Two behaviors are critical in this regard: verifying outputs and providing feedback to GenAI. Verification helps users detect potential errors before accepting or sharing them, while feedback helps the system learn from mistakes over time. To understand these two behaviors, we examine how trust in GenAI shapes them and, upstream, how such trust originates at the system level. Using survey data from 544 users, we find that trust has a dual effect: it increases users' willingness to provide feedback, but reduces their tendency to verify outputs. This duality highlights a hidden risk: as trust increases, users may reinforce existing errors in the system by providing unverified feedback (e.g., clicking “thumb up” for an inaccurate response). We further find that GenAI's versatility, creativity, and multimodality strengthen trust, suggesting that the very features that make these systems appealing may also contribute to this hidden risk. These findings offer implications for designing interventions that calibrate trust and support the responsible governance of GenAI technology.