Much of the contemporary research into artificial intelligence alignment investigates model sycophancy—the tendency of large language models to bend toward user preconceptions to avoid disagreement. However, in a rigorous empirical study published on arXiv on October 22, 2026, researchers Deepthika S, Pawni Yadav, and B. Sankar investigated the inverse dynamic: when an artificial assistant is confident, articulate, and completely mistaken, how readily do human users defer to the machine?
To evaluate this question, the researchers conducted a within-subjects experimental trial involving 32 participants who learned five scientific topics by conversing with conversational assistants built on GPT-5-mini. Unknown to the participants, the researchers systematically altered the assistants' reference briefings, replacing 0%, 25%, 50%, 75%, or 100% of factual statements with plausible falsehoods. Across all conditions, the assistants were programmed to communicate with unwavering confidence and never concede error under user interrogation.
The findings revealed an alarming vulnerability in human cognitive calibration. Across 104 surprise comprehension quizzes comprising 1,040 answers, participant scores dropped precipitously from 7.4 out of 10 under baseline conditions down to 2.9 out of 10 under full falsification. When presented with a multiple-choice question on a falsified topic, participants selected the planted hallucination 60% of the time, compared to just 11% for correctly taught controls—representing an odds ratio of 12.5. Detailed transcript coding demonstrated that whenever an assistant stated a falsehood, human subjects absorbed it approximately 65% of the time across all corruption tiers.
Crucially, the experiment proved that humans adopt artificial hallucinations with genuine conviction rather than cautious doubt. Participants rated their confidence in incorrect answers at an average of 8.4 out of 10. Consequently, the statistical discrimination of subjective confidence for factual correctness collapsed from an Area Under the ROC Curve (AUROC) of 0.70 down to pure chance (0.50). Despite this epistemic degradation, post-task user trust scores remained virtually unchanged. Introducing a baseline-corrected 'Human Deference Index', the authors concluded that reliability must become an explicit, claim-by-claim visual property of AI interfaces rather than trusting human users to detect confident errors.