Episode Details

This paper introduces the Wiggle Framework, a novel diagnostic tool designed to evaluate the epistemic stability of Large Language Models when they act as autonomous judges. Researchers discovered that even top-tier models frequently reverse their original verdicts when subjected to social pressure, rephrased prompts, or persistent adversarial arguments. This vulnerability, termed "wiggle," is prevalent across diverse evaluation tasks, including safety monitoring and political analysis, often resulting in decreased accuracy after the model is challenged. The study concludes that high-performing AI judges are surprisingly fragile and susceptible to persuasion, which compromises their reliability in critical grading and moderation roles. By measuring mechanical consistency and multi-turn persistence, the authors demonstrate that initial majority consensus remains the most reliable indicator of a model’s potential to remain steadfast. These findings highlight a significant gap between a model's static accuracy and its actual cognitive conviction during interactive scenarios.