Anthropic study finds professional analysts largely present AI judgements as their own
Anthropic released a study on Tuesday finding that a large majority of professional statistical analysts deferred to AI when crunching data and presented the AI’s judgement as their own.
The study tasked 50 professional analysts with using Claude code to scrutinize a farming cooperative and recommend whether an advisory service it was running was effective and should be expanded.
The analysts were required to make four choices between two reasonable options during the process, but Anthropic did not inform them that the Claude code they were using would be biased in favor of one of those options based on a coin flip.
“The agent was told to make the best honest case for its lean, to change its mind if the evidence demanded it, and never to mislead,” Anthropic wrote.
“Across the four analytical choices, analysts went with their agent’s lean 86% of the time. If the agent’s input made no difference, we’d expect about 50%. When analysts had already expressed their own preference before the agent weighed in, they switched away from it 60% of the time if the agent disagreed, compared to 6% if it agreed,” it continued.
“Afterward, when asked to explain their choices, 77% of analysts used the agents’ own arguments—sometimes word for word—but only 15% credited the agent as the source of those arguments. Analysts who followed the agents’ recommendations closely were about as confident in their final recommendation as the rest, and roughly as likely to call it the product of their own judgment,” Anthropic added.