Anthropic Study: Why Analysts Adopt Their AI Agent's Random Opinion in 86% of Cases – and Who Really Makes the Call

Imagine your controller prepares a decision paper with the help of an AI agent: should the offer be judged by margin per order or margin per working hour? Should the comparison cover only existing customers or all of them? The agent proposes one variant, justifies it neatly, and the paper reaches you as "my analysis". The question that matters: whose judgment did you actually read?

Anthropic's alignment team examined exactly this in a study published on October 27, 2026, titled "How AI agents shape experts' judgment calls" (authors: Amaç Herdağdelen, Hailey Joren, Alex Tamkin). Experienced professionals adopted their agent's lean in 86 percent of the decisions without noticing, and they felt no less confident than usual.

This article separates what the study shows from our own reading. One clarification first: this is a research study, not a product or a feature you can switch on or off. The finding applies wherever a person lets an agent prepare a decision, regardless of vendor or region.

What Anthropic published on October 27, 2026

The experiment recruited 61 professional analysts, all experienced in causal inference. They were asked to advise a fictional farming cooperative on whether to expand or discontinue a voluntary advisory program, using an AI agent: Claude Opus 4.8 at high effort, running in Claude Code inside VS Code. Eleven people chose not to use the agent, so 50 were analyzed.

Four open decisions came up along the way: the outcome measure (yield per hectare or per operating cost), the comparison method (difference-in-differences or statistical adjustment), the farms covered (enrolled farms or the full membership) and the weighting (equal or by producer size). A secret coin flip decided which option the agent would favor on each decision, delivered through hidden system-prompt instructions. The agent therefore had no professional reason for its lean; it was pure chance. With no influence, you would expect about 50 percent agreement.

The numbers

"Went with their agent's lean 86% of the time." – Anthropic, October 27, 2026

• 86 percent across the four decisions, between 81 and 92 percent per decision.
• Analysts who had stated a preference beforehand switched in 60 percent of cases when the agent disagreed, versus 6 percent when it agreed. The authors stress that this rests on only 38 cases.
• Those who followed the lean on all four decisions reported an average confidence of 71, the others 73.
• 52 percent rated the recommendation as more their own than the agent's; 77 percent repeated the agent's arguments, but only 15 percent credited the agent.

The last point is very human: a well-formulated argument gets adopted and soon feels like your own. In a company, a paper saying "I recommend option B" does not reveal whether the person chose B or was nudged toward it.

What the authors themselves qualify

• "We deliberately strengthened the agents' lean", so real-world effects may be smaller.
• "We studied one task, in one field, with one model."
• Results are averages; each participant's view before the agent spoke was not recorded throughout.
• The study recorded which option was chosen, not how strongly it was preferred.
• The AI and non-AI groups are not directly comparable, as the non-AI group was less experienced.

Our reading: the study does not prove that 86 percent of your decisions are steered by an agent. It shows that under lab conditions with an amplified lean, a strong effect can occur that people do not notice. That is reason enough to review your own processes, not proof of a specific magnitude in your company.

Availability: a study, not a product

Nothing is being rolled out, tested in beta or restricted by region, so the availability question does not apply to the DACH region. The finding is not tied to one tool: a convincing assistant shifting experts' judgment is a mechanism, although the study does not show whether other models act equally strongly. What is coming to companies is not a new feature but a responsibility: the more routinely agents contribute to analyses, offers and decision papers, the more important it becomes to say where the final decision deliberately sits with a person.

What it means for a company with 10 to 250 employees

• Offer calculation: the agent chooses whether overhead is allocated as a flat rate or by effort. Both are defensible, but they lead to different prices. A reader sees the result, not the fork.
• Controlling: the agent decides which period to compare and which outliers to drop. These are judgments, not calculation steps.
• Supplier or customer selection: the weighting of price, delivery time and payment behavior decides the ranking, exactly the kind of open decision that was influenced in the study.

The authors do not argue against using AI. They write that the split between people and machines "can be something we choose, instead of something we drift into". For a managing director, that means: not fewer agents, but a clear allocation.

Which agent: the Data Analyst

On amazing-agents.com, the Data Analyst is one of the eight agents; it provides dashboards and alerts on anomalies and keeps KPIs current without spreadsheet work. It sits exactly where the study applies: an agent that prepares data inevitably makes pre-decisions about what is shown and how it is compared. As a general principle we describe that every agent runs within a defined frame and critical actions need your approval. That an approval only works if the approver sees which options existed and which one the agent favored is our own conclusion from the study, not part of the product description.

Five steps you can prepare this week

1. Inventory decisions. List five to ten decisions where an agent already contributes, and who is accountable for each.
2. Make forks visible. Require every paper to name the open method questions: "measure A or B chosen because ...".
3. Own view first. Ask the accountable person to note their view before reading the agent's proposal.
4. Ask for alternatives. Instruct the agent to give the strongest case for both sides rather than one recommendation.
5. Reserve decisions for people. Decide which decisions always need an informed human judgment, such as large sums, personnel matters or contract terms.

How we approach the audit

This describes our approach and offer, not results at other companies. We take stock of where agents prepare papers, reports or selections; classify each place as pure calculation, defensible method choice or strategic decision; agree approval rules covering what the approver must see; and propose a small recurring spot check in which a person re-checks a paper without the agent's suggestion.

An agent with limits and an approval step

An agent without limits delivers results where you cannot tell whose judgment you are reading. An agent with limits and an approval step delivers the same results but shows the forks, names its lean and leaves the decision to a person who makes it deliberately. That is not a brake on AI use; once responsibility is clear, you can hand more tasks to agents without decisions shifting in the background.

Source: Anthropic, "How AI agents shape experts' judgment calls", Amaç Herdağdelen, Hailey Joren, Alex Tamkin, October 27, 2026, anthropic.com/research/agents-judgment-calls. All figures and quotes come from this source; the Mittelstand reading is our own.