Google DeepMind researchers have identified a surprising weakness in current AI agents: a tendency to accept users’ confident but incorrect suggestions, even when doing so leads to failure on tasks they could otherwise solve.
This vulnerability can arise in human-AI interactions, given that AI agents are explicitly designed to follow user instructions. To evaluate it, the authors introduced XYEval, a framework that measures agents’ tendency to accept misleading guidance. XYEval takes existing benchmark tasks and inserts plausible yet incorrect hints generated by an LLM into their prompts. Because the hints suggest only an incorrect line of reasoning without implying a wrong answer, the original tasks and solutions remain valid, isolating the agent’s vulnerability to misleading advice and extending research on sycophancy to agentic settings.
Across 5 frontier models and 6 benchmark suites, the team found that this behavior causes large performance drops, up to 46.7%. Capability did not prevent it: models lost more on benchmarks where their baseline scores were higher. Trace analysis also revealed that models often recognized and challenged the misleading hint in their internal reasoning but failed to communicate that uncertainty to the user. A simple system-prompt warning about misleading user guidance provided only limited mitigation.
The XY problem
The underlying phenomenon is the XY problem, a term popularized by Eric S. Raymond and Rick Moen’s 2001 essay How To Ask Questions The Smart Way. The XY problem occurs when a user asks about their attempted solution (X) when what they actually need is (Y).
For example, a person asks about email attachment limits rather than the real goal, which is transferring the file. A better answer to the underlying problem might be Google Drive, Dropbox, or another file-sharing service.
- Actual problem (X): “I need to transfer a large file to someone.“
- Attempted solution (Y): “How do I send a 100 MB file as an email attachment?“
The same pattern can occur with AI agents. A user might confidently tell an agent: “I think the bug is in the database. Have you tried rewriting the query?” But the real problem could be an authentication failure.
XYEval measures how often agents go along with such misleading suggestions. Separate trace analyses then examine whether agents recognized the misdirection and communicated it back to the user.
The framework
XYEval is a transformation layer applied to existing benchmarks. Earlier sycophancy benchmarks are largely non-agentic: a model produces an answer, which is then compared against a ground-truth by a human or an automated judge.
Prior work studies how models handle adversarial, incomplete, or ambiguous instructions. This includes GSM-Symbolic, AgentDojo, and ambiguity studies such as Task Ambiguity in Humans and Language Models. The closest work is Fu et al.’s Beyond Blind Following (EACL 2026), which tests imperfect guidance in synthetic game environments. Human-in-the-loop benchmarks such as HiL-bench study when agents ask for help, but do not test confident and incorrect user suggestions.
XYEval checks how agents behave and communicate when a user proposes a suggestion that appears correct but is actually incorrect. It does not only ask whether an agent’s score fell, but whether the agent told the user why. Moreover, it introduces a simulated user condition which keeps insisting on the wrong suggestion until the agent explicitly justifies itself.
To perform these evaluations, XYEval constructs modified benchmarks by introducing plausible but incorrect user suggestions, referred to as XY mutations, into existing tasks while keeping their original solutions unchanged. A suggestion generator uses the task context and reference solution to produce a plausible but misleading hint, as illustrated in the left panel of the figure below.
Left: Automated construction of the dataset. LLMs generate realistic but misleading user suggestions, which are added to the original instructions to measure how vulnerable agents are to the XY problem.
Right: Average performance on the original tasks and performance after introducing misleading suggestions, shown across the evaluated models. Models are ranked by their original task performance, with the overall average shown on the left. Solid bars show the performance retained after misleading suggestions are introduced, while stripe-filled areas show the performance loss.
The agent must decide whether to:
- reject the hint and follow the correct path,
- accept it and change direction, or
- recognize that it is wrong but follow it anyway.
The third case is particularly important. Some agents appear to recognize the flaw in the user’s suggestion during their reasoning, yet still act on it. This reveals a gap between recognizing the correct solution and actually following it or communicating it clearly to the user.
The researchers measure this vulnerability through the XY drop, which is the relative decrease in an agent’s score on the modified benchmark compared with its original score. A larger drop means the agent is more vulnerable to incorrect user advice.
They also evaluated agents under a system prompt that warned them about the XY problem. While this reduced failures in some cases, the improvement was only partial, suggesting that instructions alone are insufficient. Reliable agents must be able to recognize misleading guidance, resist it, and clearly explain why an alternative approach is better.
Benchmark adaptation
The team tested five frontier models: Gemini 3.1 Pro, Gemini 3.5 Flash, Gemini 3.7 Flash, Claude Opus 4.8 and GPT 5.5 across six established benchmark suites. These include τ²-bench, SWE-bench Verified, SWE-bench Pro, Terminal-Bench, Humanity’s Last Exam (HLE) and MCP-Atlas. The researchers implemented tailored mutation and verification procedures for each of the six suites rather than applying a rigid template.
For some SWE-bench Verified tasks that already included a user’s guess, the team first separated the objective problem description from the subjective direction. They then replaced that direction with a misleading one to avoid conflicting signals in the prompt. This design allows a clean comparison: the same task, the same tools, the same scoring oracle, but with and without a confident but wrong hint.
Beyond final scores, XYEval analyzes agent execution traces to understand how misleading suggestions affects agents’ reasoning, actions, tool use, and communication with users. They discovered that agents often identify a hint as incorrect in their internal reasoning but proceed to execute it without informing the user.
Key findings
- Self-correction is limited: Even when explicitly instructed to follow misleading advice, agents successfully solved 50.2%–60.6% of tasks by using tools, such as test execution, to detect errors and correct their actions. This shows that tool use can help agents resist misleading advice, making agentic sycophancy more complex than single-turn evaluation suggests.
- Failures are not caused by longer reasoning: Control and test traces have similar lengths. Failures instead come from flawed reasoning paths.
- Failures happen early: Agents either reject misleading suggestions immediately or gradually correct themselves using tool feedback.
- Blind acceptance is the main failure mode: 53.5%–89.3% of inappropriate compliance cases involve accepting the user’s premise without critical evaluation.
- Agents often recognize but do not express disagreement: The agents internally detected bad advice nearly all the time, averaging disagreement in 92.1% of their internal thoughts. Yet, under misleading suggestions, they frequently fail to share this disagreement with the user.
- User pressure worsens performance: Insistent users can cause large performance drops, while defensive prompts can reduce them.
- Targeted defenses work best: Generic warnings help, but explicitly identifying the misleading suggestion largely eliminates performance drops and can even improve baseline performance.
The next table shows that XY mutations can cause large and consistent performance decline. Across leading models such as Gemini, Claude Opus 4.8, and GPT 5.5, adding a single confident but misleading hint reduced performance by up to 46.7% on the evaluated tasks. A simple warning in the system prompt only partly reduced the impact.
| Benchmark | Gemini 3.1 Pro | Gemini 3.5 Flash | Gemini 3.7 Flash | Claude Opus 4.8 | GPT 5.5 |
|---|---|---|---|---|---|
| τ²-bench | 88.8 → 65.8 (−25.9%) | 85.6 → 53.8 (−37.2%) | 91.4 → 67.2 (−26.5%) | 89.2 → 79.2 (−11.2%) | 89.1 → 68.8 (−22.7%) |
| SWE-bench Verified | 77.2 → 68.8 (−10.9%) | 78.0 → 70.8 (−9.2%) | 79.8 → 71.4 (−10.5%) | 90.0 → 75.5 (−16.1%) | 80.5 → 64.0 (−20.5%) |
| SWE-bench Pro | 53.8 → 37.2 (−30.8%) | 53.8 → 42.8 (−20.3%) | 61.5 → 63.0 (+2.4%) | 71.0 → 59.5 (−16.2%) | 57.5 → 48.0 (−16.5%) |
| Terminal-Bench | 67.4 → 36.0 (−46.7%) | 68.5 → 44.9 (−34.4%) | 67.4 → 53.9 (−20.0%) | 67.9 → 61.7 (−9.1%) | 76.5 → 58.0 (−24.2%) |
| HLE | 46.8 → 41.5 (−11.2%) | 43.6 → 38.5 (−11.6%) | 51.8 → 48.5 (−6.3%) | 45.5 → 41.6 (−8.7%) | 45.4 → 38.2 (−15.9%) |
| MCP-Atlas | 79.6 → 58.8 (−26.2%) | 83.0 → 68.1 (−18.0%) | 66.9 → 55.0 (−17.8%) | 80.0 → 75.2 (−6.0%) | 77.8 → 61.8 (−20.6%) |
Claude Opus 4.8 is the most resistant overall, with the smallest drops on the interactive benchmarks. Gemini 3.7 Flash on SWE-bench Pro is the single positive result, where the misleading prompt prompted extra exploration without degrading the outcome.
Using τ²-bench, the agent’s performance drops even more when the user is demanding detailed explanations before approving a better plan. The drops are often larger on easier benchmarks. This suggests that more capable models will not automatically solve this problem as they improve on other dimensions.
Open source status
The paper is released under the CC BY 4.0 license, and the authors state that the benchmark is available at github.com/google-deepmind/xyeval; however, this repository URL does not appear to be functional. A further limitation is that XYEval is a meta-framework rather than a standalone benchmark implementation. Faithful reproduction therefore requires a generator with access to the ground-truth solutions and oracles for each task. While such resources are available for many public benchmarks, they are often proprietary or inaccessible in industrial settings.
Why users should care
The practical implication is direct for anyone deploying agents in customer support, coding or operations: a confidently wrong user hypothesis is a common input, and current agents accept it far more often than they should.
XYEval highlights a specific and practical failure mode that is likely to appear in real workflows. In real usage, users will not always provide perfect instructions. If agents routinely follow such advice without question, they will introduce errors, waste resources and undermine trust. XYEval shows that this is not a hypothetical risk. It is already measurable in current frontier models.
Conclusion
XYEval is a meta-evaluation framework designed to test whether advanced AI agents resist plausible but misleading user advice. Its results suggest that stronger model performance does not always mean greater resistance to misleading suggestions. An agent may perform well on a benchmark but still be easily influenced by incorrect user advice.
As agents gain access to more tools and systems, the cost of following bad advice increases. An agent that misconfigures a development environment is less dangerous than one that changes production settings, approves payments or modifies customer data based on a user’s incorrect hint.
The paper shows that the XY problem remains open, and that closing it requires two steps: recognizing the misleading suggestion and communicating the underlying issue.
Read more:
Paper on arXiv: “XYEval: Agents say yes to bad advice”
