Counting the User Into the Attack Surface: DUMA-Bench Reveals Agent Security as an Emergent Property

A paper posted to arXiv on September 21 (arXiv:2609.24662) challenges a default assumption in agent security evaluation: most benchmarks assume passive users and static control, while in real deployments both user and agent influence the shared environment state. The authors introduce DUMA-Bench, a security benchmark under dual-control multi-agent interaction, and report a jarring number: introducing dual-control interaction raises attack success rate from 26.9% to 41.1%.

Start with the method. DUMA-Bench extends tau-squared-bench, covering eight vulnerability classes including RAG poisoning, cross-agent manipulation, and unsafe output handling — this is no longer the static test of "can the model say something bad," but "can the agent, while using real tools, be led astray by bad data in its environment." It evaluates 14 models from five model families (OpenAI, Anthropic, DeepSeek, Qwen, and more) across eight domains and multiple user-behavior regimes. The key variable is dual control: the user side is allowed to change environment state too, simulating real-world cases where a user clicks a wrong link, pastes malicious text, or is phished into feeding hostile instructions to the agent.

The result is bad news with a reminder attached. Attack success jumps from 26.9% to 41.1% — the 14.2-point rise is not the model getting weaker; it is the evaluation getting more realistic. The paper's core claim: agent security is not a property of the model alone but an emergent property of the interaction among model, user, and environment. That sounds like academic boilerplate, but it is concrete in engineering terms: an agent that is safe in static evaluation may be anything but safe in the hands of a real user who clicks around; a security team that only tests the model without changing user-behavior entry points has fixed only half the problem.

The paper's value is adding a missing evaluation layer. Red-teaming and jailbreak tests measure model resistance to malicious inputs; DUMA-Bench measures security in deployed configurations — including scenarios where the user drags the agent into trouble. The limits must be stated: attack success rate is an aggregate across the benchmark, and the paper does not break down per-domain and per-family differences; the task distribution of tau-squared-bench bounds coverage; dual control is more realistic, but "user-behavior regimes" are themselves simulated, so the real-world distribution of user malice remains uncertain. Overall it offers a new measurement layer for agent security, not the final answer.

Three implications are worth spelling out. First, for safety teams, the result argues that agent security review must include the integration layer — the tools, permissions, and data sources the agent touches — not just the model card, because the same model behaves differently across environments. Second, for benchmark builders, the 14.2-point swing is a warning that any static agent-safety benchmark that reports a single attack-success number is understating risk by construction; the paper's user-behavior regimes are the beginning of that correction, not the end. Third, for product teams shipping agentic features to consumers, the finding reframes "unsafe user" from an edge case to a design input: if the agent is going to act on whatever it reads, then the product must assume the user will occasionally feed it garbage, and build confirmation boundaries accordingly. None of this invalidates model-level red-teaming; it just means the field now has a way to measure the gap between a model that passes static tests and an agent that survives contact with real users.