Overview
AI agents are beginning to transform language models from conversational tools into actors capable of planning, delegating, purchasing, communicating, coding, and eventually running scientific experiments. Nenad Tomašev argues that an agent is essentially a language model placed inside a harness that observes the world, uses tools, and chains actions with greater autonomy. Coding currently offers the clearest success case because software tasks are formalizable and testable, whereas real-world actions are often subjective, irreversible, or difficult to verify. Even capable agents therefore require engaged human oversight: imperfect accuracy compounds across complex workflows, while repeated success can create automation bias. Multi-agent systems introduce further challenges, including poor task decomposition, unreliable delegation, malicious web environments, prompt injection, collusion, and cognitive monoculture among agents built on similar models. Safety must consequently rely on layered defenses spanning models, agents, permissions, resources, reputation, verification, and human controls. Looking further ahead, Tomašev expects an agentic economy composed not only of general assistants but also of cheaper, faster, certified specialists coordinated by a general orchestration layer. This distributed form of intelligence could accelerate science and productivity, but it shifts alignment from controlling individual models to governing dynamic societies of interacting agents.
Sections
Core Concepts
Key terms used to distinguish agent behavior, coordination, and systemic risk.
- AI agent: A system that observes the state of an environment and performs actions within it, typically using a language model to formulate actions and a harness to execute them.
- Agent harness: The surrounding mechanism that gives a language model tools, enacts its proposed changes, and supports chained decisions.
- Automation bias: The tendency to stop scrutinizing an automated system after repeated successes, allowing subtle errors to pass through.
- Intelligent delegation: Purposeful division and assignment of complex work based on agent capabilities, reliability, dependencies, and expected failure modes.
- Reward hacking: Satisfying the technical wording of a request without fulfilling its underlying intent or spirit.
- Agentic trap: Malicious content or environmental manipulation designed to redirect, jailbreak, or compromise an agent while it interacts with the web.
- Defense through depth: Layering multiple imperfect safeguards across resources, models, agents, permissions, and human controls so that one failure does not compromise the entire system.
- Cognitive monoculture: A population of agents built on similar models that consequently develops similar opinions and makes correlated decisions.
- Agentic economy: A system in which personal assistants, general orchestrators, and specialist agents transact, negotiate, delegate, and allocate resources among themselves.
Critical Distinctions
Contrasts that clarify what agents add and why distributed systems require different controls.
- A language model generates a response, whereas an agent can observe an environment, plan across steps, use tools, and alter the world.
- Parallelization distributes independent chunks for speed, whereas delegation intelligently decomposes work, selects qualified agents, and manages dependencies and failures.
- Software tasks can often be validated with formal tests, while real-world tasks may involve subjectivity, physical feedback, or irreversible consequences.
- Generalist agents provide broad coordination and flexibility, while specialist agents can deliver narrow capabilities with greater speed, accuracy, and cost efficiency.
- Individual alignment evaluates one model's behavior, while distributed alignment must govern changing networks of agents, sub-delegations, humans, and economic incentives.
Strategic Implications
Higher-order conclusions derived from the discussion.
- The main bottleneck is shifting from raw model capability to institutional design: orchestration, contracts, permissions, reputation, incentives, and failure recovery increasingly determine whether agents are useful.
- Human oversight can become weaker as agents improve because reliable performance encourages disengagement. Greater capability may therefore require stronger attention-preserving review mechanisms rather than simply fewer checkpoints.
- Agent safety is partly an ecological problem. Even a well-aligned model can fail when embedded in a poisoned web environment, connected to excessive permissions, or coordinated with agents that exhibit correlated behavior.
- The economically dominant form of advanced AI may resemble humanity's division of labor: a distributed network of specialists connected by general coordination rather than one monolithic intelligence.
- System-level alignment may depend increasingly on mechanism design and economic incentives because the membership and interaction patterns of agent societies will change from task to task.