Overview
Production AI agents are following a maturation path similar to microservices: enthusiasm arrives first, while reliable architecture and operating practices emerge through repeated failures. Drawing on Navan’s production experience, Roberto Milev and Uday describe a developing stack for agentic systems. Stateful runtimes must support persistent, isolated sessions rather than imitate stateless APIs. Memory evolves from short conversational history into consolidated long-term and episodic knowledge, while context is assembled dynamically from reusable skills that combine domain instructions with tool execution. Because agents produce non-deterministic decisions across long trajectories, conventional logs and deterministic tests are insufficient; teams need trace-level observability, decision signals, confidence indicators, and trajectory-based evaluation. Enterprise deployments also require fine-grained authorization around every tool call because an agent may act for a user, through a service identity, or in an ambiguous combination of both. Navan currently favors a master agent that progressively loads skills and selectively invokes sub-agents, reserving agent-to-agent protocols for genuine organizational boundaries. Runtime scaling, memory, and tool invocation appear increasingly mature, but cost control, replay, debugging, observability standards, testing, and multi-agent orchestration remain unsettled. The practical conclusion is to avoid premature complexity and build the missing operational layers deliberately.
Sections
Strategic Insights
Higher-order implications derived from Navan’s production experience.
- The emerging agent stack resembles a distributed-systems stack: model capability is only one layer, while reliability depends on durable state, bounded context, traces, policies, evaluation, and orchestration.
- Larger context windows do not eliminate context engineering; they shift the problem from fitting information to selecting information that preserves the agent’s focus.
- Multi-agent architecture is most defensible when it mirrors a genuine organizational or ownership boundary, rather than serving as a default decomposition technique.
- Non-determinism changes the unit of quality assurance from exact execution sequences to bounded trajectories, observable decisions, and acceptable outcomes.
Production Lessons
Practical lessons the speakers learned while building and operating agents.
- Perfect a single agentic loop before adding multi-agent coordination.
- Design session persistence, rehydration, and isolation explicitly because managed runtimes may not cover every application requirement.
- Use skills as independently testable packages of instructions and tools, then disclose their context progressively.
- Instrument decisions and tool boundaries at development time rather than relying on retrospective inspection of large log streams.
- Evaluate goal-directed trajectories and regression signals instead of expecting deterministic step-by-step reproduction.
- Apply authorization policies before and after tool calls, especially when an agent can spend money or handle sensitive data.
Technical Architecture
Specific components, protocols, signals, and operational mechanisms mentioned in the talk.
- Navan runs on AWS and uses AWS AgentCore Runtime, supplemented with custom session persistence and rehydration.
- The memory lifecycle includes ingestion, extraction, consolidation, and retrieval, spanning short-term, long-term, and episodic memory.
- A skill combines contextual instructions and setup with tool execution, allowing dynamic composition, reuse, independent testing, and progressive disclosure.
- Suggested interception points include pre-tool calls, post-tool calls, pre-decision events, and post-decision events.
- Trace signals include the current goal, reasons for operations, belief state, tool calls, confidence, and whether an answer was inferred.
- Trajectory evaluations compare an execution’s movement from its source state toward its destination goal to assess efficiency and completeness.
- MCP is described as the de facto tool-invocation protocol, while A2A is presented as a younger protocol for contracts and communication across agent or team boundaries.
- OpenTelemetry is an emerging observability direction, but its suitability for agentic calls remains an open question.
Architectural Comparisons
Contrasts used to explain how agent systems differ from earlier software patterns.
- Traditional APIs are commonly stateless and request-scoped, whereas agents require persistent sessions, isolation, and longer-lived state.
- Conventional programs follow expected deterministic sequences, while agents can choose different steps on each execution.
- A single master agent with progressively loaded skills is easier to control than a broadly orchestrated multi-agent system; multiple independent agents are more justified across real team boundaries.
- Traditional logs record events but become cognitively overwhelming for long agent runs; structured traces expose spans, decisions, goals, and tool interactions.
Memorable Quotes
Verbatim statements that capture the talk’s main arguments.
- If you can't build a single agentic loop, why go in and try to build a multi-agent orchestrated system?
- agents are stateful by nature.
- Agents output a lot of thinking. There's too much to consume.
- It makes up its own steps every time differently.
- we know what we need and it's up to us to write and build it.