Overview
The talk demonstrates a GitHub pull-request review agent implemented three ways, progressively replacing application code with declarative files. The first version uses a hand-built Python loop that parses model outputs, routes function calls, executes tools, handles errors, and maintains state. An agent framework then removes much of this boilerplate by generating schemas from function signatures and managing the execution loop, although developers still maintain Python tools and their runtime environment. The final version uses a hosted general-purpose agent connected to an isolated Linux sandbox. Its behavior is defined mainly through an AGENTS.md instruction file, skills, setup scripts, source files, network policy, and securely injected credentials. Instead of receiving narrowly defined functions for every action, the agent uses general capabilities such as Bash, a filesystem, Google Search, and the GitHub CLI to discover how to complete tasks. Server-side state, context compaction, background execution, and tool iteration are handled by the platform. The broader thesis is that stronger models reduce the need for rigid orchestration: capabilities can increasingly be extended by adding or modifying files rather than changing application code. This does not eliminate engineering responsibility; it relocates it toward domain instructions, clean capability boundaries, credential isolation, outcome verification, and evaluations.
Sections
Core Concepts
Terms used to explain the transition from code-driven orchestration to file-configured hosted agents.
- LLM agent: a model-driven system that repeatedly uses tools until it achieves a specified goal.
- Interactions API: a unified Gemini interface for invoking models or agents with tool calling, multimodal understanding and generation, server-side state, and background execution.
- Step-based timeline: a flat interaction history containing typed events such as user input, reasoning, function calls, and function results instead of only alternating chat roles.
- Agent harness: the runtime and orchestration layer that connects the model to tools, executes calls, returns results, handles errors, and continues the loop.
- Agent files: instruction, skill, configuration, setup, and context files that define or extend agent behavior without requiring changes to the primary application code.
- Environment: a hosted isolated sandbox populated with sources such as repositories, cloud-storage objects, inline files, scripts, tools, and network-access rules.
Implementation and Runtime Details
Concrete mechanisms used by the three agent implementations and the hosted sandbox.
- The raw implementation parses model output, detects text or function calls, matches calls to JSON schemas and Python functions, executes them, appends results or errors, and repeats.
- The framework implementation derives function-call schemas from Python function signatures and handles looping, routing, retries, error handling, and execution mapping.
- The file-based implementation uses an AGENTS.md instruction file plus a Bash setup script that checks for the GitHub CLI and installs it when absent.
- Environment sources may include a GitHub repository, a GCS bucket, or inline files.
- Authenticated outbound requests pass through a network proxy that injects credentials without exposing the underlying token to the agent.
- The GitHub example configures credentials for both the GitHub API and github.com because API operations and Git commands use different endpoints.
- Multi-turn continuity is maintained by passing the previous interaction ID, while server-side state and automatic context compaction handle long-running conversations.
- A reusable custom agent can be assigned an ID together with a base agent, system instruction, environment, tools, and credentials, then invoked through the same interface as a model.
Three Generations of Agent Construction
How responsibility moves from application code to frameworks and finally to a hosted, file-configured runtime.
- Raw Python versus an agent framework: both depend on explicitly implemented tools, but the framework removes manual loops, routing, retries, error handling, and JSON-schema authoring.
- Agent framework versus hosted remote agent: the framework still requires Python tools and a developer-managed runtime, while the hosted agent executes general tools in an isolated sandbox and accepts behavior through files.
- Specific functions versus general-purpose tools: narrow functions tightly constrain available actions, whereas Bash, filesystems, CLIs, and search allow the model to discover broader execution paths.
- Execution-path control versus outcome control: traditional harnesses encode how each task must proceed, while the proposed model emphasizes domain instructions, clean boundaries, and verification of the final result.
Strategic Implications
Higher-level deductions arising from the demonstrated architectural progression.
- The main architectural shift is not from Python to Markdown as languages; it is from encoding execution paths to encoding intent, constraints, capabilities, and evaluation criteria.
- File-based agents turn prompts, skills, setup scripts, and persistent notes into an inspectable control plane that can be versioned and reused separately from the hosted execution engine.
- General-purpose tools increase adaptability but also enlarge the behavioral search space, making sandboxing, network policy, credential mediation, and outcome evaluation more important rather than less important.
- Frequent harness rewrites may indicate that orchestration assumptions are tied too closely to the limitations of a particular model generation.
- The most durable agent investment may be high-quality domain knowledge and evaluations because both remain valuable as models and execution harnesses change.