Overview
The talk reframes agent-to-agent collaboration as a context-engineering problem: how can an LLM obtain the information required for a good answer or tool call when relevant data is distributed across private silos? The theoretical ideal is one agent with access to all information, but privacy, security, and transaction costs make that impossible. Multi-agent architectures should therefore be judged by how closely they approximate that ideal without exposing information improperly. The speaker examines five strategies: broad access within a trust boundary, privacy-preserving custom tools, shared knowledge silos, human-mediated exchange, and a controlled black-box agent that searches private data before requesting narrowly targeted approval. He argues that manually configured boundaries and tools provide immediate utility but do not naturally improve as models become more capable. AI-maintained wikis or databases offer the clearest near-term return, especially in smaller, high-trust companies, because they continuously move eligible knowledge from private systems into common spaces. Longer term, policy-driven “auto mode” could let models decide when information is sufficiently low-risk to share. That trajectory creates serious challenges—including prompt injection, persistent false information, accidental disclosure, auditability, and unclear approval rules—but could eventually enable valuable information exchange both within and across companies.
Sections
Higher-Order Insights
Implications derived from the five approaches and their shared constraints.
- The real optimization target is permissible context completeness: systems need enough relevant information to act well, but no more disclosure than the task justifies.
- Architectures based on policies and sensitivity tiers can benefit automatically from model improvements, whereas manually enumerated tools and boundaries remain bottlenecked by human design work.
- Shared organizational memory creates a compounding loop in both directions: accurate knowledge improves future work, while a single durable error may repeatedly poison later decisions.
- Cross-company collaboration may produce the strongest network effects, but it also moves the trust problem from internal governance to negotiated, multi-party governance.
Core Concepts
Terms used to describe cross-silo agent architectures.
- Agentic search: an agent uses multiple tools to explore an information space and assemble the context required before returning an answer or making a tool call.
- Trust boundary: a household, team, or organization within which an agent receives broad access comparable to that granted to an authorized human member.
- Shared silo: a common wiki, database, skill repository, or similar space where information approved for group use accumulates and becomes accessible to multiple agents.
- Sweeper AI: an agent operating inside a private silo that reviews new information, applies a sharing policy, and transfers eligible material into shared spaces.
- Black-box approach: a protected agent process that searches across private data, keeps its trace inaccessible, and requests approval only from the owners of information needed for the final disclosure or action.
- Low-sensitivity zone: a predefined risk category in which an LLM may make sharing decisions automatically without case-by-case human approval.
Architecture Trade-offs
Contrasts among the proposed methods for accessing information across silos.
- Broad trust-boundary access is simple and compatible with established SaaS security models, while privacy-preserving tools expose only constrained outputs but require workflow-specific engineering.
- Human-mediated exchange provides explicit consent but may notify many irrelevant people, whereas black-box search identifies the relevant owner before requesting approval.
- Human review of proposed wiki contributions is easier to trust but creates recurring operational work, while LLM-enforced policy offers scalable automation at greater disclosure risk.
- Trust boundaries and custom tools are manually defined and relatively static, while shared-memory and auto-mode architectures can expand their capability as models improve.
Implementation and Governance Details
Concrete mechanisms, controls, and failure modes identified in the talk.
- A relationship-discovery tool can inspect company email, filter contacts by company and role, rank employees by relationship strength, and return only the ranking rather than raw messages.
- A sweeper agent needs access to one private silo, an explicit policy describing what must remain private, and descriptions of the shared destinations into which eligible information may be written.
- The black-box design delays authorization until immediately before a result is shared or a consequential tool call is executed, limiting approval requests to the owners of necessary information.
- Prompt injection inside a searchable silo can influence downstream agent behavior when malicious content is retrieved into the working context.
- Shared knowledge requires correction and provenance controls because an erroneous model-generated fact can persist and affect future retrievals.
- Production governance must define who approves disclosures, what activity is logged, which actions are reversible, and who can audit the supposedly black-box process.