Skip to main content
Trinity
Updated Nov 17, 2026main@87e21ff
Architecture

Multi-Agent Collaboration

Trinity enables agents to communicate, delegate, and collaborate through MCP tools, event subscriptions, and system manifests. These three pillars of collaboration — direct communication, pub/sub events, and recipe-based deployment — form the foundation of multi-agent systems.

The Multi-Agent Platform I Run My Company On

May 2026

How We Automate Our Own Ops & Marketing

Mar 2026

Agent Network and Communication

Orchestrator-worker patterns, delegation chains, and multi-agent systems

Agent network: System orchestrator connected to Scribe, Sage, and Scout agents with async message flows

Concepts

Agent-to-Agent Communication — Agents call each other through Trinity MCP tools using agent-scoped API keys. The chat_with_agent MCP tool sends a message to another agent and returns the response.

Async Collaboration — For long-running tasks, use chat_with_agent(parallel=true, async=true) which returns an execution_id immediately. Poll with get_execution_result(agent_name, execution_id) until complete. This keeps the call under the MCP server's own call bound (MCP_CHAT_TIMEOUT_MS, default 25 seconds, set below the 30–60 second ceiling most MCP gateways enforce). A synchronous call that outlives that bound is not lost either: the tool answers with a receipt — status: "queued_timeout" plus the execution_id — instead of a transport error. Poll get_execution_result(agent_name, execution_id) and never re-send a reworded version, which dispatches a second execution. Every agent is taught this rule in its platform prompt (the delegation contract), and the chat_with_agent tool description carries the same text. You rarely need to poll: by default the calling agent is woken with one turn when the run it delegated ends (see Wake on Ending below). Subscribing to the worker's task-completion events is the other way to be told (see Event Subscriptions).

The Delegation Contract — One text, taught to every caller: in each agent's platform prompt on every turn, verbatim in the chat_with_agent and per-agent chat_with_<agent> tool descriptions, and repeated by fan_out and send_message. Its rules:

  • A reply or a receipt comes back. A receipt is an execution_id (a fan_out_id for a batch) with a status such as accepted, queued, queued_timeout or fan_out_timeout. It means the work arrived and is queued, running or done. Every chat_with_* receipt ends with Do not re-send: read the outcome with get_execution_result(...).
  • Never re-send because a call timed out or delivery could not be confirmed. An exact repeat from the same turn is answered with the original; a reworded one can run the work twice. The same text sent from a later turn of the caller is a new delegation and runs again, while a scheduled retry of the same run replays its first attempt's delegations instead of starting them twice.
  • Read the result with get_execution_result (get_fan_out_result for a batch). running is not stuck. The receipt's wake field says who brings the ending back: on_ending means a turn is started on the caller when the run ends, so it ends its turn and sets no reminder; none means nobody will, and wake_reason says why, so the caller sets a reminder naming the execution_id and ends its turn. A receipt is not a result, so the caller never reports the work as done.
  • An error without an execution_id is not proof that nothing ran. The caller looks for its exact message in list_recent_executions first, and re-sends word for word only if it is not there. agent_busy is the exception: the queue was full and nothing ran, so the caller re-sends after retry_after_seconds. A rate_limited result means the run started and hit a usage limit; it names the failed run to read with get_execution_result.
  • pending_approval means nothing ran. The caller does not retry or route it through another agent; the outcome is sent to it.

Wake on Ending — When a run one agent delegated to another ends (success, failure, timeout or cancellation), Trinity starts one turn on the delegating agent, recorded with the trigger caller_wake, so it acts on the outcome instead of polling for it. The turn's message quotes the worker's reply, up to 2,000 characters; a longer reply ends with [cut here: the first N of M characters], and the full result stays readable with get_execution_result. How it behaves:

  • On by default. The owner switches it per agent with Wake this agent when its asks end under Settings › Guardrails › Reliability. That one switch covers both kinds of wake: an ask a person ended, and a delegated run that ended (the label names only asks, but the switch also governs wakes for delegated runs). Its API is GET/PUT /api/agents/{name}/operator-resume, which also takes {"mode": "default" | "on" | "off"}; default follows the instance default, which is on. An agent reads its own setting, and which layer decided it, as wake_on_ending in get_agent_info, but cannot change it.
  • The receipt says whether a wake is coming. chat_with_agent answers a delegation with wake: "on_ending", or with wake: "none" and a wake_reason: nothing_ran (the agent was busy and nothing started), fan_out (a batch's legs never wake the caller; read the batch with get_fan_out_result) or opted_out (the switch is off).
  • No wake for a run a person started, a task an agent sent to itself, a run whose own trigger was a wake, or a caller whose container is stopped (the result is still readable when it next runs; nothing auto-starts a stopped caller). A caller that already subscribes to the worker's task-completion events gets that report-back task instead, so one ending is one turn.
  • Bursts are folded. At most one wake starts per calling agent every 15 seconds. Runs that had already ended when a wake started are listed by that wake's list_recent_executions; a run that ends while a wake is still running is held and delivered once that wake ends, never dropped.
  • Whose turn it is. A wake for work the agent's owner asked for runs as the owner's turn, so the agent's key can read the result it was woken for. A wake for anyone else's work runs with no person on it.
  • Cost. Each wake is one turn, billed to the owner. Wakes between one pair of agents are capped per hour (see the chain-depth limit below).

Pull-Pilot Routing (experimental) — An alternative routing path for agent-to-agent chat_with_agent calls, behind a default-OFF flag (MCP_AGENT_CHAT_PULL_ENABLED). When enabled, a sequential agent-to-agent call is dispatched through Trinity's durable async task path instead of a held synchronous call: the caller gets an immediate receipt with an execution_id and polls get_execution_result(agent_name, execution_id)to read the result. It is an opt-in proof-of-concept for pull/work-stealing coordination — it does not change human chat, parallel calls, or self-calls. Leave it off unless you are piloting it.

This MCP flag is separate from the per-agent pull pilot. When the target agent is itself a pull-pilot agent, every interactive trigger on it is queued and claimed by its worker, including a sequential chat_with_agent call to it. The caller still waits for the reply, but a turn that no worker claims within one agent timeout answers with a capacity error (429) instead of running later for nobody.

On an agent in the pull pilot, a turn can be delivered again after its lease expires, so Trinity passes each turn's execution id to the agent's MCP tools automatically. An outbound message, call, file share, or A2A call made without a usable execution id is refused with effect_unguarded, and the Operating Room gets a Side effect refused: no execution idalert. A pulled turn never runs longer than its lease: its time limit is capped at the agent's current timeout when a worker claims it.

Chain-Depth Limit — Every agent-to-agent hop through chat, tasks, or fan-out is counted along the call chain. A hop past the limit is refused before any work starts, with a 403 carrying inter_agent_depth_exceeded. This stops two agents from calling each other in an endless loop. The default limit is 8 hops (range 1–32), set by the inter_agent_max_chain_depth operator setting or INTER_AGENT_MAX_CHAIN_DEPTH in .env. The MCP tools return the refusal as a result marked retryable: false, and the refusal is recorded in the audit log and as a failed collaboration activity on the calling agent.

The count carries through the other ways one agent can start work: starting a loop (every iteration keeps the starter's depth), triggering a schedule by hand (trigger_agent_schedule, retries included), sending a turn into a chat session, and emitting an event that a subscription turns into a task. Each is refused past the limit with the same 403, and run_agent_loop, trigger_agent_schedule and emit_event return it as a retryable: false result. A task-completion event counts as one hop past the run that finished. Webhook triggers, cron-fired schedule runs and self-reminders still start a new chain. When a run an agent delegated ends and the agent is woken for it, the wake stands at the depth the agent stood at when it delegated, so an agent that keeps delegating through wakes does not climb toward the limit. Those wakes are capped per hour for each pair of agents instead, by the caller_wake_max_per_hour operator setting or CALLER_WAKE_MAX_PER_HOUR in .env (default 120, 0 = unlimited); past it the wake is skipped, the delegating agent gets one wakes capped notification for the hour, and the run's result stays readable with get_execution_result. The refusal names the running execution that set the depth (exhausted_by) and says what lowers it (resets). Event subscriptions also have an hourly dispatch cap per agent pair, which catches an event chain that re-enters too often to ever get deep (see Event Subscriptions).

Timeline Replay — Collaboration is surfaced on the Dashboard as a Timeline of executions, color-coded by trigger type with collaboration arrows linking calls between agents. (The old live node/edge graph view was retired; the underlying collaboration data still flows and feeds the Timeline.)

Grid Dashboard — The Grid view lays out the fleet as tiles with per-agent runtime, autonomy, and health chips — a fast at-a-glance fleet status alongside the Timeline. A third List view shows the same fleet as rows. Press v to cycle Timeline → Grid → List.

How It Works

01

The Dashboard offers a Timeline view, a Grid view, and a List view of the fleet.

02

Timeline shows execution boxes color-coded by trigger type (manual, schedule, MCP, chat) with collaboration arrows drawn between calls that hand off between agents.

03

Grid lays agents out as tiles with runtime, autonomy, and health chips.

04

Click through to any agent's detail page for its full activity.

05

Collaboration events stream in real time over WebSockets (below) and are replayed on the Timeline.

WebSocket Events

  • •agent_collaboration — Fired when one agent calls another via MCP
  • •agent_activity — State changes (started, completed, failed) for all activity types
  • •Source agent is detected via the X-Source-Agent header on the chat endpoint

MCP Tools

ToolDescription
chat_with_agent(agent_name, message)Send a message to another agent and wait for the response
chat_with_agent(agent_name, message, parallel=true, async=true)Send a message asynchronously. Returns an execution_id
chat_with_agent(..., parallel=true, async=true) from a Slack, Telegram or Workspace turnSame receipt, plus report_back: "requested": when the delegated run ends, success or failure, it posts its outcome into that conversation (if the conversation allows such notes). On by default for async delegation; pass execution_id="manual" to turn it off for one call. A parallel=true sync call opts in by passing your own execution_id. A plain sequential call reports back only when it returns a receipt instead of the reply (the platform stopped waiting); when it answers in time, you have the reply and nothing is posted
get_execution_result(agent_name, execution_id)Read the result of an async execution — after a caller_wake turn, or when polling
list_recent_executions(agent_name)List recent executions for an agent
get_agent_activity_summary(agent_name)Activity summary over a configurable time window

Building Multi-Agent Systems

  • •Use System Manifests to deploy pre-configured multi-agent setups
  • •Configure permissions to control which agents can call which
  • •Use shared folders for file-based collaboration between agents
  • •Use event subscriptions for pub/sub patterns between agents
  • •Use the A2A Protocol to call agents on another Trinity instance, or any A2A agent, including signed calls under AAuth

Shared Folder Configuration

Settings › Storage › Shared folders showing Expose Shared Folder and Mount Shared Folders toggles with step-by-step workflow

Agents can share files via Docker volumes:

01

On the source agent, open Settings › Storage › Shared folders and enable Expose Shared Folder. Files placed in /home/developer/shared-out are made available to permitted agents.

02

Grant the consuming agent permission via the Permissions tab.

03

On the consuming agent, enable Mount Shared Folders. Exposed folders appear at /home/developer/shared-in/{agent-name}.

04

Restart both agents to apply the volume mounts.

Removing the permission (or turning Mount Shared Folders off) takes the folder away at the consuming agent's next start. A running agent keeps the mount until it restarts. See Agent Permissions.

Event Subscriptions

Lightweight pub/sub system for inter-agent event pipelines

Concepts

Event — A named occurrence emitted by an agent with a structured JSON payload. Stored in the agent_events table.

Subscription — A rule that says “when agent X emits event type Y, send agent Z an async task with message template M”. Stored in the agent_event_subscriptions table.

Message Template — Supports {{payload.field}}interpolation. The subscriber's task message is built from the event payload.

Permission-Gated — Uses existing agent_permissions. The subscribing agent must have permission to call the source agent.

How It Works

01

Agent A emits an event: emit_event(event_type="report_ready", payload={"url": "...", "summary": "..."})

02

Trinity checks all subscriptions matching agent A + event type report_ready.

03

For each matching subscription, an async task is dispatched to the subscribing agent.

04

The task message is built from the subscription's template with payload fields interpolated.

05

Events are persisted and visible via API.

06

WebSocket broadcast provides real-time event visibility.

MCP Tools

ToolDescription
emit_event(event_type, payload)Emit a named event with data
subscribe_to_event(source_agent, event_type, message_template)Create a subscription
list_event_subscriptions(agent_name)List subscriptions
delete_event_subscription(subscription_id)Remove a subscription

API Endpoints

EndpointMethodDescription
/api/agents/{name}/event-subscriptionsPOSTCreate subscription
/api/agents/{name}/event-subscriptionsGETList subscriptions
/api/event-subscriptions/{id}GETGet by ID
/api/event-subscriptions/{id}PUTUpdate
/api/event-subscriptions/{id}DELETEDelete
/api/eventsPOSTEmit event (agent-scoped)
/api/agents/{name}/emit-eventPOSTEmit for specific agent
/api/agents/{name}/eventsGETEvent history
/api/eventsGETAll events

System Manifests

Recipe-based multi-agent deployment via YAML manifest files

Concepts

System Manifest — A YAML file defining a set of agents, their templates, permissions, shared folders, and schedules. All agents in a manifest are deployed as a single unit.

System View — A saved filter/view in the UI that groups related agents by tags. Use system views to monitor and manage agents that belong to the same manifest.

How It Works

01

Create a system manifest YAML file defining agents and their relationships.

02

Deploy via the deploy_system MCP tool or the REST API.

03

All agents are created, configured, and started according to the manifest.

04

Agents appear on the Dashboard with appropriate tags for grouping.

What a Manifest Describes

  • •A list of agents with name, template, and configuration
  • •Permission presets (which agents can call which)
  • •Shared folder configuration for inter-agent file access
  • •Schedule definitions for autonomous execution
  • •Auto-start settings controlling which agents launch on deploy

MCP Tools

ToolDescription
deploy_system(manifest)Deploy a system from a manifest
list_systems()List all deployed systems
restart_system(name)Restart all agents in a system
get_system_manifest(name)Retrieve the manifest for a deployed system

The Four Pillars of Deep Agency

The collaboration features above implement the first pillar. Together with the remaining three, they define what separates a deep agent from a chatbot.

1. Hierarchical Delegation

Complex goals decompose into task graphs. An orchestrator agent breaks work into sub-tasks and delegates to specialized worker agents, each with distinct skills and access credentials. Workers can delegate further, creating hierarchies of arbitrary depth. Context is quarantined between execution boundaries.

2. Persistent Memory

Deep agents accumulate knowledge over time. They remember past decisions, build working artifacts, maintain dashboards of ongoing state, and evolve their own instructions. Per-agent Docker volumes, GitHub sync, shared folders, and dashboard.yaml provide the substrate for continuous improvement.

3. Extreme Context Engineering

Every token in the prompt carries signal. Instructions are layered: platform-level directives, role-specific CLAUDE.md files, skill definitions, and runtime state combine into a coherent instruction set that gives the model exactly what it needs.

4. Autonomous Operations

Agents execute on schedules, monitor their own health, recover from failures, and escalate to operators only when they judge it necessary. The platform provides scheduling, alerting, and self-healing infrastructure; the agent decides when and how to use it.

Pillars Working Together

The four pillars are not independent features — they compound. An orchestrator delegates to a worker whose persistent memory contains the project context, whose engineered prompt defines exactly how to process the task, running on a cron schedule with no human in the loop. Remove any pillar and the system degrades from autonomous agent to interactive assistant.