How Trinity Works
One Monday-morning task — a scheduled job on one agent — followed through the machinery that admits it, runs it, pauses it for a human, and writes it into the record. The 4 min 44 s film below is a live render, not a video: the example is composed, but every mechanism and every number drawn on screen is real and in the repository. Click any beat to jump there.
The Four Systems Around Every Run
Each agent is an isolated Docker container. Everything an agent does — no matter how it was triggered — passes through one admission path, and four systems stand around that path so a fleet can run unattended without becoming ungoverned.
Ledger · Everything on the record
Every run lands with status, duration, and cost. The audit log is append-only — each entry hash-chained to the one before it — and a canary re-proves the system's invariants every five minutes.
Operator · A human in the loop
An agent can stop mid-run and ask. The question waits in a queue a person actually sees; the answer flows back and the run resumes where it paused. Autonomous is not unsupervised.
Breaker · Failure, contained
Three consecutive auth failures open a per-agent circuit breaker: feeding stops in seconds, waiting work fails loudly instead of rotting, and a single probe closes the breaker once the agent is healthy again.
Capacity · Work waits its turn
Each agent runs a bounded number of tasks at once (three, by default). Everything past that waits in a durable, ordered line that survives restarts and drains the moment a slot frees.
What's Real, and What's Illustration
The agent, the “weekly digest” task, and the approval question are a composed, typical example — not a recorded run. Everything structural is real and checkable: the mechanisms, the flow, and every number drawn on screen comes from the Trinity source tree, not from copy.
Pull-based coordination — agents claiming work instead of being pushed it — is mid-build behind flags, so the film doesn't draw it. Everything shown is machinery that ships today.
- 3 consecutive failures open the dispatch breaker
services/dispatch_breaker.py · DISPATCH_FAILURE_THRESHOLD - 5s heartbeat; 3 missed beats raise an alert
agent_server/heartbeat.py · services/heartbeat_service.py - 3 parallel tasks per agent by default
db/schema.py · agent_ownership.max_parallel_tasks - 24h idempotency window at every trigger boundary
db/idempotency.py · claim(ttl_hours=24) - 15min git auto-sync — agents commit their own state back to their repo
agent_server/auto_sync.py · _DEFAULT_INTERVAL - 5min canary cycle re-proving orchestration invariants
services/canary_service.py - Agents and the platform core live on separate Docker networks — an agent has no route to Redis
docker-compose.yml - Audit log is append-only at the database layer (UPDATE/DELETE raise)
db/schema.py · audit_log triggers