New Research — A Measurably Self-Improving Multi-Agent Forecasting System
“A Measurably Self-Improving Multi-Agent Forecasting System: Five Feedback Loops, No Retraining” is now published on GitHub. It describes a multi-agent forecasting system that has been running on Trinity since February 2026 and improves while it runs — no retraining, no fine-tuning. Seven specialized forecasting agents (the oracle fleet), a meta-analyst (Cleon) that measures quality and routes corrections, and a knowledge-base agent (Cornelius) that turns surprising failures into insights served back before future predictions.
Watch the walkthrough → youtu.be/_BFKrp9GnNQ
Read the paper → research/closed-loop-forecasting
The record so far
- 96,161 logged predictions, 73,552 resolved outcomes
- Brier 0.194 vs 0.250 always-0.5 and 0.217 base-rate baselines; skill score +0.105
- 93.5–99.7% of prediction error is noise, not bias — calibration alone can't fix that
- 56,347 scored hidden-state hypotheses, with a monotone win-rate gradient across lifecycle tiers (strong 75.0%, emerging 68.2%, fading 59.0%)
- Three of five compared agents improved both Brier score and surprise rate; two moved against their feedback — a limitation reported, not hidden
The five feedback loops
- Noise-adaptive calibration— routes to process-consistency guidance when noise dominates
- Surprise-driven insight extraction— confident misses become knowledge
- Knowledge retrieval before prediction
- Hidden-state hypothesis injection— theses about unobserved world state, scored by the predictions that use them
- A generative loopthat decides what is worth forecasting — abducing theses from cross-domain anomalies and commissioning falsifiable predictions
Live dashboard → brierhq.ai — predictions, resolutions, and Brier scores published as they land.
Discuss on GitHub → github.com/Abilityai/trinity/discussions/1613
Built and running entirely on Trinity — free and open source (Apache 2.0).