Voice Replies
Agents reply in text by default. Voice is a per-message choice: during a channel turn, the agent calls the send_voice_replytool to speak that one reply as a voice note on Telegram, Slack, or WhatsApp. Nothing is spoken unless the agent asks for it — there is no always-on voice mode.
Delivery is fail-soft. If synthesis fails, the channel is disabled, or the platform has no voice key, the agent simply falls back to plain text. A voice problem never loses a message.
Concepts
send_voice_replytool. It is not a channel-wide “speak everything” setting.21m00Tcm4TlvDq8ikWAM) picked by the owner. It selects the voice the agent speaks with.How It Works
1. Admin — provide a voice key (once)
Set the ElevenLabs key under Settings → General → Voice (ElevenLabs) (stored setting, with ELEVENLABS_API_KEY in .env as the fallback for older deployments). Optionally set a platform default voice there too. Until a key resolves, the whole feature is unavailable — the tts_available flag is false and the agent config UI is disabled.
2. Owner — enable and configure the agent
Open the agent's Settings tab and pick Channels in the rail → Voice replies (under How it speaks outside).
Turn voice replies on and pick a voice— or rely on the platform default voice. Enabling with neither an agent voice nor a platform default is rejected.
Set the per-channel allow flags in each channel's own panel on the Channels tab (Telegram, Slack, WhatsApp). Each channel is independent; leave a channel off to keep it text-only.
3. Agent — opt a reply into voice
During a channel turn, the agent calls send_voice_reply to speak that specific reply. The backend resolves the channel destination from the current execution, checks the agent-level enable plus the per-channel flag, synthesizes the audio, and delivers it. If any check fails, it reports “not delivered” and the agent sends text instead.
4. Agent — speak to someone proactively (scheduled briefs)
A scheduled run, loop, or reminder has no conversation to reply into, so a plain send_voice_reply call there is refused with guidance. To send a voice note from such a run, the agent also passes who to speak to — a role with to (primary, approver or viewer; the platform picks the person, and the agent's owner receives it when nobody holds the primary seat) or an email with recipient_email — and a channel (telegram or whatsapp). This works like send_message:
idempotency_key so a retried run does not speak twice; a note that did not go out does not use the key up. The key covers voice notes only, so the same name on a text is a separate delivery.Keep proactive voice notes short. Text longer than the cap (TTS_MAX_CHARS, 1500 characters by default) is refused with text_too_long, so a daily brief should be a short spoken summary followed by the detail in text.
Per-Channel Delivery
| Channel | Delivery format |
|---|---|
| Telegram | OGG/Opus voice note (sendVoice); in groups it replies to the triggering message |
| Slack | Inline MP3 clip uploaded into the thread (Slack renders MP3 with a built-in player) |
| OGG voice note delivered via Twilio media (hosted transiently by Trinity) |
WhatsApp voice notes work even when the agent's file-sharing toggle is off — voice replies are gated only by their own flags.
For Agents
| Endpoint | Method | Description |
|---|---|---|
| /api/agents/{name}/voice-replies | GET | {enabled, voice_id, channels:{telegram,slack,whatsapp}, effective_voice_id, default_voice_id, available} — available reflects a resolvable platform key |
| /api/agents/{name}/voice-replies | PUT | Owner-only partial update of {enabled?, voice_id?, channels?}; 400 unless enabled or channels is present, or when enabling with no agent voice and no platform default |
| /api/agents/{name}/voice-reply | POST | Per-message delivery backing the tool; fail-soft {delivered, channel, reason}; 409 on an in-flight duplicate for the same turn |
| /api/settings/elevenlabs | GET / PUT | Admin key + default voice: {key_configured, key_source, default_voice_id}, plus the key's speech-to-text status for Workspace dictation (stt_capability, stt_detail, stt_checked_at, stt_last_failure); key stored encrypted, never echoed |
MCP tool:
| Tool | Description |
|---|---|
| send_voice_reply | Speak the current channel reply as a voice note. Backend resolves the destination from the execution, gates on the agent + per-channel flags, and returns fail-soft so the agent can fall back to text. Pass to (a role) or recipient_email, plus channel (telegram or whatsapp), to speak to a person from a scheduled or other non-chat run; consent and rate limits apply as for send_message |
See the Backend API Docs (/docs on your instance) for full request/response schemas.