SpeechifyAI Agents Tools - System, Webhook, Client, MCP

A tool is a function the LLM can invoke mid-conversation. Tools come in four kinds:

System

Worker built-ins - end_call, transfer_to_number, transfer_to_agent, play_keypad_touch_tone, skip_turn. No HTTP; they run in the agent process in milliseconds.

Webhook

Your backend. Any business logic - look up an order, book an appointment, charge a card. Signed with HMAC-SHA256.

Client

The caller’s browser or SDK. UI actions - navigate the page, fill a form, update a cart. Dispatched over the session’s tools data channel.

MCP

A customer-hosted Model Context Protocol server. The worker opens the configured transport at session start, discovers remote tools, and proxies tool calls through.

Webhook, client, and MCP tools are created once and attached to any number of agents; system built-ins are added directly to an agent. To wire them up, see Add tools.

Parameters

Every tool declares the parameters the LLM may pass:

1 "params": [
2   { "name": "order_id", "type": "string",  "description": "Customer's order ID", "required": true },
3   { "name": "notify",   "type": "boolean", "description": "Email confirmation",   "required": false }
4 ]

type is one of string, number, integer, or boolean. A string parameter can also declare an enum of allowed values.

Slow tools

A webhook or MCP tool that takes more than a second or two leaves the caller listening to silence. Add a long_running block to the tool’s config and the agent says a holding phrase instead, and refuses to call the tool a second time while the first call is still in flight.

1 "long_running": {
2   "filler_message": "one sec, pulling that up",
3   "filler_delay_ms": 2000,
4   "on_duplicate": "reject"
5 }

Field	Default	What it does
`filler_message`	none	Spoken word for word. Write it in your agent’s own language and voice - Speechify ships no default phrase. Omit it to guard against duplicate calls without the agent saying anything.
`filler_delay_ms`	`2000`	How long the tool must have been running, with nobody speaking, before the phrase is said. The wait restarts whenever the agent or the caller speaks, so the phrase never lands on top of live speech.
`filler_interval_ms`	say it once	Cooldown before the phrase may be said again while the tool is still running. Set it only for a tool that can run for tens of seconds.
`additional_filler_messages`	none	Up to 7 phrases for the second and later fills, in order, so a long wait does not repeat one phrase word for word. The last one repeats after that. Requires `filler_interval_ms`.
`on_duplicate`	`reject`	`reject` refuses a second call to the same tool while the first is running. Use `allow` only for a read-only tool the agent legitimately looks several things up with in one breath - duplicates are matched by tool name, never by arguments.

Four things to know before you turn it on:

The tool still waits for the real result. The agent never receives a synthetic “working on it” result, so it cannot describe an outcome your endpoint has not returned yet.
A call already on the wire is never cancelled. Your endpoint may already have charged a card by the time the agent would want to give up, so the request always runs to completion - which is also why on_duplicate defaults to refusing.
The caller can talk over the holding phrase. Barging in stops the phrase; your request keeps running and still answers them.
Agent Tests stay silent. A test run is turn-based and has no dead air, so the phrase is never added to a test transcript you assert against.

On a multilingual agent the phrase is spoken only while the call is still in the agent’s configured language. After the agent switches language mid-call it stays quiet rather than say your phrase in a language the caller has just left.

long_running cannot be combined with fire_and_forget - that tool returns to the agent before your endpoint answers, so there is no wait to fill. Saving both is refused rather than storing a holding phrase that would never be spoken.

Client realtime protocol

Client tools use version 1 of the simba.tools protocol on LiveKit’s reliable data channel. The agent sends a tool_request with an id, name, and JSON object arguments. The client returns a tool_response with the same id and exactly one of result or structured error. Omitting protocol_version is read as version 1 for compatibility, but new clients should write it. Packets are limited to 14 KiB.

The platform binds the request and response to the client participant that started the session. Native clients should accept tool requests only from the agent participant and target responses back to that participant. See Add tools for the packet shapes and error codes.

Client tools are independent from text chat. A native client sends typed user messages through LiveKit’s standard lk.chat text stream and consumes message or transcription callbacks through LiveKit’s APIs. The simba.tools topic carries client tools only.

How invocations are recorded

Every tool call is persisted on the transcript with role=tool, tool_name, tool_args, and tool_result - available via GET /v1/agents/conversations/{id}/messages.