Run All Agent Tests | SpeechifyAI API

curl -X POST https://api.speechify.ai/v1/agents/agent_01jqr8x9zg5k2m3n4p5q6r7s8t/tests/runs \
     -H "Authorization: Bearer <token>" \
     -H "Content-Type: application/json" \
     -d '{
  "config_override": {
    "prompt": "example",
    "model": "gpt-4o",
    "tool_ids": [
      "tool_01jqr8x9zg5k2m3n4p5q6r7s8t"
    ]
  },
  "flow_version_id": "fver_01arz3ndektsv4rrffq69g5fav"
}'

{
  "runs": [
    {
      "id": "run_01kpvdq02cbtxqqsca60m5scng",
      "test_id": "test_01jqr8x9zg5k2m3n4p5q6r7s8t",
      "agent_id": "agent_01jqr8x9zg5k2m3n4p5q6r7s8t",
      "status": "pending",
      "created_at": "2026-06-24T18:03:11Z",
      "verdict": "passed",
      "started_at": "2026-06-24T18:03:11Z",
      "ended_at": "2026-06-24T18:03:11Z",
      "result": {
        "test_type": "reply",
        "passed": true,
        "rationale": "The agent greeted Alice by name as required.",
        "duration_ms": 1240,
        "reply": {
          "agent_response": "Hi Alice! How can I help you today?",
          "passed": true,
          "rationale": "Response contains the caller's name.",
          "score": 0.95,
          "duration_ms": 1240
        },
        "tool_call": null,
        "simulation": null
      },
      "error": "example"
    }
  ],
  "suite_run": {
    "id": "srun_01kpvdq02cbtxqqsca60m5scng",
    "trigger": "run_all",
    "status": "pending",
    "total": 10,
    "completed": 7,
    "failed": 1,
    "running": 1,
    "pending": 1,
    "cancelled": 1,
    "results": {
      "passed": 1,
      "failed": 1
    },
    "created_at": "2026-06-24T18:03:11Z",
    "agent_id": "agent_01jqr8x9zg5k2m3n4p5q6r7s8t",
    "agent_name": "example",
    "parent_suite_run_id": "parent_suite_run_01jqr8x9zg5k2m3n4p5q6r7s8t",
    "verdict": "passed",
    "ended_at": "2026-06-24T18:03:11Z",
    "config_override": {
      "prompt": "example",
      "model": "gpt-4o",
      "tool_ids": []
    },
    "flow_version_id": "fver_01arz3ndektsv4rrffq69g5fav",
    "flow_version_number": 1
  }
}

Enqueue runs for every test on the agent concurrently. Up to 50 tests are dispatched in one call. Each returned run starts in pending status; poll GET /v1/agents/tests/runs/{test_run_id} for the terminal result (completed / failed / cancelled, with the pass/fail judgment in the separate verdict field).

An optional request body runs the whole suite against a proposed config: a config_override (prompt / model / tools) applied to every test without editing the tests, and/or a flow_version_id to target a specific flow version instead of the agent’s active flow. Omit the body to run against the agent’s live config and active flow.

Enqueue runs for every test on the agent concurrently. Up to 50 tests are dispatched in one call. Each returned run starts in `pending` status; poll `GET /v1/agents/tests/runs/{test_run_id}` for the terminal result (`completed` / `failed` / `cancelled`, with the pass/fail judgment in the separate `verdict` field). An optional request body runs the whole suite against a proposed config: a `config_override` (prompt / model / tools) applied to every test without editing the tests, and/or a `flow_version_id` to target a specific flow version instead of the agent's active flow. Omit the body to run against the agent's live config and active flow.

Authentication

AuthorizationBearer

Enter your API key with the Bearer prefix, e.g. ‘Bearer sk_…’.

Path parameters

agent_idstringRequired

Agent ID.

Headers

Idempotency-KeystringOptional<=255 characters

A client-generated key (an opaque string, max 255 chars) that makes a side-effect POST safe to retry: the server runs the operation exactly once and replays the first response (its status and body) for 24 hours. Reusing a key with a different request body, or while the first request is still in flight, returns 409 idempotency_conflict. A replayed response carries the Idempotent-Replayed: true header.

A client-generated key (an opaque string, max 255 chars) that makes a side-effect POST safe to retry: the server runs the operation exactly once and replays the first response (its status and body) for 24 hours. Reusing a key with a different request body, or while the first request is still in flight, returns `409 idempotency_conflict`. A replayed response carries the `Idempotent-Replayed: true` header.

Request

This endpoint expects an object.

config_overrideobjectOptional

A run-level config override applied to every test in a Run All. Layered on top of the agent’s stored config for the duration of the suite run, so the whole suite can be validated against a proposed prompt / model / toolbelt without editing any test. An absent field leaves the agent’s value untouched; a run-level override wins over a deprecated per-test system_prompt_override / model_override.

flow_version_idstringOptional

Targets a specific flow version (an agent_versions row) instead of the agent’s active flow — version-targeted regression. Must be a flow version of the agent under test. Prefixed external id (fver_...).

Response headers

X-Request-IDstring

Unique identifier for this request, present on every response (2xx and non-2xx alike). If the caller sends an X-Request-ID request header the server echoes it back (sanitized and length-capped) so one logical request can be traced end-to-end; otherwise the server generates a fresh value. Log it on every response and quote it in support requests - it is the stable handle that ties your observation to Speechify’s server-side logs, and it matches the request_id field in the error envelope.

Unique identifier for this request, present on every response (2xx and non-2xx alike). If the caller sends an `X-Request-ID` request header the server echoes it back (sanitized and length-capped) so one logical request can be traced end-to-end; otherwise the server generates a fresh value. Log it on every response and quote it in support requests - it is the stable handle that ties your observation to Speechify's server-side logs, and it matches the `request_id` field in the error envelope.

Response

Queued runs for all tests on the agent.

runslist of objects

suite_runobject or null

The suite run grouping the queued runs.

Errors

400

Bad Request Error

401

Unauthorized Error

404

Not Found Error

409

Conflict Error