API ReferenceTests

Run All Agent Tests

Enqueue runs for every test on the agent concurrently. Up to 50 tests are dispatched in one call. Each returned run starts in `queued` status; poll `GET /v1/agents/tests/runs/{id}` for the terminal result. An optional request body runs the whole suite against a proposed config: a `config_override` (prompt / model / tools) applied to every test without editing the tests, and/or a `flow_version_id` to target a specific flow version instead of the agent's active flow. Omit the body to run against the agent's live config and active flow.

Authentication

AuthorizationBearer

Enter your API key with the Bearer prefix, e.g. ‘Bearer sk_…’.

Path parameters

idstringRequired

Request

This endpoint expects an object.
config_overrideobjectOptional

A run-level config override applied to every test in a Run All. Layered on top of the agent’s stored config for the duration of the suite run, so the whole suite can be validated against a proposed prompt / model / toolbelt without editing any test. An absent field leaves the agent’s value untouched; a run-level override wins over a deprecated per-test system_prompt_override / model_override.

flow_version_idstringOptional

Targets a specific flow version (an agent_versions row) instead of the agent’s active flow — version-targeted regression. Must be a flow version of the agent under test. Raw UUID; flow versions carry no prefixed wire id.

Response

Queued runs for all tests on the agent.
runslist of objects
suite_runobject or null
The suite run grouping the queued runs.

Errors

400
Bad Request Error
401
Unauthorized Error
404
Not Found Error