Agent streaming

POST /v1/agent/stream is the core of the relay: one leg of an agent conversation. Your client owns the canonical message list and re-posts it (plus any new tool results) each leg; the relay is a thin proxy that resolves the model, injects server-side context such as project memory, and streams the provider's response back as newline-delimited JSON (application/x-ndjson).

The endpoint requires auth when an auth store is configured: you get 401 {"error": "unauthorized", ...} or 403 {"error": "forbidden", ...} before any streaming begins. See Overview & auth.

Request body

{
  "device_id": "stable-per-install-id",
  "turn_id": "one-id-per-user-turn",
  "model": "auto",
  "bias": "responsive",
  "reasoning": null,
  "autonomous": false,
  "tool_role": null,
  "project_id": "a1b2c3d4e5f6",
  "cwd": "/home/you/project",
  "conversation_id": "stable-across-legs",
  "messages": [],
  "tools": []
}

• model — a catalog id, or "auto" for the heuristic router.
• bias — auto-routing bias: "genius", "cost-efficient", or "responsive". The tier is the bias field, not a model suffix: there is no auto-genius or auto-efficient model id — sending one returns {"error": ..., "message": "unknown model 'auto-genius'"}. bias only applies when model is "auto"; it is ignored for a pinned id.
• reasoning — optional reasoning level for models that support it.
• autonomous — true for hands-off autonomous or orchestration runs.
• tool_role — orchestration role ("orchestrator", "worker", ...); see Tool definitions for how it gates the toolset.
• project_id — memory namespace: sha256(git remote ?: root ?: cwd)[:12]. cwd is the server-side fallback for deriving it.
• turn_id — one id per user turn; tool legs within the turn share it.

There is no user_id field: identity is always derived server-side from the Bearer token.

Every /v1/agent/stream request is one paid provider call and charges one quota unit — including tool legs.

Message

{
  "role": "user",
  "content": "the text",
  "tool_call_id": "call_1",
  "tool_calls": [
    { "id": "call_1", "name": "read_file", "arguments": "{\"path\":\"src/main.rs\"}" }
  ]
}

• role — "system", "user", "assistant", or "tool".
• tool_call_id — only on role:"tool": pairs the result to the call it answers.
• tool_calls — only on role:"assistant": the calls the model made.

Accepted content shapes — all normalized to flat text server-side:

• a plain string (the canonical shape);
• null or absent → treated as "";
• an array of content blocks → the {"type": "text", "text": ...} blocks are joined with newlines. Non-text blocks — including image blocks — are dropped: relay content is flat text, and there is no image field on the wire;
• a single {"type": "text", "text": "..."} object → its text.

Accepted tool_calls shapes — all normalized to the flat form:

• flat {"id", "name", "arguments"} (canonical);
• nested {"id", "type": "function", "function": {"name", "arguments"}};
• arguments as either a JSON string or a JSON object (an object is re-serialized to its JSON string; absent → "{}"). A missing name defaults to "" and fails at tool dispatch with a per-tool error rather than rejecting the request.

Responses always emit the canonical shapes — string content, flat tool_calls with string arguments — regardless of which accepted input form was sent.

tools is an array of ToolDef objects: {"name", "description", "parameters": {JSON Schema}} — see Tool definitions.

Response events

One JSON object per line:

A body that fails to parse returns a pre-stream 400 {"error": "bad_request", "message": ...} whose message names the required fields and the accepted shapes.

Usage

{ "prompt_tokens": 812, "completion_tokens": 240, "cached_tokens": 512 }

cached_tokens is the provider's prompt-cache hit count, normalized across providers (OpenAI/Groq/Gemini/Moonshot prompt_tokens_details.cached_tokens, DeepSeek prompt_cache_hit_tokens, Anthropic-native cache_read_input_tokens); 0 when the provider reports nothing. It is an additive field — old clients simply ignore it.

Streaming with curl

curl -sN https://<relay-host>/v1/agent/stream \
  -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{
    "device_id": "docs-demo",
    "turn_id": "t1",
    "model": "auto",
    "conversation_id": "c1",
    "cwd": "/home/you/project",
    "messages": [{"role": "user", "content": "Say hi in five words."}],
    "tools": []
  }'
# {"event":"router","model":"...","provider":"...","reason":"..."}
# {"event":"meta","model":"...","provider":"..."}
# {"event":"token","text":"Hi"}
# ...
# {"event":"done","usage":{"prompt_tokens":...,"completion_tokens":...,"cached_tokens":...}}

-N disables curl's buffering so you see events as they stream.