> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ninjachat.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Batch

> Run up to 20 independent chat requests in one call, held up front and settled to actual token usage.

Run up to 20 independent chat requests in parallel with `POST /api/v1/batch` or `batch.create`. Return all results together or stream them as they finish.

```bash cURL theme={null}
curl https://www.ninjachat.ai/api/v1/batch \
  -H "Authorization: Bearer $NINJACHAT_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: batch-2026-09-01-0001" \
  -d '{
    "requests": [
      { "model": "gpt-5.4", "messages": [{ "role": "user", "content": "Classify: \"My order never arrived.\"" }], "max_tokens": 32 },
      { "model": "claude-sonnet-5", "messages": [{ "role": "user", "content": "Classify: \"How do I reset my password?\"" }], "max_tokens": 32 }
    ],
    "fail_on_any_error": false
  }'
```

## Request

| Parameter                | Type    | Default  | Description                                                                                                                         |
| ------------------------ | ------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| `requests`               | array   | required | 1–20 jobs.                                                                                                                          |
| `requests[].model`       | string  | `gpt-5`  | A chat model ID from [`GET /models`](/models).                                                                                      |
| `requests[].messages`    | array   | required | 1–30 `system`, `user`, or `assistant` messages; string content up to 50,000 chars.                                                  |
| `requests[].temperature` | number  | `0.7`    | 0–2.                                                                                                                                |
| `requests[].max_tokens`  | integer | `2048`   | Output ceiling per job, 1–8,192. Also sizes the hold.                                                                               |
| `fail_on_any_error`      | boolean | `false`  | When `true`, one failed job fails the whole batch (`500 batch_request_failed` with `failed_index`) and the entire hold is refunded. |
| `stream`                 | boolean | `false`  | Stream results over SSE as jobs complete.                                                                                           |

Send an `Idempotency-Key` header so a retried batch replays instead of running twice. A streamed batch can't be replayed — reusing its key returns `409 stream_not_replayable`.

## Response

```json theme={null}
{
  "results": [
    {
      "index": 0,
      "success": true,
      "model": "gpt-5.4",
      "requested_model": "gpt-5.4",
      "content": "shipping",
      "cost_cents": 0.02,
      "latency_ms": 640,
      "tokens": { "prompt": 18, "completion": 2, "total": 20 },
      "usage": { "inputTokens": 18, "outputTokens": 2, "present": true }
    },
    {
      "index": 1,
      "success": false,
      "model": "claude-sonnet-5",
      "requested_model": "claude-sonnet-5",
      "error": "...",
      "cost_cents": 0,
      "latency_ms": 1200
    }
  ],
  "succeeded": 1,
  "failed": 1,
  "total_cost_cents": 0.02,
  "total_cost": "$0.0002",
  "balance": "$24.99",
  "metadata": { "total_latency_ms": 1250, "batch_size": 2, "parallelism": 2 },
  "request_id": "req_..."
}
```

`results` is ordered by `index`. Each job keeps the same silent model-fallback chain a single chat request has, so `model` is the model that actually served the job and `requested_model` is the one you asked for. A success result carries `usage_estimated: true` when the provider didn't report usage and the tokens were estimated. `balance_warning` appears when your balance is at or below \$5.

### Streaming

With `stream: true` the response is `text/event-stream`. Each finished job arrives as a `data:` frame with `type: "result"` and the same fields as above; a final `type: "summary"` frame carries `succeeded`, `failed`, `total_cost_cents`, `total_cost`, and `request_id`, followed by `data: [DONE]`.

## Billing and limits

* **Hold, then settle.** Before any job runs, the gateway holds the sum of every job's metered maximum (input estimate plus `max_tokens` at that model's rates). Each successful job bills its actual metered usage at the serving model's rates; failed jobs cost nothing; the unused hold is refunded when the batch completes. If your balance can't cover the hold you get `402 insufficient_credits` with `estimated_cost` and `balance`; a project or key spend limit returns `402 spend_limit_exceeded`.
* **Rate limits.** A batch of N jobs consumes N units of the chat requests-per-minute and concurrency budgets and the summed token estimate of the tokens-per-minute budget, so it draws on the same 60-per-minute-per-key pool as `/chat/completions`. A `429` from this route adds `units` (the fan-out size). See [Rate limits](/rate-limits).
* **Body size.** The whole request must fit the batch route's JSON limit; keep long documents to a few jobs per call.

<CardGroup cols={2}>
  <Card title="Compare models" icon="scale-balanced" href="/compare">
    One prompt, several models, ranked results.
  </Card>

  <Card title="Spend controls" icon="coins" href="/budget-routing">
    Estimate holds before you send.
  </Card>
</CardGroup>