Batch

curl https://www.ninjachat.ai/api/v1/batch \
  -H "Authorization: Bearer $NINJACHAT_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: batch-2026-09-01-0001" \
  -d '{
    "requests": [
      { "model": "gpt-5.4", "messages": [{ "role": "user", "content": "Classify: \"My order never arrived.\"" }], "max_tokens": 32 },
      { "model": "claude-sonnet-5", "messages": [{ "role": "user", "content": "Classify: \"How do I reset my password?\"" }], "max_tokens": 32 }
    ],
    "fail_on_any_error": false
  }'

​

Request

​

Response

{
  "results": [
    {
      "index": 0,
      "success": true,
      "model": "gpt-5.4",
      "requested_model": "gpt-5.4",
      "content": "shipping",
      "cost_cents": 0.02,
      "latency_ms": 640,
      "tokens": { "prompt": 18, "completion": 2, "total": 20 },
      "usage": { "inputTokens": 18, "outputTokens": 2, "present": true }
    },
    {
      "index": 1,
      "success": false,
      "model": "claude-sonnet-5",
      "requested_model": "claude-sonnet-5",
      "error": "...",
      "cost_cents": 0,
      "latency_ms": 1200
    }
  ],
  "succeeded": 1,
  "failed": 1,
  "total_cost_cents": 0.02,
  "total_cost": "$0.0002",
  "balance": "$24.99",
  "metadata": { "total_latency_ms": 1250, "batch_size": 2, "parallelism": 2 },
  "request_id": "req_..."
}

​

Streaming

​

Billing and limits

• Hold, then settle. Before any job runs, the gateway holds the sum of every job’s metered maximum (input estimate plus max_tokens at that model’s rates). Each successful job bills its actual metered usage at the serving model’s rates; failed jobs cost nothing; the unused hold is refunded when the batch completes. If your balance can’t cover the hold you get 402 insufficient_credits with estimated_cost and balance; a project or key spend limit returns 402 spend_limit_exceeded.
• Rate limits. A batch of N jobs consumes N units of the chat requests-per-minute and concurrency budgets and the summed token estimate of the tokens-per-minute budget, so it draws on the same 60-per-minute-per-key pool as /chat/completions. A 429 from this route adds units (the fan-out size). See Rate limits.
• Body size. The whole request must fit the batch route’s JSON limit; keep long documents to a few jobs per call.