Batches

Updated 8 November 2026

Send up to 2,000 chat requests at once and get the answers within a day, usually within minutes, at half the price. For work nobody is waiting on: labelling, summarising, extracting, evaluating. Works on Gemini models.

import os, time, json, requests

API = "https://wekoya-api.wekoya.tech/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['WEKOYA_API_KEY']}"}

tickets = {"t-1": "My card was charged twice", "t-2": "How do I change my plan?"}

batch = requests.post(f"{API}/batches", headers=HEADERS, json={
    "model": "gemini-3.5-flash-lite",
    "requests": [
        {
            "custom_id": ticket_id,
            "body": {
                "messages": [
                    {"role": "system", "content": "Label the ticket: billing, account or other. One word."},
                    {"role": "user", "content": text},
                ],
                "max_tokens": 20,
            },
        }
        for ticket_id, text in tickets.items()
    ],
}).json()

while batch["status"] in ("in_progress", "cancelling"):
    time.sleep(60)
    batch = requests.get(f"{API}/batches/{batch['id']}", headers=HEADERS).json()

results = requests.get(f"{API}/batches/{batch['id']}/results", headers=HEADERS)
for line in results.text.splitlines():
    result = json.loads(line)
    if result["error"]:
        print(result["custom_id"], "failed:", result["error"]["message"])
    else:
        print(result["custom_id"], result["response"]["body"]["choices"][0]["message"]["content"])

Results are one JSON line per request, in the order sent, in OpenAI's batch output shape:

{"id": "batch_req_03dffe99_0", "custom_id": "t-1", "error": null,
 "response": {"status_code": 200, "request_id": "…", "body": {
   "object": "chat.completion", "model": "gemini-3.5-flash-lite",
   "choices": [{"index": 0, "message": {"role": "assistant", "content": "billing"}, "finish_reason": "stop"}],
   "usage": {"prompt_tokens": 31, "completion_tokens": 1, "total_tokens": 32}}}}
{"id": "batch_req_03dffe99_1", "custom_id": "t-2", "response": null,
 "error": {"code": "request_failed", "message": "…"}}

Each request's body takes messages, max_tokens, temperature, top_p, stop and response_format. custom_id must be unique. Streaming, tools, n and reasoning_effort are refused, naming the request, before anything is charged.
The worst case (every request using its full max_tokens) is held when the batch starts, and you pay only what was used when it ends. Set max_tokens to hold less.
Up to 20 MB per batch, files included, and 5 running at once. A batch not done in 48 hours expires; answers it finished are still returned and billed.
POST /v1/batches/{id}/cancel stops it. Answers already finished are returned and billed; the rest cost nothing.
GET /v1/batches lists them. Results are kept for 30 days. Add a webhook for batch.completed instead of polling.