Streaming

Updated 8 November 2026

Set stream: true and the reply arrives in pieces. This is what makes an assistant feel like it is thinking rather than hanging. Long reasoning models are kept alive while they work, so a slow first token still arrives.

stream = client.chat.completions.create(
    model="auto",
    stream=True,
    messages=[{"role": "user", "content": "Write a haiku about Lagos."}],
)

for chunk in stream:
    piece = chunk.choices[0].delta.content
    if piece:
        print(piece, end="", flush=True)