Bridge by Wekoya / Docs / Media

Voice agents

Updated 8 November 2026

A voice agent is three Bridge calls in a loop: it listens with live transcription (flux-general-en, which says when the caller has finished), thinks with a chat model (any model on Bridge, streamed), and speaks with aura-2, sentence by sentence as the reply is written. One key and one balance cover all three. The code below is complete apart from your audio input and output, and was run against Bridge before it was published.

import OpenAI from "openai"

const bridge = new OpenAI({ apiKey: process.env.WEKOYA_API_KEY, baseURL: "https://wekoya-api.wekoya.tech/v1" })
const MODEL = "gemini-3.5-flash-lite" // fast to its first word; see "Choosing the model"

const history = [
  {
    role: "system",
    content:
      "You are the phone assistant for Ada's Bakery in Lagos. You are speaking, not writing: answer in one or two " +
      "short sentences, no lists, no markdown, no emoji. Say amounts the way people say them, like 'two thousand " +
      "five hundred naira'. Ask one question at a time.",
  },
]

let current // the reply being spoken, so the caller can interrupt it

async function respond(callerSaid, play) {
  history.push({ role: "user", content: callerSaid })
  const abort = new AbortController()
  const turn = (current = { abort, spoken: "" })
  const stream = await bridge.chat.completions.create(
    { model: MODEL, messages: history, stream: true, max_tokens: 150 },
    { signal: abort.signal },
  )

  // Each sentence's speech starts as soon as the sentence is written. Playback
  // keeps their order and starts on the first audio bytes, not the whole clip.
  let playing = Promise.resolve()
  const say = (sentence) => {
    const audio = bridge.audio.speech.create(
      { model: "aura-2", voice: "thalia", input: sentence, response_format: "pcm" }, // 24 kHz 16-bit mono
      { signal: abort.signal },
    )
    playing = playing.then(async () => {
      const response = await audio
      for await (const chunk of response.body) {
        if (abort.signal.aborted) return
        play(chunk)
      }
      turn.spoken += (turn.spoken ? " " : "") + sentence
    })
  }

  let buffer = ""
  try {
    for await (const chunk of stream) {
      buffer += chunk.choices[0]?.delta?.content ?? ""
      let sentence
      while ((sentence = buffer.match(/^(.+?[.!?])(\s+|$)/s))) {
        say(sentence[1].trim())
        buffer = buffer.slice(sentence[0].length)
      }
    }
    if (buffer.trim()) say(buffer.trim())
    await playing
  } catch (error) {
    if (!abort.signal.aborted) throw error
  } finally {
    // Remember only what the caller actually heard.
    if (turn.spoken) history.push({ role: "assistant", content: turn.spoken })
    if (current === turn) current = undefined
  }
}

function interrupt() {
  current?.abort.abort()
  current = undefined
  stopPlayback() // your audio output: clear whatever is still queued
}

// Listen with Flux, which says when the caller has finished their turn.
const ticket = await (
  await fetch("https://wekoya-api.wekoya.tech/v1/audio/live/tickets", {
    method: "POST",
    headers: { Authorization: `Bearer ${process.env.WEKOYA_API_KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify({ model: "flux-general-en", encoding: "linear16", sample_rate: 16000 }),
  })
).json()

const socket = new WebSocket(ticket.url)
socket.onopen = () => callerAudio.on("data", (pcm) => socket.send(pcm)) // 16 kHz 16-bit mono, continuously
socket.onmessage = (event) => {
  const message = JSON.parse(event.data)
  if (message.type !== "turn") return
  if (message.event === "start" && current) interrupt() // the caller talked over the agent
  if (message.event === "end") respond(message.text, play)
}

What makes it feel natural

• Speak sentence by sentence. Waiting for the whole reply before speaking adds seconds. Above, speech starts when the first sentence is written, and playback starts on the first audio bytes.
• Let the caller interrupt. A new turn starting while the agent speaks cancels the reply, stops playback, and keeps only what was actually said in the history, so the model does not believe it said things the caller never heard.
• Write the system prompt for the ear. Short sentences, no lists or markdown, amounts and dates the way people say them, one question at a time. Ask for an opening phrase that is short, so the first sentence (and the first sound) comes quickly.
• Keep the history short on long calls. Every turn resends it; past a few minutes, summarise older turns into one message.

Choosing the model

In conversation, the pause before the agent answers matters more than anything. Measured on Bridge: gemini-3.5-flash-lite starts its answer in about 0.5 seconds, gemini-2.5-flash-lite in about 0.9, and speech starts about 0.4 seconds after a sentence is sent. Expect the agent to start talking 1 to 2 seconds after the caller stops. Larger models think better and start later; reasoning models also spend time thinking before their first word, so keep reasoning_effort low for voice.

Fast enough for a live call, measured on Bridge (time to the first word): gemini-3.5-flash-lite, ministral-3-3b, gemini-2.5-flash-lite, nova-lite, ministral-3-8b and gpt-oss-20b (about 0.5 to 0.6 seconds); claude-haiku-5-5 and gpt-6-luna answer better and start in about 0.9. Too slow for a call, because they think first: gemini-3.8-flash, deepseek-v4-flash, qwen3.8-flash, gemma-4 and glm-5.3-flash (2 to 11 seconds). For the shortest pause of all, and for answering in Yoruba, Hausa or Pidgin, see Live voice (Gemini Live): one connection that hears and speaks.

Answering sooner: eager end of turn

Flux can say a turn is probably over before it is sure. Start the reply then, and throw it away if the caller carries on. This saves a few hundred milliseconds, at the cost of some replies generated and not used.

// Ask for early ends of turn when getting the ticket:
//   { "model": "flux-general-en", "encoding": "linear16", "sample_rate": 16000, "eager_eot_threshold": 0.5 }
// then start thinking on "eager_end", and drop that draft if the caller carries on.
let draft
socket.onmessage = (event) => {
  const message = JSON.parse(event.data)
  if (message.type !== "turn") return
  if (message.event === "eager_end") draft = startReply(message.text)           // generate, do not speak yet
  if (message.event === "resumed") draft = draft?.cancel()                     // they were not finished
  if (message.event === "end") draft?.text === message.text ? draft.speak() : respond(message.text, play)
}

Taking actions

// Tools work as in chat. Say something while the tool runs, so the line is not silent.
const tools = [{ type: "function", function: { name: "order_status", parameters: { type: "object", properties: { order: { type: "string" } }, required: ["order"] } } }]
// When the stream ends with a tool call:
say("Let me check that for you.")
const status = await orderStatus(args.order)
history.push(assistantToolCallMessage, { role: "tool", tool_call_id: call.id, content: JSON.stringify(status) })
// …then call chat.completions.create again with the same history, and speak the answer.

Phone calls

Phone audio is 8 kHz mu-law (or A-law). Listen with encoding: "mulaw" and sample_rate: 8000, and ask speech for response_format: "mulaw", so audio passes straight through with no conversion.

// Phone lines carry 8 kHz mu-law. Listen in it, and speak in it, so nothing is converted.
const ticket = await getTicket({ model: "flux-general-en", encoding: "mulaw", sample_rate: 8000 })
const audio = await bridge.audio.speech.create({ model: "aura-2", voice: "thalia", input: sentence, response_format: "mulaw" })

// Twilio Media Streams, for example, sends base64 mu-law in JSON and takes the same back:
twilio.on("message", (raw) => {
  const event = JSON.parse(raw)
  if (event.event === "media") socket.send(Buffer.from(event.media.payload, "base64"))
})
const play = (chunk) => twilio.send(JSON.stringify({ event: "media", streamSid, media: { payload: Buffer.from(chunk).toString("base64") } }))
const stopPlayback = () => twilio.send(JSON.stringify({ event: "clear", streamSid })) // on interruption

What a call costs

Give each caller an end user (X-Bridge-End-User on the ticket and on every chat and speech call) with a budget, and one long or abusive call cannot spend more than you allowed it: the stream closes with code 4402 when the budget is used.

Using Bridge from a voice platform

If you build on a voice platform instead, you can still use Bridge as its brain wherever it lets you bring an OpenAI-compatible LLM: Bridge implements OpenAI's streaming chat completions, which is what those platforms call.

# Any voice platform that lets you bring an OpenAI-compatible LLM can use Bridge as the brain.
Base URL:  https://wekoya-api.wekoya.tech/v1
API key:   your Bridge key (wk_live_…)
Model:     any id from GET /v1/models, for example gemini-3.5-flash-lite