Bridge by Wekoya / Docs / Media

Live voice (Gemini Live)

Updated 8 November 2026

One WebSocket that hears the caller and answers out loud: Gemini Live, through Bridge, on your key and balance. There is no separate listening and speaking step, so it answers sooner than a voice agent built from three calls, it handles the caller talking over it, and it speaks many languages, Yoruba, Hausa and Pidgin among them.

The three-call agent takes about 1.8 to 2.3 seconds with the fastest models, and its fast voice speaks English only. Use it when you need a particular chat model, Flux's turn events, or your own speech; use Gemini Live for the quickest, most natural conversation.

Build it right

A voice conversation (a voice assistant or agent, voice chat, people talking to or speaking with the AI, a phone or call assistant, spoken tutoring, live interpreting) is a live, two-way conversation: build it on Gemini Live (wss://wekoya-api.wekoya.tech/v1/realtime), fully duplex. Never as "record, stop, upload, transcribe, reply, play": that is a voice-note feature, slower, and it cannot be interrupted. Voice notes, uploaded recordings and dictation are not conversations: use POST /audio/transcriptions for those.

• The key stays on your server. Your server makes a one-minute, single-use ticket (POST /v1/realtime/tickets) for each call, and returns the ticket's url and the Live setup (system instruction, tools, voice) to the app, so the instructions are set on the server.
• Ask for the microphone before the ticket, so a refused microphone costs nothing. Use echoCancellation, noiseSuppression and autoGainControl, and start audio from a click or tap (browsers block sound until then). On laptop speakers, check the assistant does not hear itself and stop mid-sentence; headphones avoid it.
• Stream the microphone continuously for the whole call, silence included, as 16-bit mono PCM in 20 to 100 ms pieces, sent as binary messages at the ticket's sample_rate. Gemini detects when the person has finished. Do not stop sending when they pause: gemini-3.8-live never answers an app that does.
• Play the reply as it arrives: 24 kHz 16-bit PCM in serverContent.modelTurn.parts[].inlineData, each piece scheduled right after the last. Keep the microphone open while it plays so the person can talk over it; on serverContent.interrupted, stop and drop everything queued at once.
• Captions and records come from the session itself: put inputAudioTranscription: {} and outputAudioTranscription: {} in the setup and read serverContent.inputTranscription.text and outputTranscription.text. Never send the call's audio to /audio/transcriptions as well.
• Lookups and actions are the session's own tools: declare them in setup.tools as functionDeclarations. On toolCall, run each call on your server (with the signed-in person's permissions, never more; treat the arguments as the caller's words, not as trusted input) and send toolResponse with each call's id, name and response, including an error as { "error": "..." }. Skip calls listed in toolCallCancellation. Do not run a separate chat model for them.
• turnComplete ends one turn, not the call: a tool call is a turn of its own before the spoken answer. The call ends only when the person hangs up.
• Keep the latest sessionResumptionUpdate.newHandle (when resumable is true). Bridge carries the conversation over when Gemini rotates its connection (about every ten minutes), so the app sees nothing. If the app's own connection closes with 1001, 1006, 1013 or 4413, get a new ticket and send the same setup with sessionResumption: { handle } to carry on; give up after a few tries.
• Do not retry 4400 (bad setup), 4401 (a message Gemini refused), 4402 (balance or spending limit), 4403 (key revoked) or 4408 (idle): show the error message Bridge sent just before closing.
• Hang up with {"type":"close"}, then stop the microphone tracks and close the audio context. The last message is usage with final: true and the call's cost. Hang up on pagehide too, and when the microphone track ends (a headset unplugged).
• Push-to-talk only when the user asks for it: send realtimeInputConfig: { automaticActivityDetection: { disabled: true } } in the setup, { realtimeInput: { activityStart: {} } } on press and { realtimeInput: { activityEnd: {} } } on release. While muted or between presses, send {"type":"keep_alive"} at least every 4 minutes, or Bridge closes the session as idle.
• Phone lines (Twilio, Africa's Talking and others) run on your server: connect with ?model=...&sample_rate=8000 and the Authorization header, decode the line's 8 kHz mu-law to 16-bit PCM before sending, turn the 24 kHz reply into 8 kHz mu-law for the line, and clear the line's playback on interrupted.
• A guest key for live voice needs uses: ["realtime"] and a max_spend of at least 150: starting a session holds about ₦120. Up to 10 sessions at once per account; a session lasts at most 2 hours.

From a browser

As with live transcription, your server asks for a ticket and the browser connects with it, so the key never reaches the page. A guest key can make tickets too, if it allows realtime in its uses. The server also returns the call's setup, so its instructions and tools are decided on the server. Below is a complete client: continuous microphone, playback as the reply arrives, interruptions, captions, tools and reconnecting, tested end to end in a browser.

// On your server: a ticket for one call, and the call's setup. The key never leaves the server.
app.post("/api/talk", async (req, res) => {
  const user = await signedInUser(req) // your own sign-in check
  if (!user) return res.status(401).json({ error: "Sign in first." })
  const sampleRate = Math.min(48000, Math.max(8000, Math.round(Number(req.body?.sample_rate) || 16000)))
  const reply = await fetch("https://wekoya-api.wekoya.tech/v1/realtime/tickets", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.WEKOYA_API_KEY}`,
      "Content-Type": "application/json",
      "X-Bridge-End-User": user.id, // optional: per-user spend and limits
    },
    body: JSON.stringify({ model: "gemini-3.1-flash-live-preview", sample_rate: sampleRate }),
  })
  const ticket = await reply.json()
  // 402: out of balance or over a limit. 429: too many calls at once. Say so; do not retry at once.
  if (!reply.ok) return res.status(reply.status).json({ error: ticket.error?.message ?? "Could not start the call." })
  res.json({
    url: ticket.url,
    setup: {
      systemInstruction: { parts: [{ text: "You are the phone assistant for Ada's Bakery in Lagos. You are speaking, not writing: one or two short sentences, no lists. Answer in the caller's language." }] },
      generationConfig: { speechConfig: { voiceConfig: { prebuiltVoiceConfig: { voiceName: "Kore" } } } },
      inputAudioTranscription: {}, // what the caller said, as text
      outputAudioTranscription: {}, // what the assistant said, as text
      tools: [{ functionDeclarations: [{
        name: "order_status",
        description: "Where a customer's order is",
        parameters: { type: "object", properties: { order: { type: "string" } }, required: ["order"] },
      }] }],
    },
  })
})

// Tools run here, as the signed-in person, from an allow-list. The arguments are the caller's words: check them.
app.post("/api/tool", async (req, res) => {
  const user = await signedInUser(req)
  if (!user) return res.status(401).json({ error: "Sign in first." })
  const tools = {
    order_status: async ({ order }) => ({ status: await findOrderStatus(user, String(order ?? "")) }),
  }
  const run = tools[req.body?.name]
  if (!run) return res.json({ error: "Unknown tool." })
  try {
    res.json(await run(req.body.args ?? {}))
  } catch {
    res.json({ error: "That lookup failed." })
  }
})

// In the browser: a live, two-way voice call. Call startCall() from a click or tap.
// The microphone streams all the time, the reply plays as it arrives, the person can
// talk over it, tools run on your server, and a dropped connection carries on.
export async function startCall({ onStatus = () => {}, onCaption = () => {}, onEnd = () => {} } = {}) {
  const mic = await navigator.mediaDevices
    .getUserMedia({ audio: { channelCount: 1, echoCancellation: true, noiseSuppression: true, autoGainControl: true } })
    .catch((error) => {
      throw new Error(error?.name === "NotAllowedError" ? "Allow the microphone to talk." : "No microphone was found.")
    })
  const audio = new AudioContext()
  await audio.resume()
  const rate = Math.min(16000, audio.sampleRate) // the rate we send; the ticket is made for the same

  // Microphone to 16-bit PCM at `rate`, in 40 ms pieces, averaged down from the device's rate
  // (inside the worklet, `sampleRate` is the device's).
  const worklet = `registerProcessor("mic", class extends AudioWorkletProcessor {
    constructor() { super(); this.step = sampleRate / ${rate}; this.pos = 0; this.sum = 0; this.count = 0; this.size = ${Math.round(rate * 0.04)}; this.out = new Int16Array(this.size); this.n = 0 }
    process([input]) {
      const samples = input[0]
      if (!samples) return true
      for (let i = 0; i < samples.length; i++) {
        this.sum += samples[i]; this.count++; this.pos += 1
        if (this.pos < this.step) continue
        this.pos -= this.step
        this.out[this.n++] = Math.max(-1, Math.min(1, this.sum / this.count)) * 0x7fff
        this.sum = 0; this.count = 0
        // Handing the buffer over empties it, so the next one is made at the size kept aside.
        if (this.n === this.size) { this.port.postMessage(this.out.buffer, [this.out.buffer]); this.out = new Int16Array(this.size); this.n = 0 }
      }
      return true
    }
  })`
  await audio.audioWorklet.addModule(URL.createObjectURL(new Blob([worklet], { type: "text/javascript" })))
  const source = audio.createMediaStreamSource(mic)
  const capture = new AudioWorkletNode(audio, "mic")
  source.connect(capture)

  let socket
  let setup
  let lastError
  let handle // Gemini's latest resume handle
  let live = false // the session is accepting audio
  let ended = false
  let retries = 0
  const cancelled = new Set()
  const playing = new Set()
  let playAt = 0

  capture.port.onmessage = (event) => {
    if (live && socket?.readyState === WebSocket.OPEN) socket.send(event.data)
  }

  async function connect() {
    onStatus(handle ? "reconnecting" : "connecting")
    const reply = await fetch("/api/talk", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ sample_rate: rate }) })
    const body = await reply.json().catch(() => ({}))
    if (!reply.ok) return end(body.error ?? "Could not start the call.")
    setup = body.setup
    socket = new WebSocket(body.url)
    socket.onmessage = (event) => receive(JSON.parse(event.data))
    socket.onclose = (event) => {
      live = false
      if (ended) return
      // Going away, network, try again, or the 2-hour limit: carry on from the handle.
      if ([1001, 1006, 1013, 4413].includes(event.code) && handle && retries < 3) {
        retries++
        stopPlayback()
        setTimeout(() => connect().catch((error) => end(error.message)), 500 * retries)
        return
      }
      end(lastError ?? "The call ended.")
    }
  }

  function receive(message) {
    if (message.type === "ready") {
      socket.send(JSON.stringify({ setup: handle ? { ...setup, sessionResumption: { handle } } : setup }))
      return
    }
    if (message.type === "error") lastError = message.message // the close that follows says why it ended
    if (message.type === "usage" && message.final) onStatus(`cost ${message.cost} ${message.currency}`)
    if (message.setupComplete) {
      live = true
      retries = 0
      onStatus("listening")
    }
    const update = message.sessionResumptionUpdate
    if (update?.resumable && update.newHandle) handle = update.newHandle
    const content = message.serverContent
    if (content?.interrupted) stopPlayback() // they talked over it
    for (const part of content?.modelTurn?.parts ?? []) if (part.inlineData?.data) play(part.inlineData.data)
    if (content?.inputTranscription?.text) onCaption("caller", content.inputTranscription.text)
    if (content?.outputTranscription?.text) onCaption("assistant", content.outputTranscription.text)
    for (const id of message.toolCallCancellation?.ids ?? []) cancelled.add(id)
    if (message.toolCall) runTools(message.toolCall.functionCalls ?? [])
    // turnComplete ends a turn, not the call: keep listening.
  }

  async function runTools(calls) {
    const functionResponses = await Promise.all(
      calls.map(async (call) => {
        const reply = await fetch("/api/tool", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ name: call.name, args: call.args }) }).catch(() => undefined)
        const response = reply?.ok ? await reply.json().catch(() => ({ error: "The tool answered badly." })) : { error: "The tool is unavailable." }
        return { id: call.id, name: call.name, response }
      }),
    )
    const wanted = functionResponses.filter((item) => !cancelled.has(item.id))
    if (wanted.length && live) socket.send(JSON.stringify({ toolResponse: { functionResponses: wanted } }))
  }

  function play(base64) {
    const bytes = Uint8Array.from(atob(base64), (c) => c.charCodeAt(0))
    const samples = new Int16Array(bytes.buffer, 0, bytes.length >> 1)
    if (!samples.length) return
    const buffer = audio.createBuffer(1, samples.length, 24000) // replies are 24 kHz
    buffer.getChannelData(0).set(Float32Array.from(samples, (x) => x / 32768))
    const node = audio.createBufferSource()
    node.buffer = buffer
    node.connect(audio.destination)
    playAt = Math.max(playAt, audio.currentTime + 0.02)
    node.start(playAt)
    playAt += buffer.duration
    playing.add(node)
    node.onended = () => playing.delete(node)
  }

  function stopPlayback() {
    for (const node of playing) node.stop()
    playing.clear()
    playAt = 0
  }

  function end(reason) {
    if (ended) return
    ended = true
    live = false
    if (socket?.readyState === WebSocket.OPEN) socket.send(JSON.stringify({ type: "close" })) // Bridge sends the final cost, then closes
    stopPlayback()
    capture.port.onmessage = null
    source.disconnect()
    for (const track of mic.getTracks()) track.stop()
    setTimeout(() => audio.close(), 300)
    removeEventListener("pagehide", hangUp)
    document.removeEventListener("visibilitychange", wake)
    onEnd(reason)
  }

  function hangUp() {
    end("You hung up.")
  }

  for (const track of mic.getTracks()) track.onended = () => end("The microphone was disconnected.")
  addEventListener("pagehide", hangUp)
  // Phones suspend sound in the background; pick it up again on return.
  const wake = () => document.visibilityState === "visible" && audio.resume()
  document.addEventListener("visibilitychange", wake)
  await connect()
  return { hangUp }
}

From a server

Connect with your key in the Authorization header and the options in the URL (model, sample_rate, user). Add X-Wekoya-Org to act for a company with a personal key.

// From a server: your key in the header, the model in the URL. Text in, text and audio out,
// and a tool the model can call.
import WebSocket from "ws"

const socket = new WebSocket("wss://wekoya-api.wekoya.tech/v1/realtime?model=gemini-3.8-live", {
  headers: { Authorization: `Bearer ${process.env.WEKOYA_API_KEY}` },
})

let spoke = false
socket.on("message", (data) => {
  const message = JSON.parse(data.toString())
  if (message.type === "ready") {
    socket.send(JSON.stringify({
      setup: {
        tools: [{ functionDeclarations: [{ name: "order_status", description: "Where an order is", parameters: { type: "object", properties: { order: { type: "string" } }, required: ["order"] } }] }],
        outputAudioTranscription: {},
      },
    }))
  }
  if (message.setupComplete) {
    socket.send(JSON.stringify({ clientContent: { turns: [{ role: "user", parts: [{ text: "Where is order 4471?" }] }], turnComplete: true } }))
  }
  if (message.toolCall) {
    const functionResponses = message.toolCall.functionCalls.map((call) => ({ id: call.id, name: call.name, response: { status: "On the way, arriving by 4pm" } }))
    socket.send(JSON.stringify({ toolResponse: { functionResponses } }))
  }
  if (message.serverContent?.outputTranscription) process.stdout.write(message.serverContent.outputTranscription.text)
  // A tool call ends a turn before the spoken answer: hang up after a turn that said something.
  if (message.serverContent?.outputTranscription?.text) spoke = true
  if (message.serverContent?.turnComplete && spoke) socket.send(JSON.stringify({ type: "close" }))
  if (message.type === "usage" && message.final) console.log("\ncost", message.cost, message.currency)
})

Messages

Everything you send and receive is Gemini Live's own format ↗: setup first, then realtimeInput (audio, video frames, text), clientContent and toolResponse. Raw 16-bit mono PCM can also go as binary messages, at the ticket's sample_rate; Bridge wraps it. Bridge adds three messages of its own, each with a type: ready (send your setup), usage and error. Send {"type":"close"} to finish.

{ "type": "ready", "id": "5c0e…", "model": "gemini-3.1-flash-live-preview", "sample_rate": 16000 }   // Bridge
{ "setupComplete": {} }                                                                          // Gemini, from here on
{ "serverContent": { "inputTranscription": { "text": "Can I change my delivery address?" } } }
{ "serverContent": { "modelTurn": { "parts": [{ "inlineData": { "mimeType": "audio/pcm;rate=24000", "data": "…" } }] } } }
{ "serverContent": { "outputTranscription": { "text": "Yes, just tell me the new address." } } }
{ "type": "usage", "tokens": { "input": 261, "input_audio": 112, "output": 134, "output_audio": 134 }, "cost": 3.06, "currency": "NGN", "final": false }   // Bridge, the cost so far, after each turn
{ "usageMetadata": { "promptTokenCount": 261, "responseTokenCount": 134, … } }
{ "serverContent": { "turnComplete": true } }
{ "sessionResumptionUpdate": { "newHandle": "dc87…", "resumable": true } }
{ "type": "usage", "tokens": { "input": 261, "input_audio": 112, "output": 134, "output_audio": 134 }, "cost": 3.06, "currency": "NGN", "final": true }   // Bridge, always last

Bridge sets the model from the ticket (a setup naming another is refused), and turns on a sliding context window at 32,000 tokens unless you send your own contextWindowCompression: Gemini Live bills the whole conversation again on every turn, so without it a long call gets dearer by the minute. Gemini ends each connection after about nine minutes; Bridge carries the conversation over to a new one itself, with Gemini's resume handle and your setup, so your app sees no break. Keep the latest sessionResumptionUpdate handle anyway: if your own connection to Bridge drops, send it in a new session's setup.sessionResumption to carry on the same conversation.

What it costs

Exactly what Gemini reports for each turn, at its rates per kind of token: audio heard at about ₦7.43 a minute, audio spoken at about ₦26.73 a minute, and text far cheaper. Each turn also re-reads the conversation so far, which the sliding window keeps in check. A typical minute of a call, with the agent speaking for about 25 seconds of it, comes to roughly ₦19, against about ₦34 for the three-call agent. A turn cut off by a dropped connection is billed from the audio that flowed. Starting a session needs about ₦120 free (the first block, held and released as it is used), so a guest key for live voice needs a max_spend of at least ₦150; a usage message gives the cost so far after each turn, and always one at the end. Up to 10 sessions at once per account. Key scopes, model allow-lists and end users (X-Bridge-End-User on the ticket request) apply as for every other call. Audio is not used to train any model.