# Live voice (Gemini Live) URL: https://bridge.wekoya.tech/docs/realtime Updated: 2026-11-08 > One WebSocket that hears the caller and answers out loud: Gemini Live, through Bridge, on your key and balance. There is no separate listening and speaking… One WebSocket that hears the caller and answers out loud: Gemini Live, through Bridge, on your key and balance. There is no separate listening and speaking step, so it answers sooner than a [voice agent built from three calls](https://bridge.wekoya.tech/docs/voice-agents), it handles the caller talking over it, and it speaks many languages, Yoruba, Hausa and Pidgin among them. Model | First sound after the caller stops (measured) | `gemini-3.1-flash-live-preview` (default) | About 1.3 seconds. The fastest. A preview, so its name may change. | `gemini-3.8-live` | About 1.7 seconds, with fuller answers. | `gemini-3.8-live-extended-thinking` | Slower: it thinks before answering, for harder questions. | The three-call agent takes about 1.8 to 2.3 seconds with the fastest models, and its fast voice speaks English only. Use it when you need a particular chat model, Flux's turn events, or your own speech; use Gemini Live for the quickest, most natural conversation. #### Build it right A voice conversation (a voice assistant or agent, voice chat, people talking to or speaking with the AI, a phone or call assistant, spoken tutoring, live interpreting) is a live, two-way conversation: build it on Gemini Live (`wss://wekoya-api.wekoya.tech/v1/realtime`), fully duplex. Never as "record, stop, upload, transcribe, reply, play": that is a voice-note feature, slower, and it cannot be interrupted. Voice notes, uploaded recordings and dictation are not conversations: use `POST /audio/transcriptions` for those. - The key stays on your server. Your server makes a one-minute, single-use ticket (`POST /v1/realtime/tickets`) for each call, and returns the ticket's `url` and the Live `setup` (system instruction, tools, voice) to the app, so the instructions are set on the server. - Ask for the microphone before the ticket, so a refused microphone costs nothing. Use `echoCancellation`, `noiseSuppression` and `autoGainControl`, and start audio from a click or tap (browsers block sound until then). On laptop speakers, check the assistant does not hear itself and stop mid-sentence; headphones avoid it. - Stream the microphone continuously for the whole call, silence included, as 16-bit mono PCM in 20 to 100 ms pieces, sent as binary messages at the ticket's `sample_rate`. Gemini detects when the person has finished. Do not stop sending when they pause: `gemini-3.8-live` never answers an app that does. - Play the reply as it arrives: 24 kHz 16-bit PCM in `serverContent.modelTurn.parts[].inlineData`, each piece scheduled right after the last. Keep the microphone open while it plays so the person can talk over it; on `serverContent.interrupted`, stop and drop everything queued at once. - Captions and records come from the session itself: put `inputAudioTranscription: {}` and `outputAudioTranscription: {}` in the setup and read `serverContent.inputTranscription.text` and `outputTranscription.text`. Never send the call's audio to `/audio/transcriptions` as well. - Lookups and actions are the session's own tools: declare them in `setup.tools` as `functionDeclarations`. On `toolCall`, run each call on your server (with the signed-in person's permissions, never more; treat the arguments as the caller's words, not as trusted input) and send `toolResponse` with each call's `id`, `name` and `response`, including an error as `{ "error": "..." }`. Skip calls listed in `toolCallCancellation`. Do not run a separate chat model for them. - `turnComplete` ends one turn, not the call: a tool call is a turn of its own before the spoken answer. The call ends only when the person hangs up. - Keep the latest `sessionResumptionUpdate.newHandle` (when `resumable` is true). Bridge carries the conversation over when Gemini rotates its connection (about every ten minutes), so the app sees nothing. If the app's own connection closes with 1001, 1006, 1013 or 4413, get a new ticket and send the same setup with `sessionResumption: { handle }` to carry on; give up after a few tries. - Do not retry 4400 (bad setup), 4401 (a message Gemini refused), 4402 (balance or spending limit), 4403 (key revoked) or 4408 (idle): show the `error` message Bridge sent just before closing. - Hang up with `{"type":"close"}`, then stop the microphone tracks and close the audio context. The last message is `usage` with `final: true` and the call's cost. Hang up on `pagehide` too, and when the microphone track ends (a headset unplugged). - Push-to-talk only when the user asks for it: send `realtimeInputConfig: { automaticActivityDetection: { disabled: true } }` in the setup, `{ realtimeInput: { activityStart: {} } }` on press and `{ realtimeInput: { activityEnd: {} } }` on release. While muted or between presses, send `{"type":"keep_alive"}` at least every 4 minutes, or Bridge closes the session as idle. - Phone lines (Twilio, Africa's Talking and others) run on your server: connect with `?model=...&sample_rate=8000` and the `Authorization` header, decode the line's 8 kHz mu-law to 16-bit PCM before sending, turn the 24 kHz reply into 8 kHz mu-law for the line, and clear the line's playback on `interrupted`. - A guest key for live voice needs `uses: ["realtime"]` and a `max_spend` of at least 150: starting a session holds about ₦120. Up to 10 sessions at once per account; a session lasts at most 2 hours. #### From a browser As with live transcription, your server asks for a ticket and the browser connects with it, so the key never reaches the page. A guest key can make tickets too, if it allows `realtime` in its `uses`. The server also returns the call's setup, so its instructions and tools are decided on the server. Below is a complete client: continuous microphone, playback as the reply arrives, interruptions, captions, tools and reconnecting, tested end to end in a browser. ``` // On your server: a ticket for one call, and the call's setup. The key never leaves the server. app.post("/api/talk", async (req, res) => { const user = await signedInUser(req) // your own sign-in check if (!user) return res.status(401).json({ error: "Sign in first." }) const sampleRate = Math.min(48000, Math.max(8000, Math.round(Number(req.body?.sample_rate) || 16000))) const reply = await fetch("https://wekoya-api.wekoya.tech/v1/realtime/tickets", { method: "POST", headers: { Authorization: `Bearer ${process.env.WEKOYA_API_KEY}`, "Content-Type": "application/json", "X-Bridge-End-User": user.id, // optional: per-user spend and limits }, body: JSON.stringify({ model: "gemini-3.1-flash-live-preview", sample_rate: sampleRate }), }) const ticket = await reply.json() // 402: out of balance or over a limit. 429: too many calls at once. Say so; do not retry at once. if (!reply.ok) return res.status(reply.status).json({ error: ticket.error?.message ?? "Could not start the call." }) res.json({ url: ticket.url, setup: { systemInstruction: { parts: [{ text: "You are the phone assistant for Ada's Bakery in Lagos. You are speaking, not writing: one or two short sentences, no lists. Answer in the caller's language." }] }, generationConfig: { speechConfig: { voiceConfig: { prebuiltVoiceConfig: { voiceName: "Kore" } } } }, inputAudioTranscription: {}, // what the caller said, as text outputAudioTranscription: {}, // what the assistant said, as text tools: [{ functionDeclarations: [{ name: "order_status", description: "Where a customer's order is", parameters: { type: "object", properties: { order: { type: "string" } }, required: ["order"] }, }] }], }, }) }) // Tools run here, as the signed-in person, from an allow-list. The arguments are the caller's words: check them. app.post("/api/tool", async (req, res) => { const user = await signedInUser(req) if (!user) return res.status(401).json({ error: "Sign in first." }) const tools = { order_status: async ({ order }) => ({ status: await findOrderStatus(user, String(order ?? "")) }), } const run = tools[req.body?.name] if (!run) return res.json({ error: "Unknown tool." }) try { res.json(await run(req.body.args ?? {})) } catch { res.json({ error: "That lookup failed." }) } }) ``` ``` // In the browser: a live, two-way voice call. Call startCall() from a click or tap. // The microphone streams all the time, the reply plays as it arrives, the person can // talk over it, tools run on your server, and a dropped connection carries on. export async function startCall({ onStatus = () => {}, onCaption = () => {}, onEnd = () => {} } = {}) { const mic = await navigator.mediaDevices .getUserMedia({ audio: { channelCount: 1, echoCancellation: true, noiseSuppression: true, autoGainControl: true } }) .catch((error) => { throw new Error(error?.name === "NotAllowedError" ? "Allow the microphone to talk." : "No microphone was found.") }) const audio = new AudioContext() await audio.resume() const rate = Math.min(16000, audio.sampleRate) // the rate we send; the ticket is made for the same // Microphone to 16-bit PCM at `rate`, in 40 ms pieces, averaged down from the device's rate // (inside the worklet, `sampleRate` is the device's). const worklet = `registerProcessor("mic", class extends AudioWorkletProcessor { constructor() { super(); this.step = sampleRate / ${rate}; this.pos = 0; this.sum = 0; this.count = 0; this.size = ${Math.round(rate * 0.04)}; this.out = new Int16Array(this.size); this.n = 0 } process([input]) { const samples = input[0] if (!samples) return true for (let i = 0; i < samples.length; i++) { this.sum += samples[i]; this.count++; this.pos += 1 if (this.pos < this.step) continue this.pos -= this.step this.out[this.n++] = Math.max(-1, Math.min(1, this.sum / this.count)) * 0x7fff this.sum = 0; this.count = 0 // Handing the buffer over empties it, so the next one is made at the size kept aside. if (this.n === this.size) { this.port.postMessage(this.out.buffer, [this.out.buffer]); this.out = new Int16Array(this.size); this.n = 0 } } return true } })` await audio.audioWorklet.addModule(URL.createObjectURL(new Blob([worklet], { type: "text/javascript" }))) const source = audio.createMediaStreamSource(mic) const capture = new AudioWorkletNode(audio, "mic") source.connect(capture) let socket let setup let lastError let handle // Gemini's latest resume handle let live = false // the session is accepting audio let ended = false let retries = 0 const cancelled = new Set() const playing = new Set() let playAt = 0 capture.port.onmessage = (event) => { if (live && socket?.readyState === WebSocket.OPEN) socket.send(event.data) } async function connect() { onStatus(handle ? "reconnecting" : "connecting") const reply = await fetch("/api/talk", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ sample_rate: rate }) }) const body = await reply.json().catch(() => ({})) if (!reply.ok) return end(body.error ?? "Could not start the call.") setup = body.setup socket = new WebSocket(body.url) socket.onmessage = (event) => receive(JSON.parse(event.data)) socket.onclose = (event) => { live = false if (ended) return // Going away, network, try again, or the 2-hour limit: carry on from the handle. if ([1001, 1006, 1013, 4413].includes(event.code) && handle && retries < 3) { retries++ stopPlayback() setTimeout(() => connect().catch((error) => end(error.message)), 500 * retries) return } end(lastError ?? "The call ended.") } } function receive(message) { if (message.type === "ready") { socket.send(JSON.stringify({ setup: handle ? { ...setup, sessionResumption: { handle } } : setup })) return } if (message.type === "error") lastError = message.message // the close that follows says why it ended if (message.type === "usage" && message.final) onStatus(`cost ${message.cost} ${message.currency}`) if (message.setupComplete) { live = true retries = 0 onStatus("listening") } const update = message.sessionResumptionUpdate if (update?.resumable && update.newHandle) handle = update.newHandle const content = message.serverContent if (content?.interrupted) stopPlayback() // they talked over it for (const part of content?.modelTurn?.parts ?? []) if (part.inlineData?.data) play(part.inlineData.data) if (content?.inputTranscription?.text) onCaption("caller", content.inputTranscription.text) if (content?.outputTranscription?.text) onCaption("assistant", content.outputTranscription.text) for (const id of message.toolCallCancellation?.ids ?? []) cancelled.add(id) if (message.toolCall) runTools(message.toolCall.functionCalls ?? []) // turnComplete ends a turn, not the call: keep listening. } async function runTools(calls) { const functionResponses = await Promise.all( calls.map(async (call) => { const reply = await fetch("/api/tool", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ name: call.name, args: call.args }) }).catch(() => undefined) const response = reply?.ok ? await reply.json().catch(() => ({ error: "The tool answered badly." })) : { error: "The tool is unavailable." } return { id: call.id, name: call.name, response } }), ) const wanted = functionResponses.filter((item) => !cancelled.has(item.id)) if (wanted.length && live) socket.send(JSON.stringify({ toolResponse: { functionResponses: wanted } })) } function play(base64) { const bytes = Uint8Array.from(atob(base64), (c) => c.charCodeAt(0)) const samples = new Int16Array(bytes.buffer, 0, bytes.length >> 1) if (!samples.length) return const buffer = audio.createBuffer(1, samples.length, 24000) // replies are 24 kHz buffer.getChannelData(0).set(Float32Array.from(samples, (x) => x / 32768)) const node = audio.createBufferSource() node.buffer = buffer node.connect(audio.destination) playAt = Math.max(playAt, audio.currentTime + 0.02) node.start(playAt) playAt += buffer.duration playing.add(node) node.onended = () => playing.delete(node) } function stopPlayback() { for (const node of playing) node.stop() playing.clear() playAt = 0 } function end(reason) { if (ended) return ended = true live = false if (socket?.readyState === WebSocket.OPEN) socket.send(JSON.stringify({ type: "close" })) // Bridge sends the final cost, then closes stopPlayback() capture.port.onmessage = null source.disconnect() for (const track of mic.getTracks()) track.stop() setTimeout(() => audio.close(), 300) removeEventListener("pagehide", hangUp) document.removeEventListener("visibilitychange", wake) onEnd(reason) } function hangUp() { end("You hung up.") } for (const track of mic.getTracks()) track.onended = () => end("The microphone was disconnected.") addEventListener("pagehide", hangUp) // Phones suspend sound in the background; pick it up again on return. const wake = () => document.visibilityState === "visible" && audio.resume() document.addEventListener("visibilitychange", wake) await connect() return { hangUp } } ``` #### From a server Connect with your key in the `Authorization` header and the options in the URL (`model`, `sample_rate`, `user`). Add `X-Wekoya-Org` to act for a company with a personal key. ``` // From a server: your key in the header, the model in the URL. Text in, text and audio out, // and a tool the model can call. import WebSocket from "ws" const socket = new WebSocket("wss://wekoya-api.wekoya.tech/v1/realtime?model=gemini-3.8-live", { headers: { Authorization: `Bearer ${process.env.WEKOYA_API_KEY}` }, }) let spoke = false socket.on("message", (data) => { const message = JSON.parse(data.toString()) if (message.type === "ready") { socket.send(JSON.stringify({ setup: { tools: [{ functionDeclarations: [{ name: "order_status", description: "Where an order is", parameters: { type: "object", properties: { order: { type: "string" } }, required: ["order"] } }] }], outputAudioTranscription: {}, }, })) } if (message.setupComplete) { socket.send(JSON.stringify({ clientContent: { turns: [{ role: "user", parts: [{ text: "Where is order 4471?" }] }], turnComplete: true } })) } if (message.toolCall) { const functionResponses = message.toolCall.functionCalls.map((call) => ({ id: call.id, name: call.name, response: { status: "On the way, arriving by 4pm" } })) socket.send(JSON.stringify({ toolResponse: { functionResponses } })) } if (message.serverContent?.outputTranscription) process.stdout.write(message.serverContent.outputTranscription.text) // A tool call ends a turn before the spoken answer: hang up after a turn that said something. if (message.serverContent?.outputTranscription?.text) spoke = true if (message.serverContent?.turnComplete && spoke) socket.send(JSON.stringify({ type: "close" })) if (message.type === "usage" && message.final) console.log("\ncost", message.cost, message.currency) }) ``` #### Messages Everything you send and receive is [Gemini Live's own format ↗](https://ai.google.dev/gemini-api/docs/live): `setup` first, then `realtimeInput` (audio, video frames, text), `clientContent` and `toolResponse`. Raw 16-bit mono PCM can also go as binary messages, at the ticket's `sample_rate`; Bridge wraps it. Bridge adds three messages of its own, each with a `type`: `ready` (send your setup), `usage` and `error`. Send `{"type":"close"}` to finish. ``` { "type": "ready", "id": "5c0e…", "model": "gemini-3.1-flash-live-preview", "sample_rate": 16000 } // Bridge { "setupComplete": {} } // Gemini, from here on { "serverContent": { "inputTranscription": { "text": "Can I change my delivery address?" } } } { "serverContent": { "modelTurn": { "parts": [{ "inlineData": { "mimeType": "audio/pcm;rate=24000", "data": "…" } }] } } } { "serverContent": { "outputTranscription": { "text": "Yes, just tell me the new address." } } } { "type": "usage", "tokens": { "input": 261, "input_audio": 112, "output": 134, "output_audio": 134 }, "cost": 3.06, "currency": "NGN", "final": false } // Bridge, the cost so far, after each turn { "usageMetadata": { "promptTokenCount": 261, "responseTokenCount": 134, … } } { "serverContent": { "turnComplete": true } } { "sessionResumptionUpdate": { "newHandle": "dc87…", "resumable": true } } { "type": "usage", "tokens": { "input": 261, "input_audio": 112, "output": 134, "output_audio": 134 }, "cost": 3.06, "currency": "NGN", "final": true } // Bridge, always last ``` Bridge sets the model from the ticket (a setup naming another is refused), and turns on a sliding context window at 32,000 tokens unless you send your own `contextWindowCompression`: Gemini Live bills the whole conversation again on every turn, so without it a long call gets dearer by the minute. Gemini ends each connection after about nine minutes; Bridge carries the conversation over to a new one itself, with Gemini's resume handle and your setup, so your app sees no break. Keep the latest `sessionResumptionUpdate` handle anyway: if your own connection to Bridge drops, send it in a new session's `setup.sessionResumption` to carry on the same conversation. Close code | Why | `1000` | Finished normally. | `1001` | Bridge is restarting, or Gemini's connection could not be resumed. Reconnect with your resume handle. | `1013` | Temporarily unavailable. Reconnect with backoff. | `4400` | The setup was refused; the error message before it gives Gemini's reason. Not charged. | `4401` | A message Gemini could not accept, or a frame over its size (4 MB for JSON, 256 KB for binary audio). | `4402` | The balance or the end user's budget ran out. | `4403` | The key that started the session was revoked. | `4408` | Nothing arrived from you for 5 minutes. | `4413` | The session reached Bridge's 2-hour limit. Reconnect with your resume handle to go on. | `4429` | Audio arrived much faster than real time. | #### What it costs Exactly what Gemini reports for each turn, at its rates per kind of token: audio heard at about ₦7.43 a minute, audio spoken at about ₦26.73 a minute, and text far cheaper. Each turn also re-reads the conversation so far, which the sliding window keeps in check. A typical minute of a call, with the agent speaking for about 25 seconds of it, comes to roughly ₦19, against about ₦34 for the three-call agent. A turn cut off by a dropped connection is billed from the audio that flowed. Starting a session needs about ₦120 free (the first block, held and released as it is used), so a guest key for live voice needs a `max_spend` of at least ₦150; a `usage` message gives the cost so far after each turn, and always one at the end. Up to 10 sessions at once per account. Key scopes, model allow-lists and end users (`X-Bridge-End-User` on the ticket request) apply as for every other call. Audio is not used to train any model.