Bridge by Wekoya / Docs / Media
Live transcription
Updated
Stream audio over a WebSocket and get the words back while people are still speaking: live captions, call notes, voice commands, and the listening half of a voice agent. Two models, both by Deepgram: nova-3 for transcripts, and flux-general-en for voice agents, which also says when the speaker has finished their turn.
From a browser
Never put your key in the page. Your server asks for a ticket, a one-minute, single-use pass for one stream, and the browser connects with it. Every check (balance, limits, options) happens when the ticket is made, so a refusal arrives as ordinary JSON your server can show.
// On your server: a one-minute, single-use ticket. The browser never sees your key.
app.post("/api/listen", async (req, res) => {
const reply = await fetch("https://wekoya-api.wekoya.tech/v1/audio/live/tickets", {
method: "POST",
headers: { Authorization: `Bearer ${process.env.WEKOYA_API_KEY}`, "Content-Type": "application/json" },
// encoding "container": the browser sends webm/opus as MediaRecorder makes it.
body: JSON.stringify({ model: "nova-3", encoding: "container", language: "en" }),
})
if (!reply.ok) return res.status(reply.status).json(await reply.json()) // e.g. 402 when the balance is low
res.json(await reply.json()) // { ticket, url, expires_in, model, mode }
})// In the browser: microphone in, words out as the person speaks.
const { url } = await (await fetch("/api/listen", { method: "POST" })).json()
const socket = new WebSocket(url)
const mic = await navigator.mediaDevices.getUserMedia({ audio: true })
const recorder = new MediaRecorder(mic, { mimeType: "audio/webm;codecs=opus" })
socket.onopen = () => recorder.start(250) // a piece every 250 ms
recorder.ondataavailable = (event) => socket.readyState === 1 && socket.send(event.data)
socket.onmessage = (event) => {
const message = JSON.parse(event.data)
if (message.type === "transcript" && message.is_final) showText(message.text)
if (message.type === "transcript" && !message.is_final) showDraft(message.text) // may still change
if (message.type === "usage" && message.final) console.log("cost", message.cost, message.currency)
if (message.type === "error") console.warn(message.code, message.message)
}
socket.onclose = (event) => console.log("closed", event.code, event.reason)
// To stop: let the last words come back, then close.
function stop() {
recorder.stop()
socket.send(JSON.stringify({ type: "close" }))
}From a server
A server can skip the ticket and connect with its key in the Authorization header, putting the options in the URL. A refusal is then the handshake's HTTP status with the same JSON body.
// From a server: your key in the header, options in the URL, raw 16 kHz audio.
// Works with any WebSocket client that sets headers (Bun's built-in, or the "ws" package on Node).
import WebSocket from "ws"
const socket = new WebSocket(
"wss://wekoya-api.wekoya.tech/v1/audio/live?model=nova-3&encoding=linear16&sample_rate=16000&utterance_end_ms=1000",
{ headers: { Authorization: `Bearer ${process.env.WEKOYA_API_KEY}` } },
)
socket.on("open", () => audioSource.on("data", (pcm) => socket.send(pcm))) // 16-bit mono PCM
socket.on("message", (data) => console.log(JSON.parse(data.toString())))Options
| Option | Meaning |
|---|---|
model | nova-3 (default) or flux-general-en (English, turn detection, raw audio only). |
encoding | linear16 (16-bit PCM, default), mulaw or alaw (phone audio), or container for webm, ogg, wav or mp3 as recorded. |
sample_rate | Required for raw audio: 16000 for a microphone, 8000 for a phone line. Not used with container. |
language | en by default; another code such as fr, or multi for mixed languages (nova-3 only). |
interim_results | true by default: drafts while a phrase is spoken, marked is_final: false. |
endpointing, utterance_end_ms | Milliseconds of silence that end a phrase, and that mark the end of an utterance with an utterance_end event (needs interim results). |
smart_format, vad_events | Numbers and dates written as such, and speech_started events. Both on by default. |
eot_threshold, eager_eot_threshold, eot_timeout_ms | flux-general-en: how sure it must be that a turn has ended (0.5 to 0.9), an earlier, eager signal, and the most silence before a turn ends anyway. |
What you receive
With nova-3:
{ "type": "ready", "id": "a955aed0-…", "model": "nova-3", "mode": "transcripts", "language": "en", "encoding": "linear16", "sample_rate": 16000 }
{ "type": "speech_started", "at": 0.13 }
{ "type": "transcript", "text": "Hello. I want to check my account balance.", "is_final": true, "speech_final": false,
"start": 0, "end": 3.74, "words": [{ "word": "Hello.", "start": 0, "end": 0.56, "confidence": 0.93 }] }
{ "type": "utterance_end", "last_word_end": 5.58 }
{ "type": "usage", "seconds": 300, "cost": 67.57, "currency": "NGN", "final": false } // after each 5 minutes
{ "type": "error", "code": "insufficient_funds", "message": "…" } // then the socket closes
{ "type": "usage", "seconds": 307, "cost": 69.15, "currency": "NGN", "final": true } // always lastWith flux-general-en, turns instead of transcripts. event is start, update, eager_end, resumed or end:
{ "type": "turn", "event": "start", "turn": 0, "text": "I would", "start": 0, "end": 0.72, "end_of_turn_confidence": 0.0047, "words": [] }
{ "type": "turn", "event": "update", "turn": 0, "text": "I would like to pay my", … }
{ "type": "turn", "event": "end", "turn": 0, "text": "I would like to pay my electricity bill, please.", "start": 0, "end": 2.56, "end_of_turn_confidence": 0.7959, … }You send audio as binary messages, in pieces of 20 to 250 ms (at most 256 KB), and three text messages: {"type":"keep_alive"} while paused, {"type":"finalize"} to get everything heard so far as final, and {"type":"close"} to finish: the last words and the final usage arrive, then the socket closes.
| Close code | Why |
|---|---|
1000 | Finished normally. |
1001 | Bridge is restarting. Reconnect; what was heard has been billed. |
1013 | Temporarily unavailable. Reconnect with backoff. |
4400 | The audio could not be read: check encoding and sample_rate match what you send. |
4401 | A frame over 256 KB. |
4402 | The balance or the end user's budget ran out. The error message before it says which. |
4408 | No audio or keep_alive for 60 seconds. |
4413 | The stream reached its 2-hour limit. Reconnect to continue. |
4429 | Audio arrived much faster than real time, faster than it can be processed. |
Billed by the second of audio you send: about ₦13.51 a minute (₦16.15 with multi), for both models. Money is held five minutes at a time before it is needed, so a stream stops cleanly, with a reason, if the balance runs out, never runs up a debt. A usage message after each five minutes, and always one at the end, says what it has cost. Up to 20 streams at once per account. Key scopes, model allow-lists and end users (X-Bridge-End-User on the ticket request) apply as for every other call. Recordings are not used to train any model.