Browser voice

Directory examples/python/realtime_browser

Connect a browser voice session to a small three-skill node. In live mode, the browser uses a short-lived OpenAI client secret for WebRTC. The long-lived provider key stays on the worker. Transcript turns are submitted through ATFGraph so they can be recorded and stored in session memory; audio travels directly between browser and provider.

Agent node label realtime-browser, version 0.1.0.

Capabilities

Capability	Kind	Signature	Module	Summary
`session_token`	skill	`(instructions: str = '', voice: Optional[str] = None) -> dict`	`examples/python/realtime_browser/agent.py`	Mint a short-lived Realtime client secret for the browser; the API key stays on this node.
`transcript`	skill	`() -> dict`	`examples/python/realtime_browser/agent.py`	The turns recorded in this session (empty without a session).
`turn`	skill	`(role: str, text: str, item_id: Optional[str] = None, realtime_session: Optional[str] = None) -> dict`	`examples/python/realtime_browser/agent.py`	Record one transcript turn from the browser so it appears in the run path and session memory.

Commands

Taken from this example's own README and module docstrings.

ATFGRAPH_UI_DIR=examples/python/realtime_browser/static target/debug/atfgraphd --listen 127.0.0.1:18880 \
REALTIME_BROWSER_MODE=live OPENAI_API_KEY=sk-... python -m atfgraph.worker --module examples.python.realtime_browser.agent:app --server http://127.0.0.1:18880
python -m atfgraph.worker --module examples.python.realtime_browser.agent:app --server http://127.0.0.1:18880

Files

examples/python/realtime_browser/README.md
examples/python/realtime_browser/__init__.py
examples/python/realtime_browser/agent.py
examples/python/realtime_browser/agentnode.yaml
examples/python/realtime_browser/static/index.html

README

Rendered from examples/python/realtime_browser/README.md; the source file is not modified.

browser  --POST /api/v2/traversers {target: realtime-browser.session_token}-->  graph server  --claim-->  node (OPENAI_API_KEY)
browser  <--poll GET /api/v2/traversers/<id> ... output.client_secret (ephemeral)--  graph server  <--complete--  node
browser  --SDP offer + Bearer <client_secret>-->  OpenAI /v1/realtime/calls  (audio + data channel, direct)
browser  --POST /api/v2/traversers {target: realtime-browser.turn}-->  graph server  --claim-->  node (session memory)

Capability	Kind	What it does
`session_token(instructions="", voice=None)`	skill	Mints an ephemeral Realtime client secret with the node's `OPENAI_API_KEY` (`POST https://api.openai.com/v1/realtime/sessions`). Returns `client_secret`, `expires_at`, `model`, `voice`, `calls_url`, `mode`. The API key is never returned, logged or put in a progress event.
`turn(role, text, item_id=None, realtime_session=None)`	skill	Records one transcript turn (`user` or `assistant`) as a progress event and, when the call carries an ATFGraph session, appends it to session memory under `realtime-browser.transcript` (deduplicated on `item_id`, last 200 turns).
`transcript()`	skill	The turns recorded in the current session.

Fixture vs live

`REALTIME_BROWSER_MODE`	`session_token`	Network	Page behaviour
`fixture` (default)	returns `ek_fixture_<hex>` with `calls_url: null` and a `note` saying no provider was contacted	none	no WebRTC; a "typed turn" box exercises `realtime-browser.turn` so the round-trip through the graph server is visible without any credential
`live`	`POST REALTIME_BROWSER_SESSIONS_URL` with `Authorization: Bearer $OPENAI_API_KEY`, body `{model, voice, modalities, instructions}`; returns the provider's `client_secret.value` and `expires_at`	one HTTPS call per session	opens WebRTC to `calls_url` with the ephemeral secret, streams microphone audio, forwards `input_audio_transcription.completed` and `response.*audio_transcript.done` events as turns

Live failures are explicit: a missing key returns not_configured, provider rejections return http_<status>, and unreachable endpoints return retryable provider_unreachable. The provider request includes model/session settings, including instructions; no live credential is fabricated.

Live OpenAI behavior is not verified by this repository's tests. Endpoint/model overrides are REALTIME_BROWSER_SESSIONS_URL, REALTIME_BROWSER_CALLS_URL and REALTIME_BROWSER_MODEL. The browser-example suite uses a real daemon in fixture mode to check token shape, transcript storage and page serving. This establishes local wiring, not microphone or live-provider compatibility.

Running it

Serve the page and API from the same origin. One local option is to point the daemon's UI directory at the static files:

ATFGRAPH_UI_DIR=examples/python/realtime_browser/static target/debug/atfgraphd --listen 127.0.0.1:18880 \
    --database-url "sqlite:///tmp/realtime-browser.db?mode=rwc"

A reverse proxy can instead serve static/ and forward /api/ to the daemon. The page uses its own origin as the default server URL.

Start the node in fixture mode, or configure live mode and its worker-held provider key:

python -m atfgraph.worker --module examples.python.realtime_browser.agent:app --server http://127.0.0.1:18880
# live:
REALTIME_BROWSER_MODE=live OPENAI_API_KEY=sk-... python -m atfgraph.worker --module examples.python.realtime_browser.agent:app --server http://127.0.0.1:18880

Alternatively, install the node with python3 examples/run.py python/realtime_browser.

Open http://127.0.0.1:18880/ui/ and select Start session. Inspect the mode and token/turn traverser IDs in the log, then read a recorded path through /api/v2/traversers/<id>/path.

If the graph server requires a key, enter it in the page's token field. This example sends it using the retained Authorization: Bearer compatibility form to the graph server, not the voice provider. New integrations should use X-API-Key for ATFGraph and reserve Authorization for any edge proxy credential.

What the page does (plain JS, no bundler)

The plain JavaScript in static/index.html performs these steps:

Create an ATFGraph session so transcript turns share memory.
Submit realtime-browser.session_token and poll its traverser to completion or failure.
In live mode, create a WebRTC peer with microphone input, remote audio and an oai-events data channel. Send the SDP offer to calls_url using the ephemeral client secret, then apply its answer.
Submit transcript events as realtime-browser.turn calls. Audio bypasses the graph server.

Honest limits

Provider endpoint shapes and event names still require live validation.
Mint a client secret for each session start and respect its provider-defined expiry; the example assumes a short initial connection window.
Serve the page from the same origin as the API, or add an appropriate proxy. The daemon does not supply CORS headers for this flow.