Training sessions
A metahuman conversation is one step of a course. The step pins a published metahuman activity — who the metahuman is, what it is for (a role-play, a tutor, a coach…), what it knows, and the rules of the conversation — and the conversation runs until the learner ends it (once the activity’s minimum exchanges are met), the metahuman ends it (when the activity allows), or an idle or time limit closes it. Whatever comes before or after — a briefing, a decision, a justification, scoring, a debrief — is another course step.
A live conversation is brokered by the session-gateway. The browser holds one WebSocket to MTP and never connects to HOPE’s data plane directly. Microphone audio flows up that socket; LLM text fragments, speech audio, and facial blendshape frames flow back down.
Why a broker, not a direct embed
HOPE documents a direct browser-to-HOPE embed pattern. MTP deliberately does not use it, for four reasons:
- Ground-truth secrecy. A role-play persona’s facts — and which of them it is lying about — shape what the metahuman says. Composed in the browser, that would be one dev-tools inspection away from the trainee. The API composes every projection server-side: HOPE receives stated values only, the browser receives neither, and a fact’s hidden truth reaches only the assessment.
- Authoritative transcripts. HOPE exposes no transcript read API, so
scoring integrity requires a transcript recorder the trainee cannot
tamper with. The gateway records both directions of every exchange with an
individually addressable
exchange_indexthat assessment and coaching reference later. - Rule enforcement. Whether the conversation may end yet, who may end it, and the idle and time limits are decided by the API where they cannot be bypassed. Course progress is server-side and resumable from any workstation.
- Rate-limit stewardship. HOPE budgets agent-stream connections per credential. The gateway paces turns, pools per-tenant credentials, and backs off centrally instead of in every browser.
The cost is trivial: 16 kHz mono PCM upstream is about 32 KB/s, and in-cluster relay adds single-digit milliseconds against a sub-3-second voice-latency budget.
What flows where
| Leg | Content | Credential |
|---|---|---|
| Browser → gateway | Mic PCM, end of conversation | mtp_access session cookie |
Gateway → HOPE /stt | Relayed PCM | Machine token (per-tenant API key) |
Gateway → HOPE /agent-stream | Run envelope, then CLIENT-tool answers: stated facts only | Machine token (per-tenant API key) |
| HOPE → gateway → browser | LLM fragments, PCM audio, blendshape frames | — |
| Gateway → Postgres (via the API) | Both speakers’ turns, exchange-indexed; observations; the end | — |
Avatars
- Standard 3D avatars render in the browser with HOPE’s renderer, fed by gateway-relayed blendshape and PCM frames.
- Premium avatars are the one exception to strict brokering: the gateway creates the live-avatar session with its machine token, but the returned subscribe-only LiveKit token goes to the browser, which attaches media directly. Video never carries ground truth, and relaying WebRTC would be pure loss. On live-avatar runs the gateway suppresses its local audio relay.
Session identity and tenancy
- Each MTP training session mints an unguessable
conv-<uuid>as the session identifier sent to HOPE. - Users are identified to HOPE only by an opaque surrogate (
usr_<uuid>), never by email. The surrogate drives HOPE’s repeat-offender guardrail escalation and appears in webhook payloads. - Each MTP tenant is a separate HOPE organization with its own API keys, so guardrail policies, entitlements, and usage metering are isolated per customer.
Dictation in response steps
A course response step (a written decision or justification) can let the learner speak instead of type. The browser asks the API for a short-lived ticket bound to that step and opens a separate, speech-to-text-only gateway socket with it: no metahuman runs and nothing is recorded — the text fills the answer box.
After the conversation
When the course reaches an assessment step, the api service enqueues an
assessment job over the step’s declared inputs — say, the conversation and the
justification after it. The assessment engine makes one inference pass —
transcripts, answers, rubric, and the activity’s hidden truths in; a unified
assessment payload out — and that stored payload is the single source of truth for the
debrief, dashboards, exports, and LTI grade passback. The coaching debrief
renders entirely from the stored payload with zero LLM calls at render time.
See The content kernel for how rubrics and
coaching templates define this pipeline.
Guardrail events arrive from HOPE as HMAC-signed webhooks. A
guardrail.session_terminated event marks the MTP session as terminated by
guardrail and surfaces it in trainer and program-administrator views.