> ## Documentation Index > Fetch the complete documentation index at: https://plivo.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Stream Event Protocol > WebSocket message types for sending and receiving audio during Plivo Audio Streams During a Media Stream, Plivo sends messages to your WebSocket server that provide information about the Stream. For bidirectional Media Streams, your server can also send messages back to Plivo. This document covers each type of message that your WebSocket server can send and receive when using Plivo Audio Streams. *** ## Overview Mermaid Chart ## Table of Contents * [Messages from Plivo](#messages-from-plivo) * [start](#start) * [media](#media) * [dtmf](#dtmf) * [playedStream](#playedstream) * [clearedAudio](#clearedaudio) * [Messages to Plivo](#messages-to-plivo) * [playAudio](#playaudio) * [checkpoint](#checkpoint) * [clearAudio](#clearaudio) * [sendDTMF](#senddtmf) * [Protocol Schema Reference](#protocol-schema-reference) *** ## Messages from Plivo Plivo sends the following message types to your WebSocket server during a Stream: | Message Type | Description | | ----------------------------- | ------------------------------------------------ | | [start](#start) | Stream metadata, sent once when connection opens | | [media](#media) | Audio data chunks from the caller | | [dtmf](#dtmf) | Touch-tone keypress from the caller | | [playedStream](#playedstream) | Confirmation that a checkpoint was reached | | [clearedAudio](#clearedaudio) | Confirmation that the audio queue was cleared | *** ### `start` Plivo sends the `start` message immediately when the WebSocket connection is established. It contains metadata about the Stream and is only sent once at the beginning. | Property | Type | Description | | ------------------------------ | ------------- | ----------------------------------------------------------------------------- | | `event` | string | Always `"start"` | | `sequenceNumber` | number | Message sequence number (starts at 1, increments for each subsequent message) | | `start.callId` | string (UUID) | Unique identifier for the call | | `start.streamId` | string (UUID) | Unique identifier for the stream | | `start.accountId` | string | Your Plivo account ID | | `start.tracks` | string\[] | Audio tracks being streamed (e.g., `["inbound"]`, `["inbound", "outbound"]`) | | `start.mediaFormat.encoding` | string | Audio codec (e.g., `"audio/x-mulaw"`) | | `start.mediaFormat.sampleRate` | number | Sample rate in Hz (e.g., `8000`) | | `extra_headers` | string | Custom headers from the Stream XML `extraHeaders` attribute | **Example:** ```json theme={null} { "event": "start", "sequenceNumber": 1, "start": { "callId": "12345678-1234-1234-1234-123456789abc", "streamId": "87654321-4321-4321-4321-cba987654321", "accountId": "MAXXXXXXXXXXXXXXXXXX", "tracks": ["inbound"], "mediaFormat": { "encoding": "audio/x-mulaw", "sampleRate": 8000 } }, "extra_headers": "userId=12345;sessionId=abc-xyz" } ``` *** ### `media` Plivo sends `media` messages continuously throughout the call. Each message contains a chunk of audio data from the caller. | Property | Type | Description | | ----------------- | ------------- | ----------------------------------------------------- | | `event` | string | Always `"media"` | | `sequenceNumber` | number | Message sequence number (increments for each message) | | `streamId` | string (UUID) | The unique identifier of the Stream | | `media.track` | string | The audio track (`"inbound"` = audio from the caller) | | `media.timestamp` | string | Unix timestamp in milliseconds | | `media.chunk` | number | Chunk sequence number for this track | | `media.payload` | string | Base64-encoded audio data | | `extra_headers` | string | Custom headers from the Stream XML | **Example:** ```json theme={null} { "event": "media", "sequenceNumber": 42, "streamId": "87654321-4321-4321-4321-cba987654321", "media": { "track": "inbound", "timestamp": "1705312200000", "chunk": 41, "payload": "//uQxAAAAAANIAAAAAExBTUUzLjEwMFVV..." }, "extra_headers": "userId=12345;sessionId=abc-xyz" } ``` **Audio chunk details:** * Each chunk contains approximately **20ms** of audio * At 8kHz with μ-law encoding: \~160 bytes per chunk * Decode using: `Buffer.from(payload, 'base64')` *** ### `dtmf` Plivo sends a `dtmf` message when the caller presses a touch-tone key on their phone, typically in response to a prompt. | Property | Type | Description | | ---------------- | ------------- | ------------------------------------------- | | `event` | string | Always `"dtmf"` | | `sequenceNumber` | number | Message sequence number | | `streamId` | string (UUID) | The unique identifier of the Stream | | `dtmf.track` | string | The audio track (always `"inbound"`) | | `dtmf.digit` | string | The key pressed (`0-9`, `*`, `#`, or `A-D`) | | `dtmf.timestamp` | string | Unix timestamp in milliseconds | | `extra_headers` | string | Custom headers from the Stream XML | **Example:** ```json theme={null} { "event": "dtmf", "sequenceNumber": 50, "streamId": "87654321-4321-4321-4321-cba987654321", "dtmf": { "track": "inbound", "digit": "5", "timestamp": "1705312250000" }, "extra_headers": "userId=12345;sessionId=abc-xyz" } ``` *** ### `playedStream` Plivo sends the `playedStream` message when audio playback reaches a checkpoint that you previously set. Use this to track when specific audio segments finish playing. | Property | Type | Description | | ---------------- | ------------- | ----------------------------------- | | `event` | string | Always `"playedStream"` | | `sequenceNumber` | number | Message sequence number | | `streamId` | string (UUID) | The unique identifier of the Stream | | `name` | string | The checkpoint name you specified | **Example:** ```json theme={null} { "event": "playedStream", "sequenceNumber": 75, "streamId": "87654321-4321-4321-4321-cba987654321", "name": "greeting-complete" } ``` *** ### `clearedAudio` Plivo sends the `clearedAudio` message to confirm that the audio playback queue has been emptied after you sent a `clearAudio` command. | Property | Type | Description | | ---------------- | ------------- | ----------------------------------- | | `event` | string | Always `"clearedAudio"` | | `sequenceNumber` | number | Message sequence number | | `streamId` | string (UUID) | The unique identifier of the Stream | **Example:** ```json theme={null} { "event": "clearedAudio", "sequenceNumber": 80, "streamId": "87654321-4321-4321-4321-cba987654321" } ``` *** ## Messages to Plivo If you initiated a bidirectional Stream, your WebSocket server can send messages back to Plivo to play audio and control the Stream. | Message Type | Description | | ------------------------- | -------------------------------------------- | | [playAudio](#playaudio) | Send audio to be played to the caller | | [checkpoint](#checkpoint) | Mark a point in the audio queue for tracking | | [clearAudio](#clearaudio) | Interrupt and clear all queued audio | | [sendDTMF](#senddtmf) | Send DTMF digits into the call | *** ### `playAudio` Send a `playAudio` message to play audio to the caller. Audio messages are buffered and played in the order received. | Property | Type | Description | | ------------------- | ------ | --------------------------------------------------- | | `event` | string | Always `"playAudio"` | | `media.contentType` | string | Audio MIME type (must match stream's `contentType`) | | `media.sampleRate` | number | Sample rate in Hz (must match stream's sample rate) | | `media.payload` | string | Base64-encoded audio data | **Example:** ```json theme={null} { "event": "playAudio", "media": { "contentType": "audio/x-mulaw", "sampleRate": 8000, "payload": "//uQxAAAAAANIAAAAAExBTUUzLjEwMFVV..." } } ``` > ⚠️ **Important:** The `contentType` and `sampleRate` must match the values specified in your Stream XML. Mismatched formats will cause audio playback issues. > ⚠️ **Warning:** The `media.payload` should contain raw audio data only—do not include audio file headers (e.g., WAV headers). Including headers will cause playback distortion. *** ### `checkpoint` Send a `checkpoint` message after sending audio to receive a `playedStream` notification when playback reaches that point. This helps you track which audio has been played. | Property | Type | Description | | ---------- | ------------- | ----------------------------------------- | | `event` | string | Always `"checkpoint"` | | `streamId` | string (UUID) | The unique identifier of the Stream | | `name` | string | A custom name to identify this checkpoint | **Example:** ```json theme={null} { "event": "checkpoint", "streamId": "87654321-4321-4321-4321-cba987654321", "name": "greeting-complete" } ``` **Use cases:** * Know when a specific response finishes playing * Coordinate follow-up actions after playback * Measure end-to-end latency from audio send to playback completion *** ### `clearAudio` Send a `clearAudio` message to immediately stop playback and empty the audio buffer. Use this to implement user interruption—when the caller starts speaking while audio is playing. | Property | Type | Description | | ---------- | ------------- | ----------------------------------- | | `event` | string | Always `"clearAudio"` | | `streamId` | string (UUID) | The unique identifier of the Stream | **Example:** ```json theme={null} { "event": "clearAudio", "streamId": "87654321-4321-4321-4321-cba987654321" } ``` After sending this message, Plivo will respond with a `clearedAudio` confirmation. *** ### `sendDTMF` Send a `sendDTMF` message to transmit DTMF digits into the active call. Use this to navigate external IVR menus, enter PINs, or trigger touch-tone actions during a bidirectional stream. | Property | Type | Description | | -------- | ------ | --------------------------------------- | | `event` | string | Always `"sendDTMF"` | | `dtmf` | string | Digits to send (`0-9`, `*`, `#`, `A-D`) | **Example:** ```json theme={null} { "event": "sendDTMF", "dtmf": "1234#" } ``` **Use cases:** * Navigate external IVR systems programmatically * Enter PINs or account numbers during a call * Build AI-powered agents that interact with touch-tone menus *** ## Protocol Schema Reference ### JSON Schema ```json theme={null} { "$schema": "http://json-schema.org/draft-07/schema#", "definitions": { "StartEvent": { "type": "object", "required": ["event", "sequenceNumber", "start", "extra_headers"], "properties": { "event": { "const": "start" }, "sequenceNumber": { "type": "integer", "minimum": 1 }, "start": { "type": "object", "required": ["callId", "streamId", "accountId", "tracks", "mediaFormat"], "properties": { "callId": { "type": "string", "format": "uuid" }, "streamId": { "type": "string", "format": "uuid" }, "accountId": { "type": "string" }, "tracks": { "type": "array", "items": { "type": "string" } }, "mediaFormat": { "type": "object", "required": ["encoding", "sampleRate"], "properties": { "encoding": { "type": "string" }, "sampleRate": { "type": "integer" } } } } }, "extra_headers": { "type": "string" } } }, "MediaEvent": { "type": "object", "required": ["event", "sequenceNumber", "streamId", "media", "extra_headers"], "properties": { "event": { "const": "media" }, "sequenceNumber": { "type": "integer" }, "streamId": { "type": "string", "format": "uuid" }, "media": { "type": "object", "required": ["track", "timestamp", "chunk", "payload"], "properties": { "track": { "type": "string" }, "timestamp": { "type": "string" }, "chunk": { "type": "integer" }, "payload": { "type": "string", "contentEncoding": "base64" } } }, "extra_headers": { "type": "string" } } }, "DTMFEvent": { "type": "object", "required": ["event", "sequenceNumber", "streamId", "dtmf", "extra_headers"], "properties": { "event": { "const": "dtmf" }, "sequenceNumber": { "type": "integer" }, "streamId": { "type": "string", "format": "uuid" }, "dtmf": { "type": "object", "required": ["track", "digit", "timestamp"], "properties": { "track": { "type": "string" }, "digit": { "type": "string", "pattern": "^[0-9*#A-D]$" }, "timestamp": { "type": "string" } } }, "extra_headers": { "type": "string" } } }, "PlayedStreamEvent": { "type": "object", "required": ["event", "sequenceNumber", "streamId", "name"], "properties": { "event": { "const": "playedStream" }, "sequenceNumber": { "type": "integer" }, "streamId": { "type": "string", "format": "uuid" }, "name": { "type": "string" } } }, "ClearedAudioEvent": { "type": "object", "required": ["event", "sequenceNumber", "streamId"], "properties": { "event": { "const": "clearedAudio" }, "sequenceNumber": { "type": "integer" }, "streamId": { "type": "string", "format": "uuid" } } }, "PlayAudioEvent": { "type": "object", "required": ["event", "media"], "properties": { "event": { "const": "playAudio" }, "media": { "type": "object", "required": ["contentType", "sampleRate", "payload"], "properties": { "contentType": { "type": "string" }, "sampleRate": { "type": "integer" }, "payload": { "type": "string", "contentEncoding": "base64" } } } } }, "CheckpointEvent": { "type": "object", "required": ["event", "streamId", "name"], "properties": { "event": { "const": "checkpoint" }, "streamId": { "type": "string", "format": "uuid" }, "name": { "type": "string" } } }, "ClearAudioEvent": { "type": "object", "required": ["event", "streamId"], "properties": { "event": { "const": "clearAudio" }, "streamId": { "type": "string", "format": "uuid" } } }, "SendDTMFEvent": { "type": "object", "required": ["event", "dtmf"], "properties": { "event": { "const": "sendDTMF" }, "dtmf": { "type": "string", "description": "Digits to send (0-9, *, #, A-D)" } } } } } ```