Audio Streaming Beta
Plivo’s audio streaming feature lets businesses stream raw audio from active calls in real time to their applications or third-party systems.
You can stream audio by using XML instructions or APIs.
XML
With the
<Response>
<Stream>wss://yourstream.ngrok.io/audiostream</Stream>
</Response>See our XML reference documentation for complete details.
API
Alternatively, you can use Plivo APIs to initiate and manage audio streams.
1
2
3
4
curl -i --user AUTH_ID:AUTH_TOKEN \
-H "Content-Type: application/json" \
-d '{"service_url": "wss://yourstream.ngrok.io/audiostream"}' \
https://api.plivo.com/v1/Account/{auth_id}/Call/{call_uuid}/Stream/
See our audio streaming API reference documentation for complete details.
Retrieving an audio stream
You can retrieve a specific stream using the stream’s UUID, as shown in this cURL command.
curl -i --user AUTH_ID:AUTH_TOKEN \
https://api.plivo.com/v1/Account/{auth_id}/Call/{call_uuid}/Stream/{Stream_uuid}/
Sample response:
{
"api_id": "2053be7b-10e6-11ee-9cd1-0242ac110009",
"audio_track": "both",
"bidirectional": false,
"bill_duration": 27,
"billed_amount": "0.00300",
"call_uuid": "78737f83-4660-490d-98e1-025dfe4b5c8f",
"created_at": "2023-06-21 13:23:44.136962+00:00",
"end_time": "2023-06-21 18:53:43+05:30",
"plivo_auth_id": "MAY2RJNZKZNJMWOTXXX",
"resource_uri": "/v1/Account/MAY2RJNZKZNJMWOTXXX/Call/78737f83-4660-490d-98e1-025dfe4b5c8f/Stream/20170ada-f610-433b-8758-c02a2aab3662/",
"rounded_bill_duration": 60,
"service_url": "wss://mysocket.com/wss/v2/1/demo/",
"start_time": "2023-06-21 18:53:16+05:30",
"stream_id": "20170ada-f610-433b-8758-c02a2aab3662"
}
You can also retrieve all of a call’s streams. For syntax and details, see the audio streaming API reference.
Audio stream transmission files
Audio streams are transmitted over WebSockets as JSON files.
At the start of an audio stream, Plivo first sends an event to mark the successful connection.
Event on starting the audio stream
{
"sequenceNumber": 0,
"event": "start",
"start": {
"callId": "8c43a765-94fa-4ee9-b9a3-242703e41f63",
"streamId": "b77e037d-4119-44b5-902d-25826b654539",
"accountId": "155747",
"tracks": [
"inbound",
"outbound"
],
"mediaFormat": {
"encoding": "audio/x-l16",
"sampleRate": 8000
}
},
"extra_headers": "{}"
}
Subsequent events include the actual audio being sent to the WebSocket in Base64-encoded format.
Event on receiving an inbound media event
{
"sequenceNumber": 887,
"streamId": "20170ada-f610-433b-8758-c02a2aab3662",
"event": "media",
"media": {
"track": "inbound",
"timestamp": "1687353805345",
"chunk": 469,
"payload": "CAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAAIAAgACAA="
},
"extra_headers": "{}"
}
A similar event is sent for outbound audio streams; for them, the track value is “outbound.”
Plivo also lets you send audio back from the client via the WebSocket.
Expected format to send media event to Plivo
{
"event": "playAudio",
"media": {
"contentType": "audio/x-l16",
"sampleRate": 8000,
"payload": "base64 encoded raw audio.."
}
}
WebSocket connection failures
In the event of an unsuccessful connection, either on the initial connection attempt or if an established connection is dropped, Plivo will retry the specified WebSocket connection twice before disconnecting.
A sample audio streaming application
Here’s some sample Python code, using Flask, that illustrates managing audio streams.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
from flask import Flask
import plivo
app = Flask(__name__)
auth_id = '<auth_id>'
auth_token = '<auth_token>'
call_uuid = '506f0d9f-5961-4c3f-b595-732b36c24e29' #call uuid for which the stream needs to be initiated
service_url = 'wss://8309-49-36-97-128.ngrok.io' #WebSocket URL to which Plivo sends raw audio
bidirectional = False #specifies whether the audio streamed is one way or bidirectional
audio_track = 'both' #specifies which track needs to be forked for the audio stream
stream_timeout = 86400 #maximum duration in seconds for which the audio needs to be streamed
status_callback_url = 'https://<yourdomain>.com/events/' #URL where Plivo will send audio streaming parameters
status_callback_method = 'POST' #method used to invoke status callback URL
content_type = 'audio/x-l16;rate=16000' #preferred audio codec and sampling rate
extra_headers = {"Test": "test1"} #additional headers sent to WebSocket
client = plivo.RestClient(auth_id, auth_token)
@app.route('/start_stream')
def start_audio_stream():
try:
response = client.calls.start_stream(
call_uuid=call_uuid,
service_url=service_url,
bidirectional=bidirectional,
audio_track=audio_track,
stream_timeout=stream_timeout,
status_callback_url=status_callback_url,
status_callback_method=status_callback_method,
content_type=content_type,
extra_headers=extra_headers
)
return str(response)
except plivo.exceptions.PlivoRestError as e:
return str(e)
@app.route('/stop_stream/<stream_uuid>')
def stop_audio_stream(stream_uuid):
try:
response = client.calls.delete_specific_stream(call_uuid, stream_uuid)
return str(response)
except plivo.exceptions.PlivoRestError as e:
return str(e)
@app.route('/retrieve_stream/<stream_uuid>')
def retrieve_audio_stream(stream_uuid):
try:
response = client.calls.get_details_of_specific_stream(call_uuid, stream_uuid)
return str(response)
except plivo.exceptions.PlivoRestError as e:
return str(e)
if __name__ == '__main__':
app.run()
The Flask application defines three routes:
- /start_stream initiates the audio stream by calling the start_audio_stream function.
- /stop_stream/<stream_uuid> stops the audio stream by calling the stop_audio_stream function. The stream_uuid is passed as a - parameter in the URL.
- /retrieve_stream/<stream_uuid> retrieves the audio stream by calling the retrieve_audio_stream function. The stream_uuid is passed as a parameter in the URL.
Buffering
In case of slow or unstable WebSocket connections caused by network problems, Plivo buffers audio packets for up to 40 seconds. Any time the buffer reaches a capacity of 30, 60, or 90%, Plivo sends a connection_degraded event to the designated status callback URL to notify your application about a potentially unstable WebSocket connection.
Pricing
Audio streaming is priced at $0.003 per minute per stream, over and above the expected charges for voice minutes associated with a call.