~ audio streaming carrier-grade telephony · websocket transport

Audio Streaming for voice AI agents

Live call audio in, synthesized speech back, over one WebSocket. You own the AI pipeline; Plivo handles the phone network. Works best with Pipecat.

Start for free Read the docs

edge → your server: ~50ms
countries: 190+
uptime: 99.99%

wss://your-server.com/streambidirectional

~ customers trusted by teams at

~ how it works caller → plivo → your server → ai

One WebSocket between your call and your AI

Audio streaming exposes the raw audio of a live call over a WebSocket, in both directions, so your agent can listen and respond in the same moment.

PSTN / SIP

Caller

A caller dials your Plivo number, or you place an outbound call through the Voice API.

[01]

edge routing

Plivo Voice

Plivo answers, fetches your Stream XML from your Answer URL, and opens a secure WebSocket to your server.

[02]

WebSocket

Your server

You receive 20 ms audio frames, run your logic, and stream synthesized audio back.

[03]

STT · LLM · TTS

AI services

Any provider you choose transcribes, reasons, and speaks, with barge-in support.

[04]

~ quickstart one server · two endpoints

From dial tone to dialogue in two steps

One server, two endpoints. The Stream SDK takes care of the framing, encoding, and events.

01 /answer returns your <Stream> XML when Plivo picks up.
02 /stream receives caller audio and streams your agent's reply back.

View the full quickstart

from fastapi import FastAPI, WebSocket
from fastapi.responses import Response
from plivo_stream import PlivoFastAPIStreamingHandler

app = FastAPI()

ANSWER_XML = """<Response><Stream bidirectional="true" keepCallAlive="true"
  contentType="audio/x-mulaw;rate=8000">wss://your-server.com/stream</Stream></Response>"""

@app.get("/answer")
def answer():                              # Plivo requests this on pickup
    return Response(ANSWER_XML, media_type="application/xml")

@app.websocket("/stream")
async def stream(websocket: WebSocket):    # Plivo opens the socket here
    handler = PlivoFastAPIStreamingHandler(websocket)

    @handler.on_media
    async def on_media(event):
        audio = event.get_raw_media()          # caller audio (mu-law)
        reply = await run_ai_pipeline(audio)   # your STT -> LLM -> TTS
        await handler.send_media(reply)        # speak back to the caller

    await handler.start()

import express from 'express';
import { createServer } from 'http';
import { PlivoWebSocketServer } from '@plivo/plivo-stream-sdk';

const app = express();

const ANSWER_XML = `<Response><Stream bidirectional="true" keepCallAlive="true"
  contentType="audio/x-mulaw;rate=8000">wss://your-server.com/stream</Stream></Response>`;

app.get('/answer', (req, res) =>           // Plivo requests this on pickup
  res.type('application/xml').send(ANSWER_XML));

const server = createServer(app);
const plivo = new PlivoWebSocketServer({ server, path: '/stream' });

plivo.onMedia(async (event, ws) => {
  const audio = event.getRawMedia();           // caller audio (mu-law)
  const reply = await runAiPipeline(audio);    // your STT -> LLM -> TTS
  plivo.playAudio(ws, 'audio/x-mulaw', 8000, reply); // speak back
});

server.listen(5000);

~ framework pipecat · websocket

Ship faster on Pipecat

Plivo streams live call audio straight into your Pipecat pipeline over WebSocket. Start from a working voice agent, not telephony plumbing.

Phone call Plivo WebSocket Pipecat AI services

Native PlivoFrameSerializer in Pipecat core
Bring any STT, LLM, and TTS, or go speech-to-speech
Scaffold in one command with the Pipecat CLI

Pipecat guide Example repo

pipecat cli

# scaffold a phone agent on Plivo telephony
uv tool install "pipecat-ai[cli]"

pipecat init call-bot --bot-type telephony --transport plivo \
    --mode cascade --stt deepgram_stt --llm openai_llm --tts cartesia_tts

cd call-bot && uv sync
uv run bot.py    # expose with: ngrok http 7860

from pipecat.pipeline.pipeline import Pipeline
from pipecat.serializers.plivo import PlivoFrameSerializer
from pipecat.transports.websocket.fastapi import (
    FastAPIWebsocketParams, FastAPIWebsocketTransport)

transport = FastAPIWebsocketTransport(websocket, FastAPIWebsocketParams(
    audio_in_enabled=True, audio_out_enabled=True, add_wav_header=False,
    serializer=PlivoFrameSerializer(stream_id, call_id,
        auth_id=os.environ["PLIVO_AUTH_ID"],
        auth_token=os.environ["PLIVO_AUTH_TOKEN"])))

pipeline = Pipeline([
    transport.input(),    # caller audio in from Plivo
    stt,                  # Deepgram
    user_aggregator,
    llm,                  # OpenAI
    tts,                  # Cartesia
    transport.output(),   # agent audio back to the caller
    assistant_aggregator,
])

worker = PipelineWorker(pipeline, params=PipelineParams(
    audio_in_sample_rate=8000, audio_out_sample_rate=8000))
await runner.add_workers(worker)
await runner.run()

~ ready-made recipes built on plivo github + docs

runnable pipecat + plivo starter gemini live · pipecat openai · deepgram · openai tts grok tts · pipecat all examples →

guides openai + cartesia openai + elevenlabs gemini + cartesia gemini + elevenlabs sarvam openai realtime · s2s gemini live · s2s

on livekit? LiveKit agents connect over SIP Trunking. Same telephony, different transport.

~ capabilities 9 built in · 3 groups

Everything a production voice agent needs from its carrier

The platform handles the hard telephony work: codecs, interruption, security, and the full stream lifecycle. You focus on the conversation.

live conversation how the agent listens and talks

duplex

Bidirectional WebSocket

Receive caller audio and send synthesized speech back over a single connection, so your agent can listen and talk at the same time.

[01]

barge-in

Barge-in and interruption

Clear the playback queue the instant a caller speaks with clearAudio, and track exactly what finished playing with checkpoints.

[02]

dtmf

In-stream DTMF, both ways

Receive keypad presses as dtmf events mid-stream, and send digits out when your agent has to drive an IVR or answering machine.

[03]

voice quality · latency what the caller hears

latency

Edge-routed by design

Plivo answers calls at the edge location closest to the caller, so co-locating your server keeps the network hop short and the conversation snappy.

[04]

clarity

Noise cancellation & BVC

Filter background noise and background voices on the wire, tunable per stream, so your speech-to-text hears only the caller.

[05]

codec

Native telephony codec

Stream μ-law 8 kHz, the phone network's own format, or switch to 16-bit Linear PCM at 8 or 16 kHz when your STT model wants wideband audio.

[06]

operations · security what production demands

callbacks

Lifecycle callbacks

Subscribe to started, stopped, and failed webhooks with duration and status reason to power monitoring, billing, and retries.

[07]

metadata

Session metadata

Pass user, session, or tenant context into the start event with extraHeaders, so every stream lands on your server already identified.

[08]

security

Signed connections

Every WebSocket upgrade carries a V3 HMAC-SHA256 signature you can validate with the SDK, over wss and TLS, so only Plivo traffic reaches your server.

[09]

~ stream directions 2 modes · what teams build

Stream one way to listen, both ways to converse

One XML attribute sets the direction. Each direction unlocks a different class of product.

Both ways to converse

bidirectional

audio in and out on the same call

Voice assistants

Agents that answer, qualify, and resolve on their own.
Voice bots & smart IVR

Replace press-one menus with intent-aware routing.
Outbound conversations

Reminders and follow-ups that handle questions live.

params bidirectional=truekeepCallAliveclearAudio

One way to listen

listen-only

observe the call, send nothing back

Real-time transcription

Live captions and searchable transcripts as words are spoken.
Sentiment & QA

Score tone, intent, and compliance while the call happens.
Agent assist & co-pilot

Surface answers to human agents mid-call.

params audioTrack=both

~ your ai stack stt · llm · tts

Bring your own AI. Plivo moves the audio.

Audio streaming is model-agnostic on purpose. Swap any provider without touching the call layer, so you are never locked into one vendor's voices or models.

speech-to-text [01]

Transcribe

Forward caller audio to any streaming STT model to turn speech into text as it is spoken.

Deepgram
AssemblyAI
Sarvam

reasoning [02]

Think

Send the transcript to any LLM to decide what to say next, call tools, or look up your knowledge base.

OpenAI
Anthropic
Google Gemini

text-to-speech [03]

Speak

Synthesize the reply with any TTS voice and stream it back to the caller for a natural response.

ElevenLabs
Cartesia
Inworld

works with your favorites and any api-based provider

~ latency target < 1000 ms

Engineered for the sub-second budget

Wherever you build, Plivo has a point of presence close to your callers. Calls enter the network at the nearest edge and reach your server in about 50 ms, so the sub-second budget is yours to spend on AI.

end-to-end response < 1000 ms

Speech-to-text

< 200 ms your stack

LLM

< 500 ms your stack

Text-to-speech

< 200 ms your stack

Network

< 50 ms plivo

A point of presence near every caller

Plivo answers at the edge closest to the caller, wherever that is. Run your WebSocket server and model endpoints in the matching region and the whole turn stays inside the sub-second budget.

US traffic US East (Virginia) or US West (Oregon)
Europe Frankfurt or London
Asia-Pacific Singapore or Mumbai
Global Multi-region with geographic routing

~ security & compliance hipaa · gdpr · soc 2 · pci dss

Enterprise-grade security
& compliance

HIPAA

GDPR

SOC 2

PCI DSS

STAR

Encryption everywhere

TLS in transit. AES-256 at rest. No exceptions.

tls · aes-256
Data residency

Choose US, EU, or APAC. Your data stays where you need it.

us · eu · apac
Audit-ready

Full access logs, RBAC, and compliance reports on demand.

logs · rbac · reports