Pipecat and Deepgram | Deepgram's Docs

This guide walks you through building a voice AI agent that uses Pipecat for pipeline orchestration and Deepgram for speech-to-text (STT) and text-to-speech (TTS). By the end, you have a working voice agent that listens to a user, generates a response with an LLM, and speaks back in real-time.

Pipecat is an open-source Python framework for building voice and multimodal AI agents. It connects STT, LLM, and TTS services into a real-time pipeline and handles audio transport, turn-taking, and interruption detection.

Before you begin

Before you can use Deepgram, you need to create a Deepgram account. Signup is free and includes $200 in credit.

Daily is the WebRTC transport layer that handles audio between the browser and your agent. See the Pipecat Daily transport guide for more.

You need:

A Deepgram API key
A Daily API key
An LLM API key — this guide uses OpenAI, but Pipecat supports other providers including Anthropic, Google, and Groq
uv installed (for dependency management)
The Pipecat CLI installed
Python 3.11+
Node.js 18+ (only if you add a JavaScript or React Pipecat client later)

Install or update the Pipecat CLI:

$ uv tool install "pipecat-ai[cli]"

To update the CLI use:

$ uv tool upgrade "pipecat-ai[cli]"

Choose your developer experience

Creating a Pipecat + Deepgram integration can be accomplished by using several approaches. Choose the developer experience from the guides below that best fits your style. Note that all paths share the same prerequisite: the Pipecat CLI.

Build with a Coding Agent

You can use AI coding tools like Claude Code or Codex to generate your Pipecat agent code. Rather than relying on the tool’s training data, you give it live context from the Pipecat documentation.

Follow the Pipecat getting started guide to set up AI tools, connect the Pipecat Context Hub, and initialize a project.
Start a coding session with a prompt like the example below.

I'm building a phone assistant for my flower shop, Field & Flower, that
takes customer orders.
The bot should be able to:
  - list the available bouquets
  - check if a specific flower is in stock
  - add a flower to the order
  - get a summary of the order
  - set the delivery details
  - place the order
  - end the call
When the call starts, the bot greets the caller with exactly:
"This is Field & Flower, your local flower shop. How can I help you today?"
Services:
  - Twilio for phone calls
  - STT: Deepgram Flux
  - LLM: OpenAI
  - TTS: Deepgram Flux
  - Deploy to Pipecat Cloud
This is a demo: use a mock backend for the flower data, and "place the
order" only needs to log the order.

The init command creates a GETTING_STARTED.md file with additional guidance for your coding agent.

Use the quickstart CLI command

The quickstart uses Deepgram for STT but Cartesia for TTS. Follow the instruction from the Pipecat Quickstart documentation, then switch to Deepgram using the steps below.

To switch TTS to Deepgram, open bot.py and find the Cartesia TTS setup:

1 # Remove this:
2 from pipecat.services.cartesia.tts import CartesiaTTSService
3 
4 tts = CartesiaTTSService(
5     api_key=os.getenv("CARTESIA_API_KEY"),
6     settings=CartesiaTTSService.Settings(
7         voice="71a7ad14-091c-4e8e-a314-022ece01c121",
8     ),
9 )

Replace it with:

1 from pipecat.services.deepgram.flux.tts import DeepgramFluxTTSService
2 
3 tts = DeepgramFluxTTSService(
4     api_key=os.getenv("DEEPGRAM_API_KEY"),
5     settings=DeepgramFluxTTSService.Settings(
6         voice=os.getenv("DEEPGRAM_VOICE_ID", "flux-alexis-en")
7     ),
8 )

Remove CARTESIA_API_KEY from your .env file — it is no longer needed. No other changes are required: the STT service already uses Deepgram, and the rest of the pipeline stays the same.

Flux TTS voices use the model string format flux-{voice}-{language}, such as flux-alexis-en. This differs from the aura-2-{voice}-{language} format used by Deepgram’s Aura-2 voices. Browse the Flux TTS voice catalog to choose a different voice.

Flux TTS currently synthesizes English only. If your agent needs another language, use DeepgramTTSService with an Aura-2 voice instead — Aura-2 covers English, Spanish, German, French, Dutch, Italian, and Japanese.

Continue building by adding a Pipecat Client

Use Flux for turn detection

Flux is Deepgram’s conversational STT model with built-in turn detection. It uses acoustic and semantic cues to determine when a speaker has finished their turn, resulting in more natural conversations.

To use Flux, replace DeepgramSTTService with DeepgramFluxSTTService in your bot.py:

1 import os
2 
3 from pipecat.services.deepgram.flux.stt import DeepgramFluxSTTService
4 
5 stt = DeepgramFluxSTTService(
6     api_key=os.getenv("DEEPGRAM_API_KEY"),
7     settings=DeepgramFluxSTTService.Settings(
8         min_confidence=0.3,
9     ),
10 )

Scaffold a new Pipecat project with the CLI

Step 1: Create the project

Scaffold a new project using the Pipecat CLI.

$ pipecat init pipecat-deepgram --bot-type web --transport daily --mode cascade --stt deepgram_flux_stt --llm openai_llm --tts deepgram_flux_tts --no-deploy-to-cloud
$ 
$ Wrote pipecat-deepgram/AGENTS.md
$ Wrote pipecat-deepgram/CLAUDE.md
$ 
$ Project created successfully!
$    pipecat-deepgram

Step 2: Install dependencies

Navigate to the server directory inside your new project, create a virtual environment, and install the dependencies:

$ cd pipecat-deepgram/server
$ uv sync

Step 3: Configure your environment

Copy the example environment file and fill in your API keys:

$ cp .env.example .env

.env.example does not include a voice setting, so add one — the scaffolded bot reads DEEPGRAM_VOICE_ID and will not synthesize speech without it:

$ DEEPGRAM_VOICE_ID=flux-alexis-en

Replace the placeholder values with your API keys:

DEEPGRAM_API_KEY — from your Deepgram Console
DEEPGRAM_VOICE_ID — the voice your agent speaks with. Choose a Flux TTS voice such as flux-alexis-en, or an Aura-2 voice if you switched to DeepgramTTSService.
OPENAI_API_KEY — from your OpenAI dashboard
DAILY_API_KEY — from your Daily dashboard. Daily is the WebRTC transport layer that handles audio between the browser and your agent. See the Pipecat Daily transport guide for more.

The remaining values are defaults you can change later.

Step 4: Run the agent

Start the bot from the server directory:

$ uv run python bot.py --transport daily

Step 5: Test the conversation

Open the local URL printed in your terminal, then:

Select Daily from the Transport list and click Connect.
Allow microphone access and speak to your agent.
Ask a question and confirm the agent responds with speech.
Speak while the agent is talking — it should stop and listen.
Pause after speaking — the agent should detect the end of your turn and respond.

Continue building by adding a Pipecat Client

Next Steps

Continue building with an agent.

Follow the Pipecat getting started guide and ready the Pipecat Context Hub.

Prompt your agent to add a Pipecat client framework.

Example prompt:

Add a pipecat client for React

Go further with Deepgram

Voices — Update DEEPGRAM_VOICE_ID in your .env file. Browse the Flux TTS voice catalog for flux-{voice}-{language} voices. If you switched to DeepgramTTSService for a non-English language, pick an Aura-2 voice instead (aura-2-{voice}-{language}).
Keyterm prompting — Improve recognition of domain-specific vocabulary by passing keyterms to Nova-3 via the STT service settings.
Speaker diarization — Assign a speaker identifier to each word in the transcript using diarization via the STT service settings.
Dynamic STT settings — Pipecat supports updating Deepgram STT settings without reconnecting. See the Pipecat Deepgram STT guide for details.