Pipecat integration

⚠️
This supercedes the Krisp SDK integration guide published at https://docs.pipecat.ai/guides/features/krisp-viva

NOTE: This assumes you have a working install of Pipecat and is based on the sample bot.py that Pipecat provides. This document assumes you are using the uv package manager as recommended by the Pipecat installation guide.

Voice Isolation in Pipecat

Download the latest Python SDK from the SDK Portal..
Install Python wheel file into your venv of the Pipecat installation.

uv pip install <path to the Krisp wheel file>

Update your .env file to include the following variables:

# Krisp SDK API key (required for SDK v1.6.1+)
KRISP_VIVA_API_KEY=your_api_key_here

# Voice isolation model path
KRISP_VIVA_FILTER_MODEL_PATH=/PATH_TO_UNZIPPED_MODELS/krisp-viva-tel-v2.kef

You will need to add the following import to your bot.py:

#Add the VIVA BVC filter
from pipecat.audio.filters.krisp_viva_filter import KrispVivaFilter

To enable the VIVA BVC filter in your transport you need to add the Krisp VIVA filter to your transport params. For example if you have the following transport params in your bot.py:

async def bot(runner_args: RunnerArguments):
    """Main bot entry point for the bot starter."""
    
    transport_params = {
        "daily": lambda: DailyParams(
            audio_in_enabled=True,
            audio_out_enabled=True,
        ),
        "webrtc": lambda: TransportParams(
            audio_in_enabled=True,
            audio_out_enabled=True,
        ),
    }
    
    transport = await create_transport(runner_args, transport_params)

You need to add the KrispVivaFilter() to your audio_in_filter(s). For example:

async def bot(runner_args: RunnerArguments):
    """Main bot entry point for the bot starter."""
    transport_params = {
        "daily": lambda: DailyParams(
            audio_in_filter=KrispVivaFilter(),
            audio_in_enabled=True,
            audio_out_enabled=True,
        ),
        "webrtc": lambda: TransportParams(
            audio_in_filter=KrispVivaFilter(),
            audio_in_enabled=True,
            audio_out_enabled=True,
        ),
    }
    transport = await create_transport(runner_args, transport_params)

The filter should be added to whatever transport(s) you are using where you want the Krisp VIVA filter to execute.

Turn-Taking in Pipecat

To enable VIVA Turn taking in Pipecat you need to add the following imports to your bot.py

#Add the VIVA TT libraries
from pipecat.audio.turn.krisp_viva_turn import KrispVivaTurn

#Needed for TT
from pipecat.turns.user_stop import TurnAnalyzerUserTurnStopStrategy
from pipecat.turns.user_turn_strategies import UserTurnStrategies

Add the following to your .env file:

# Turn detection model path
KRISP_VIVA_TURN_MODEL_PATH=/PATH_TO_UNZIPPED_MODELS/krisp-viva-tt-v2.kef

To enable Krisp Turn Taking you need to modify the LLMContext. Your LLMContext may look similar to:

context = LLMContext()
    user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
        context,
        user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
    )

Add a user_turn_strategies to your LLMContext user params which will replace the default SmartTurn strategy. Your LLMContext should look similar to:

 context = LLMContext()
    user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
        context,
   user_params=LLMUserAggregatorParams(user_turn_strategies=UserTurnStrategies(
            stop=[TurnAnalyzerUserTurnStopStrategy(
                turn_analyzer=KrispVivaTurn()
            )]),vad_analyzer=SileroVADAnalyzer()),
    )

Note: Take care to properly indent the above code as the indentation is slightly off due to the page width of this documentation