CPython Based Audio Processing Solution Sample using Krisp

Overview

CPython is the most popular Python framework. There are multiple frameworks dedicated to building CPython modules using C/C++ code. At Krisp, we have published an open-source reference CPython-based audio processing sample using pybind11 to build a CPython module using C++ based Krisp Audio SDK. The audio processing sample works directly with the audio data, it does not include audio capturing logic.

Instead of mapping every function from C++ based SDK to Python, we find it more practical to provide an audio processing solution directly which requires minimum configuration to embed Krisp code into your business logic.

Usage

To use an audio processing sample you need to

1. import the compiled CPython module
2. set the sampling rate of the audio stream
3. specify Krisp NC model
4. in the loop
   1. provide an audio data buffer
   2. extract processed audio in the form of a list of 10ms sized frames

Check the attached code snippet for the details. In the sample code you need to implement the getAudioData() and the processAudioFrames() functions to fit your business logic.

import audio_processor

sample_rate = 16000
model_path = "/Users/yourname/krisp-models/any_krisp_nc_model.kw"
ap_wrapper = AudioProcessorWrapper(sample_rate, 1, model_path)

def getAudioData():
  pass
  # provide continous audio buffer of any size
  
def processAudioFrames(procesed_10ms_frames):
  # process noise canceled audio frames

while True:
  audio_data = getAudioData()
  ap_wrapper.store_audio_chunk(audio_data)
  processed_10ms_frames = ap_wrapper.get_processed_frames(
  processAudioFrames(processed_10ms_frames)
  

❗️

Only 32bit PCM Float audio data frames are supported.

🚧

PCM 16-bit audio data support will be available soon.

ℹ️

Krisp NC tech can only process 10ms sized audio frames.

When calling theap_wrapper.store_audio_chunk(audio_data), if the audio_data size is not multiple to 10ms, the remainder will be kept in the system and will be concatenated to the beginninng of the next audio data.

Build

To build CPython based audio processing solution you need to

1. download Krisp Audio SDK v7 for Windows, Linux or Mac.
2. clone the https://github.com/krispai/Krisp-SDK-Sample-Apps
3. select krisp-sdk-v7 branch
4. follow instructions in details in the README.md file