Getting started

Installation

To get started, you need to download the packed library from the (SDK Portal) and place it in the assets of your project.

❕Note: This is currently the only option to get the library.

Once you have downloaded the library, start by importing the SDK into your project. You can refer to our tutorials for guidance, or continue with general information.

import KrispSDK from '/dist/krispsdk.mjs';

Make sure the SDK is supported for the current execution context.

if (!KrispSDK.isSupported()) {
  // KrispSDK is not supported, handle your own logic to not block user's flow
}

To create an SDK instance you need to use KrispSDK class. This object provides a set of methods which are described in the coming chapters.

const sdk = new KrispSDK({
  params: {
    debugLogs: false,
    logProcessStats: false,
    useSharedArrayBuffer: false,
    models: {
      // Please find the models in the Krisp SDK Portal (https://sdk.krisp.ai)
      modelBVC: "/path/to/models/model_bvc.kef",
      model8: "/path/to/models/model_8.kef",
      modelNC: "/path/to/models/model_nc_mq.kef",
    },
  }
});

await sdk.init();

KrispSDK constructor arguments

Noise Filter Creation

Noise filter is the essential part of SDK, which is responsible for Audio NC filtration.
In order to create filter node you need to call SDK instance createNoiseFilter method. As a result the method returns AudioWorkletNode which you can use to connect your audio graph.

const filterNode = await sdk.createFilterNode(
  audioContext, 
  function onReady() {
    filterNode.enable();
    // handle you own buisness logic
  },
  function onDispose() {
    // handle you own buisness logic
  }
);

createFilterNode resolved once AudioWorkletNode is created, but there is much more things going in background (Worker thread initialization, WASM module loading, NC Model Loading). Once SDK is ready to proceed with noice cancellation onReady callback is triggered.

It can take some time until the worker thread is getting ready to proceed audio. You need to wait until onReady callback is triggered then enable audio filtration. filternode.enable()

Once you have filterNode created you can connect with your audio graph.

const audioSettings = {
  audio: {
    echoCancellation: false,
    noiseSuppression: false,
    autoGainControl: false,
  },
};

const stream = await navigator.mediaDevices.getUserMedia(audioSettings);

const source = audioContext.createMediaStreamSource(stream);
const destination = audioContext.createMediaStreamDestination();

const filterNode = await sdk.createFilterNode(
  audioContext, 
  function onReady() {
    filterNode.enable();
    // handle you own buisness logic
  },
  function onDispose() {
    // handle you own buisness logic
  }
);

source.connect(filterNode).connect(destination);

destination.stream; // destination stream is the resulting stream which can be used in your buisness logic

Model Loading

• The SDK currently accepts the following sample rates: 8000, 12000, 16000, 24000, 32000, 44100, 48000, 88200, 96000. Ensure that the AudioContext sample rate is in the list of supported rates to avoid issues such as noise cancellation malfunction or sound distortion. If the sample rate is not in the list of supported rates, you can provide a message to the user, explaining that the sample rate is not supported and recommend they change their settings. Although you can choose the preferable sample rate for you automatically depending on your needs.
• Users can switch their audio devices in the middle of using the SDK, so it's essential to handle new noise cancellation filter creation when the input source has changed. The new noise cancellation filter creation is necessary to ensure that the SDK continues to function correctly and delivers the expected results. The filter creation should be automatic and transparent to the user to avoid disrupting their experience.
• Include model files in the robots.txt file.

Inbound Noise Cancellation

Krisp SDK package includes an Inbound (Speaker) and Ringtone Detection models. To use them, firstly move the models to your project and set the URL in the initialization parameters. If the Ringtone model (modelRT) is provided in inboundModels list and "isInbound" flag is set to "true" the ringtones will be kept. If the Ringtone model wasn't provided, ringtones will be removed from the inbound stream.

const sdk = new KrispSDK({
  params: {
    // ...
    inboundModels: {
      // Please find the models in the Krisp SDK Portal (https://sdk.krisp.ai)
      model_inbound_8: "/path/to/models/model_inbound_8.kef",
      model_inbound_16: "/path/to/models/model_inbound_16.kef",
      modelRT: "/dist/models/model_rt.kef",
    }
  }
});

After this, you need to create a filter node similar to the outbound stream.

const filterNode = await krispSDK.createNoiseFilter({
  audioContext,
  isInbound: true
});

Once the FilterNode is ready, you can enable it and connect it to the destination stream.

const peer1Stream = new MediaStream(); // Inbound stream of first peer connection
const peer2Stream = new MediaStream(); // Inbound stream of second peer connection

filterNode.addEventListener('ready', () => {
  filterNode.enable();
  
  const source1 = audioContext.createMediaStreamSource(peer1Stream);
  const source2 = audioContext.createMediaStreamSource(peer2Stream);
  
  source1.connect(filterNode);
  source2.connect(filterNode);
  
	filterNode.connect(audioContext.destination);
});

❕Note: Regardless of the AudioContext sampleRate, the inbound filter node will choose the 8k model. This means that even if you have an AudioContext with a sampleRate of 44100kHz, it will still use the 8k model to filter out inbound noise.

Background Voice Cancellation

Once Krisp JS SDK is setup, there are few steps to follow in order to start using BVC.

Step 1. Configure KrispSDK to use BVC

To use BVC you should adjust KrispSDK params on creation. There are few options to configure:

const sdk = new KrispSDK({
  params: {
    // The flag indicates if you want to use BVC
    useBVC: true, 
    models: {
      // Please find the models in the Krisp SDK Portal (https://sdk.krisp.ai)
      modelBVC: "/dist/models/model_bvc.kef",
      model8: "/dist/models/model_8.kef",
   
    },
    // The URL of the BVC allowed device list (bundled with the SDK)
    bvc: {
      // mandatory argument
      allowedDevices: "/dist/assets/bvc-allowed.txt",
    },
  },
});

Step 2. Pass audio stream while creating noiseFilterNode

After configuring KrispSDK you have to pass an audio stream while creating the noise filter

const audioSettings = {
  audio: {
    echoCancellation: false,
    noiseSuppression: false,
    autoGainControl: false
  }
};

const stream = await navigator.mediaDevices.getUserMedia(audioSettings);

const noiseFilterNode = await krispSdk.createNoiseFilter(
  {
    audioContext,
    stream
  },
  // onReady: triggered when the Noise Filter is ready for audio processing
  () => {},
  // onDispose: triggered when the Noise Filter is disposed
  () => {},
);

You may find more elaborate guidance in our Background Voice Cancellation Integration for JS.

System Overload Handling

When the system experiences high load, the JS SDK may malfunction and result in choppy voice output. To address this issue, we recommend implementing the buffer_overflow scenario.

During initialization, set the bufferOverflowMS property to a value between 50-250 to determine the acceptable delay due to overload. The default and recommended value is 200ms, but you can adjust it according to your preference. The buffer_overflow event will be emitted each time the buffer difference between the previously emitted value reaches the configured value.

If the buffer keeps growing and no action is taken on the application layer, the JS SDK will drop the buffer after 500ms and emit the buffer_overflow event once again.

Below is a code sample that demonstrates how to configure and listen for the buffer_overflow event.

const sdk = new KrispSDK(
  {
    params: {
      // ...
      bufferOverflowMS: 200
    }
  }
);

const filterNode = await krispSDK.createNoiseFilter({ audioContext, stream });

filterNode.addEventListener('buffer_overflow', (event) => {
  // event.data.overflowCount - count of buffer overflow occurrences after filterNode is enabled.
  // event.data.bufferSizeMS - current buffer size in ms
  // event.data.isBufferDopped - indicates if buffer was dropped due to overflow. Max buffer size is 500ms; it will start dropping once it reaches 500ms.
});

Application Scenario

Below is the application scenario example that you can take to deal with overflow event.

let overflowTimer;

filterNode.addEventListener('buffer_overflow', (e) => {
    // cleaning previously set timer.
    clearTimeout(overflowTimer);
    const count = e.data.overflowCount;

    // disabling Noise cancellation each time buffer_overflow occured
    filterNode.disable();

    // 3 attempts to resume krisp with 2x growing wait time interval
    // Initial wait time 20 sec.
    // In case the system is under load buffer_overflow will trigger once again.
    if (count < 4) {
        const waitTime = 10 * (2 ** count) * 1000;
        overflowTimer = setTimeout(() => {
            if (filterNode) filterNode.enable();
        }, waitTime);
    }
});

Error Handling Outside The Main Thread

Krisp JS SDK is a multithreaded solution. Some errors can happen outside of the main thread. The NC model can be fetched from the user-specified URL outside of the main thread. Since the errors outside of the main thread cannot be captured with a simple try-catch block around the createNoiseFilter, a dedicated event handler is used to handle such kinds of errors.

const filterNode = await krispSdk.createNoiseFilter(
  {
    audioContext,
    stream,
    isInbound: useInboundModel.checked
  },
  onReady
);

filterNode.addEventListener('error', (e) => {
    // e.data.errorCode - indicate the error category
    // e.data.errorMessage - provides detailed error message
    const errorData = e.data;
    if (errorData.errorCode === 'MODEL_URL_FETCH_ERROR') {
        console.error('ERROR fetching the model from URL:', errorData.errorMessage);
    } else if (errorData.errorCode === 'MODEL_LOAD_ERROR') {
        console.error('ERROR during model loading:', errorData.errorMessage);
    } else if (errorData.errorCode === 'SAMPLING_RATE_NOT_SUPPORTED') {
        console.error('ERROR unsupported sampling rate:', errorData.errorMessage);
    } else {
        console.error('ERROR in the worker:', errorData.errorMessage);
    }
});

General Recommendations

• Make sure SDK is supported for the client before initializing the SDK.
• Use the microphone's default sample rate to avoid intermediate resampling.
• Recreate the AudioContext when the sample rate is changed.
• Keep the SDK up to date as audio features are getting changed and enhanced on a daily basis. And try to keep up with current technology. We recommend updating your SDK at least once a quarter.
• The JS SDK is currently available only for Firefox and Chrome browsers. Therefore, before connecting the Krisp SDK node to your nodes, make sure it's supported for the browser to not block the user's flow. You can provide a message to notify users that the SDK is not supported on their current browser and recommend they switch to a supported browser to ensure a seamless experience.

Deploying your Application

• Put your models in your CDN to ensure faster loading times and better performance.
• Make sure these models are not visible to search engines or bots by adding a Disallow directive to the robots.txt file.