Electron

There are multiple approaches to integrate Krisp SDK into Electron apps. Below, are our identified and tested integration options.

We have also included a summary of terms that will be used and code base delivery options to Electron Apps.

Definition of common terms used

Big Model: Noise-cancelation AI model of big size that produces the best quality consuming relatively more computational resources than the Small Model
Small Model: Noise Cancelation AI model of smaller size that produces very high quality consuming minimum CPU resources and preserving battery life of the device
Native Add-on: Native shared library compiled for Node.JS that is designed to work with the Electron package.

Krisp codebase delivery options

Web Assembly code
Native code

Web Assembly code will work directly on a Web Browser such as Chromium, FireFox, Edge, and Electron, which is a Chromium-based system.

Native Code will work only on the target CPU architecture and on the operating system it is designed for. There is no way to run Native Code directly on a Web Browser. However, with the Electron package, there are several ways to integrate Native Code into the Electron system.

There are two different ways of integrating Krisp Native SDK into Electron.

Node.JS Native Add-on
WebRTC Hook

Let’s review and compare all of these options:

	Option #1 Web Assembly	Option #2 Using Native Add-on	Option #3 Using Native Add-on	Option #4 Hook in WebRTC
Short Description	Stream capture and NC processing in JS and Web Assembly	Stream capture in JS. NC processing in native code.	Stream capture and NC processing in native code.	Stream capture in JS and NC processing in native code
Pros	• Easy integration • Good performance / quality using Small Model	• The best quality model is supported	• Better performance than option #2 • The best quality model is supported	Full access to Mic & Speaker streams providing best quality and performance
Cons	• Speaker stream is not production ready • Best quality model supported is to be defined	• Likely latency issues leading to sound artifacts • Will require additional integration work from Krisp • Speaker stream is not production ready	• Complex integration - It’s not clear how the output should be redirected to customer API • Complex implementation - device management must be implemented	• Requires a custom built Electron package, which may require some changes in CI • Additional IPC between Electron native add-on and WebRTC hook
Estimated Effort	Small	Medium	Medium	High
Estimated Uncertainty / Complexity	Low	High	High	Medium

Option #1 Web Assembly

One of the easiest ways to integrate Krisp SDK with the Web Browser is using Web Assembly. We will refer to the solution as Krisp JavaScript SDK.

Process Description

Chromium loads JavaScript file
Sound is captured using JavaScript
Sound is processed using Krisp Web Assembly codebase

Limitations

Only the microphone stream is tested, which means that the sound is captured using the microphone only. The speaker stream using Web Browser is not tested by Krisp at this time.
Only Small Model works in the current WASM JS SDK

Option #2 Using Native Add-on

Krisp Native SDK is compiled and integrated into Native Add-on for Node.JS Electron.

Process Description

Chromium JavaScript captures the stream of batches of 127 samples. It happens the same way it does for Option #1 when using Web Assembly
Each sound batch is passed to the Node.JS for modification
Node.JS will use Native Add-on to process each batch
Native Add-on will use Krisp Native Audio SDK for sound processing

Concerns

Check the latency and the cost between Chromium and Node.JS IPC function calls. Basically it is a cost of data delivery from Chromium JavaScript to Node.JS.
The Web Browser API provides very short window for processing each sound batch. In a case of a delay during the processing the sound batch will be discarded and a silence will be delivered to the other peer
Multiple IPC calls for the stream of short batches from Chromium Process to Node.JS process could consume significant resources. Even a single call could be costly enough to cause significant delay for each individual batch which may not allow us to use higher sampling rate and the best quality models.
We use Web Browser to capture the data and even if we use native node to process the sound we still face the same fact that Krisp has not fully tested speaker stream using the Web Browser

Option #3 Using Native Add-on

Krisp Native SDK is compiled and integrated into Native Add-on for Node.JS Electron package. However, the difference from Option #2 is that the sound is captured inside Native Add-On.

Process Description

Chromium will initiate a process from Node.JS
Node.JS will invoke a process from Native Add-on
Native Add-on captures the stream
Krisp SDK integrated into the Native Add-on filters the stream
The Native Add-on in some form returns the stream to the Chromium JavaScript

Concerns

How to integrate with customer API.

Option #4 Hook in WebRTC

Another way of Krisp Native Library integration for sound processing is to install a hook inside WebRTC. WebRTC is used to handle sound streams inside Electron package.

Integration Steps

Download Electron package with all dependent libraries including WebRTC
Install audio stream hook into WebRTC and link the hook to native Krisp SDK
Add control module for the hook so you can control the hook from Chromium JavaScript
Build custom Electron package

Process Description

Invoke methods from Chromium to enable Krisp SDK integrated into WebRTC hook. The control module can be used to enable/disable noise canceling.
The hook will enable noise canceling for microphone and speaker streams through WebRTC

Advantages

The best performance and the best control over the process.

Concerns

The most advanced and the most complicated solutions
Setup a process to integrate the solution for each continuous Electron build to ensure the customers have the latest patches from Electron upstream.