There are multiple approaches to integrate Krisp SDK into Electron apps. Below, are our identified and tested integration options.
We have also included a summary of terms that will be used and code base delivery options to Electron Apps.
- Big Model: Noise-cancelation AI model of big size that produces the best quality consuming relatively more computational resources than the
- Small Model: Noise Cancelation AI model of smaller size that produces very high quality consuming minimum CPU resources and preserving battery life of the device
- Native Add-on: Native shared library compiled for Node.JS that is designed to work with the Electron package.
- Web Assembly code
- Native code
Web Assembly code will work directly on a Web Browser such as Chromium, FireFox, Edge, and Electron, which is a Chromium-based system.
Native Code will work only on the target CPU architecture and on the operating system it is designed for. There is no way to run Native Code directly on a Web Browser. However, with the Electron package, there are several ways to integrate Native Code into the Electron system.
There are two different ways of integrating Krisp Native SDK into Electron.
- Node.JS Native Add-on
- WebRTC Hook
Let’s review and compare all of these options:
|Option #1 Web Assembly
Using Native Add-on
Using Native Add-on
Hook in WebRTC
|Stream capture and NC processing in JS and Web Assembly
|Stream capture in JS. NC processing in native code.
|Stream capture and NC processing in native code.
|Stream capture in JS and NC processing in native code
|• Easy integration
• Good performance / quality using Small Model
|• The best quality model is supported
|• Better performance than option #2
• The best quality model is supported
|Full access to Mic & Speaker streams providing best quality and performance
|• Speaker stream is not production ready
• Best quality model supported is to be defined
|• Likely latency issues leading to sound artifacts
• Will require additional integration work from Krisp
• Speaker stream is not production ready
|• Complex integration - It’s not clear how the output should be redirected to customer API
• Complex implementation - device management must be implemented
|• Requires a custom built Electron package, which may require some changes in CI
• Additional IPC between Electron native add-on and WebRTC hook
Uncertainty / Complexity
- Sound is processed using Krisp Web Assembly codebase
- Only the microphone stream is tested, which means that the sound is captured using the microphone only. The speaker stream using Web Browser is not tested by Krisp at this time.
Small Modelworks in the current WASM JS SDK
Krisp Native SDK is compiled and integrated into Native Add-on for Node.JS Electron.
- Each sound batch is passed to the Node.JS for modification
- Node.JS will use Native Add-on to process each batch
- Native Add-on will use Krisp Native Audio SDK for sound processing
- The Web Browser API provides very short window for processing each sound batch. In a case of a delay during the processing the sound batch will be discarded and a silence will be delivered to the other peer
- Multiple IPC calls for the stream of short batches from Chromium Process to Node.JS process could consume significant resources. Even a single call could be costly enough to cause significant delay for each individual batch which may not allow us to use higher sampling rate and the best quality models.
- We use Web Browser to capture the data and even if we use native node to process the sound we still face the same fact that Krisp has not fully tested speaker stream using the Web Browser
Krisp Native SDK is compiled and integrated into Native Add-on for Node.JS Electron package. However, the difference from Option #2 is that the sound is captured inside Native Add-On.
- Chromium will initiate a process from Node.JS
- Node.JS will invoke a process from Native Add-on
- Native Add-on captures the stream
- Krisp SDK integrated into the Native Add-on filters the stream
How to integrate with customer API.
Another way of Krisp Native Library integration for sound processing is to install a hook inside WebRTC. WebRTC is used to handle sound streams inside Electron package.
- Download Electron package with all dependent libraries including WebRTC
- Install audio stream hook into WebRTC and link the hook to native Krisp SDK
- Build custom Electron package
- Invoke methods from Chromium to enable Krisp SDK integrated into WebRTC hook. The control module can be used to enable/disable noise canceling.
- The hook will enable noise canceling for microphone and speaker streams through WebRTC
- The best performance and the best control over the process.
- The most advanced and the most complicated solutions
- Setup a process to integrate the solution for each continuous Electron build to ensure the customers have the latest patches from Electron upstream.
Updated 3 months ago