Electron
There are multiple approaches to integrate Krisp SDK into Electron apps. Below, are our identified and tested integration options.
We have also included a summary of terms that will be used and code base delivery options to Electron Apps.
Definition of common terms used
- Big Model: Noise-cancelation AI model of big size that produces the best quality consuming relatively more computational resources than the
Small Model
- Small Model: Noise Cancelation AI model of smaller size that produces very high quality consuming minimum CPU resources and preserving battery life of the device
- Native Add-on: Native shared library compiled for Node.JS that is designed to work with the Electron package.
Krisp codebase delivery options
- Web Assembly code
- Native code
Web Assembly code will work directly on a Web Browser such as Chromium, FireFox, Edge, and Electron, which is a Chromium-based system.
Native Code will work only on the target CPU architecture and on the operating system it is designed for. There is no way to run Native Code directly on a Web Browser. However, with the Electron package, there are several ways to integrate Native Code into the Electron system.
There are two different ways of integrating Krisp Native SDK into Electron.
- Node.JS Native Add-on
- WebRTC Hook
Let’s review and compare all of these options:
Option #1 Web Assembly | Option #2 Using Native Add-on | Option #3 Using Native Add-on | Option #4 Hook in WebRTC | |
---|---|---|---|---|
Short Description | Stream capture and NC processing in JS and Web Assembly | Stream capture in JS. NC processing in native code. | Stream capture and NC processing in native code. | Stream capture in JS and NC processing in native code |
Pros | • Easy integration • Good performance / quality using Small Model | • The best quality model is supported | • Better performance than option #2 • The best quality model is supported | Full access to Mic & Speaker streams providing best quality and performance |
Cons | • Speaker stream is not production ready • Best quality model supported is to be defined | • Likely latency issues leading to sound artifacts • Will require additional integration work from Krisp • Speaker stream is not production ready | • Complex integration - It’s not clear how the output should be redirected to customer API • Complex implementation - device management must be implemented | • Requires a custom built Electron package, which may require some changes in CI • Additional IPC between Electron native add-on and WebRTC hook |
Estimated Effort | Small | Medium | Medium | High |
Estimated Uncertainty / Complexity | Low | High | High | Medium |
Option #1 Web Assembly
One of the easiest ways to integrate Krisp SDK with the Web Browser is using Web Assembly. We will refer to the solution as Krisp JavaScript SDK.
![](https://files.readme.io/fd2125d-option_1_web_assembly.png)
Process Description
- Chromium loads JavaScript file
- Sound is captured using JavaScript
- Sound is processed using Krisp Web Assembly codebase
Limitations
- Only the microphone stream is tested, which means that the sound is captured using the microphone only. The speaker stream using Web Browser is not tested by Krisp at this time.
- Only
Small Model
works in the current WASM JS SDK
Option #2 Using Native Add-on
Krisp Native SDK is compiled and integrated into Native Add-on for Node.JS Electron.
![](https://files.readme.io/1918f87-option_2_native_add-on.png)
Process Description
- Chromium JavaScript captures the stream of batches of 127 samples. It happens the same way it does for Option #1 when using Web Assembly
- Each sound batch is passed to the Node.JS for modification
- Node.JS will use Native Add-on to process each batch
- Native Add-on will use Krisp Native Audio SDK for sound processing
Concerns
- Check the latency and the cost between Chromium and Node.JS IPC function calls. Basically it is a cost of data delivery from Chromium JavaScript to Node.JS.
- The Web Browser API provides very short window for processing each sound batch. In a case of a delay during the processing the sound batch will be discarded and a silence will be delivered to the other peer
- Multiple IPC calls for the stream of short batches from Chromium Process to Node.JS process could consume significant resources. Even a single call could be costly enough to cause significant delay for each individual batch which may not allow us to use higher sampling rate and the best quality models.
- We use Web Browser to capture the data and even if we use native node to process the sound we still face the same fact that Krisp has not fully tested speaker stream using the Web Browser
Option #3 Using Native Add-on
Krisp Native SDK is compiled and integrated into Native Add-on for Node.JS Electron package. However, the difference from Option #2 is that the sound is captured inside Native Add-On.
![](https://files.readme.io/e48a0e5-option_3_native_add-on.png)
Process Description
- Chromium will initiate a process from Node.JS
- Node.JS will invoke a process from Native Add-on
- Native Add-on captures the stream
- Krisp SDK integrated into the Native Add-on filters the stream
- The Native Add-on in some form returns the stream to the Chromium JavaScript
Concerns
How to integrate with customer API.
Option #4 Hook in WebRTC
Another way of Krisp Native Library integration for sound processing is to install a hook inside WebRTC. WebRTC is used to handle sound streams inside Electron package.
![](https://files.readme.io/6f73263-option_4_webrtc_hook.png)
Integration Steps
- Download Electron package with all dependent libraries including WebRTC
- Install audio stream hook into WebRTC and link the hook to native Krisp SDK
- Add control module for the hook so you can control the hook from Chromium JavaScript
- Build custom Electron package
Process Description
- Invoke methods from Chromium to enable Krisp SDK integrated into WebRTC hook. The control module can be used to enable/disable noise canceling.
- The hook will enable noise canceling for microphone and speaker streams through WebRTC
Advantages
- The best performance and the best control over the process.
Concerns
- The most advanced and the most complicated solutions
- Setup a process to integrate the solution for each continuous Electron build to ensure the customers have the latest patches from Electron upstream.
Updated 9 months ago