WebRTC: iOS

Suggest Edits

Introducing Krisp Audio Hook

Google WebRTC does not provide a consistent way to modify the sound stream in the application across different platforms, which is a prerequisite for getting Krisp SDK to function. We have therefore modified WebRTC to include Audio Hook, which represents a consistent approach to modifying the audio in the client application on all platforms.

The Audio Hook will allow modification of the sound stream of the microphone in the client app before the sound is sent to the other peer over the network.

Stream Flow Diagram

The Version and Relation to Chrome

Google WebRTC project is related to Google Chrome. It is already a few years since these projects used the same branch names to match each other on each release. As of Feb 2024, the stable version of Chrome is 121. We have chosen version 121 to apply modifications on top of it. You can use the following page to find the relation between the Chrome version and the branch version https://chromiumdash.appspot.com/branches

How To Build WebRTC with Audio Hook

Build Process

The build process consists of the following

  1. clone WebRTC repository from the official Google repository using Google tools
  2. checkout branch-heads/6167 branch which corresponds to Chrome version 121
  3. pull the changes allowing sound stream modification from the given location
  4. build WebRTC

After going over the process you will have the cloned WebRTC from the official Google sources for Chrome version 121 with added changes visible in the local repository.

git diff HEAD^ HEAD

will show changes similar to the ones below

This will be a useful guide for those who want to integrate the changes into another WebRTC version or fork.
The result of the build will be the Xcode library located in 

src/out_ios_libs/WebRTC.xcframework

Build Tools

Google uses the depot_tools toolset to work with WebRTC, Chromium, and other Google project repositories. It implements a wrapper over Git or SVN version control systems that gives you a super repo tool that can combine multiple projects available in different repository systems into one solution. Using depot_tools is required to work with the sources of various Google projects, including Chromium, WebRTC, and others.

iOS

How To Implement Audio Hook on IOS

In order to implement WebRTC Audio Hook you need to write audio frame processing code. We have introduced the RTCAudioProcessorDelegate abstract class for IOS.
The abstract class has frameProcess abstract method that is responsible for audio frame modification.
First of all, we should have a new class that implements RTCAudioProcessorDelegate abstraction that will be responsible for implementing audio modification logic.

Please refer to the AudioProcessor.mm following link to open the file to find the implementation example of the above-mentioned abstraction in the IOS sample app.

Inject The Audio Frame Processor Into The System

The modified WebRTC for IOS introduces a new setup static function in the RTCPeerConnectionFactory class. The function should be used to inject RTCAudioProcessorDelegate implementation into the system.

Please refer to the WebRTCClient.swift link to open the example from the IOS sample app.

Step-by-step build instructions

Step 1: Install and configure Google depot_tools in your work directory

Create working directory.

mkdir workdir
cd workdir

Install Google depot_tools.

git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git

Add the depot_tools installation root folder to the PATH variable. We need to run fetch and gclient programs from the depot_tools directory, that is why we add the directory to the PATH.

export PATH=`pwd`/depot_tools:$PATH

Step 2: Get sources and prepare for the build

Run the following two commands to pull all sources and configure the sources for the build.

fetch --nohooks webrtc_ios
cd src

Checkout specific commit from the 6167 branch

git checkout -b 6167-modified 6713461a2fb331c43d42b781fbe29da3f5d504a6

Step 3: Get the Changes and apply the Git patch

curl -O https://raw.githubusercontent.com/krispai/WebRTC-Integration/main/audio_hook/ios/audio-hook-webrtc-121.patch
git apply audio-hook-webrtc-121.patch

Step 3^: View Changes (optional)

Optionally you can view the pull changes by comparing the HEAD with the previous commit

git diff HEAD^ HEAD

Step 4: Build WebRTC for IOS

Prepare the build using depot_tools

cd ..  
gclient sync

Build the WebRTC for IOS

cd ./tools_webrtc/ios
python3 build_ios_libs.py

The output for Xcode in the form of XCF framework will be created under 

cd ../..
ls -l out_ios_libs/WebRTC.xcframework

Automation

The following BASH script should work flawlessly with patience. fetch and gclient commands take time.

mkdir workdir
cd workdir
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git
export PATH=`pwd`/depot_tools:$PATH
fetch --nohooks webrtc_ios
cd src
git checkout -b 6167-modified 6713461a2fb331c43d42b781fbe29da3f5d504a6
curl -O https://raw.githubusercontent.com/krispai/WebRTC-Integration/main/audio_hook/ios/audio-hook-webrtc-121.patch
git apply audio-hook-webrtc-121.patch
cd ..
gclient sync
cd ./src/tools_webrtc/ios
python3 build_ios_libs.py
cd ../..
ls -l out_ios_libs/WebRTC.xcframework

How To Build Sample Apps with WebRTC Audio Hook

Currently, only the WebRTC-based JS server and WebRTC-based IOS app are available. More apps on more platforms will be available soon. The document will be updated once Audio Hook becomes available on another platform.
Now let’s get started getting the sample apps exposing WebRTC Audio Hook.

Cloning the repo

First of all, let’s create a working directory and copy the whole repository.

mkdir workdir 
cd workdir
git clone https://github.com/krispai/WebRTC-Integration.git

next, let’s navigate to the local repository and let's view the samples

cd WebRTC-Integration
ls -1 samples

at this moment we will find the following folders

WebRTC-Sample-IOS
WebRTC-Sample-Server-JS

one represents IOS app exposing the Audio Hook, another folder represents the JS server

WebRTC JS Server

Dependencies

Please make sure to have available all dependencies to make sure that further process will be successful. First of all, make sure you have node installed. One way to do it is to use Home Brew.

brew install node

Installation

This component facilitates a direct connection between two peers. The is located in the following repository.

https://github.com/krispai/WebRTC-Integration/tree/main/samples/WebRTC-Sample-Server-JS

🚧 We don’t have a released tag yet, so the link to the main branch is given.

Now let’s get the JS server

First of all, let’s make sure that we have the repository content on our machine. We can skip this step if we already did it to install another app from the same repository.

mkdir workdir 
cd workdir
git clone https://github.com/krispai/WebRTC-Integration.git

Then let’s install dependencies and start the Node JS server

cd WebRTC-Integration/samples/WebRTC-Sample-Server-JS
npm install
npm start

Installing IOS App

Dependencies

Please make sure to have available all dependencies to make sure that further process will be successful.

cocoapods

First of all, let’s make sure to have cocoapods available. This is required for websockets library. One of the most popular ways to install it is using Home Brew using the following command.

brew install cocoapods

WebRTC modification from Krisp

Download the webrtc-ios-121-xcframework.zip file, which is the pre-built WebRTC with added Audio Hook from the following location.

https://github.com/krispai/webrtc/releases/download/v0.2-ios-ready-v121/webrtc-ios-121-xcframework.zip

🚧 Note: You may manually build WebRTC with Audio Hook to allow sound stream modification within the IOS app.

Let’s make sure the framework is downloaded into the following location.

$DOWNLOAD_PATH/webrtc-ios-121-xcframework.zip

Getting iOS sample

First of all, let’s make sure that we have the repository content on our machine. We can skip this step if we already did it to install another app from the same repository.

mkdir workdir 
cd workdir
git clone https://github.com/krispai/WebRTC-Integration.git

If we did this, then let’s go to the sample app

cd samples/WebRTC-Sample-IOS

Once we know that cocoapods is available on the system, use it to download dependent libraries for the Xcode project. This process will download websockets for the sample IOS app. For example, the $IOS_APP is the root directory of the sample app Xcode project.

pod install

Let’s unpack the content of the WebRTC mod into the specified folder

unzip $DOWNLOAD_PATH/webrtc-ios-121-xcframework.zip -d CustomFrameworks/webrtc/

Go to the Xcode project location and build it

open .

Double-click on the WebRTC-Sample-IOS.xcworkspace file to open the Xcode project.

Open the following SWIFT file and update the server address to match the WebRTC JS server deployment.

Automation

JS Server

Assuming that the node is installed and available in the PATH

mkdir workdir 
cd workdir
git clone https://github.com/krispai/WebRTC-Integration.git
cd WebRTC-Integration/samples/WebRTC-Sample-Server-JS
npm install
npm start

IOS App

Before building the IOS app you should know the address of the JS server.
Now let’s download, configure and build the app.

mkdir workdir 
cd workdir
git clone https://github.com/krispai/WebRTC-Integration.git
curl -OL https://github.com/krispai/webrtc/releases/download/v0.2-ios-ready-v121/webrtc-ios-121-xcframework.zip
unzip webrtc-ios-121-xcframework.zip
rm -f webrtc-ios-121-xcframework.zip
cd WebRTC-Integration/samples/WebRTC-Sample-IOS/CustomFrameworks/webrtc
ln -s `pwd`/WebRTC.xcframework WebRTC-Integration/samples/WebRTC-Sample-IOS/CustomFrameworks/webrtc/
cd WebRTC-Integration/samples/WebRTC-Sample-IOS
pod install
xcodebuild -scheme WebRTC-Sample-IOS

The last command may fail depending on XCode configuration and issues with the provisioning profile.

Please make sure to resolve it and the project will be buildable from XCode UI

Updated about 2 months ago