## Live encoding with VP9 using FFmpeg ### Encoding parameters VP9 provides a range of parameters to optimize live encoding. Some broad principles of these are discussed in [Bitrate Modes](https://developers.google.com/media/bitrate-modes/). #### FFmpeg VP9 encoding example The table below describes the parameters of an example `ffmpeg` call for VP9 encoding. | Parameter | Description | |---|---| | `-quality realtime` | `realtime` is essential for live streaming and for speeds above `5`. | | `-speed 6` | Speed `5` to `8` should be used for live / real-time encoding. Lower numbers (`5` or `6`) are higher quality but require more CPU power. Higher numbers (`7` or `8`) will be lower quality but more manageable for lower latency use cases and also for lower CPU power devices such as mobile. | | `-tile-columns 4` | Tiling splits the video into rectangular regions, which allows multi-threading for encoding and decoding. The number of tiles is always a power of two. `0` = 1 tile, `1` = 2, `2` = 4, `3` = 8, `4` = 16, `5` = 32. | | `-frame-parallel 1` | Enable parallel decodability features. | | `-threads 8` | Maximum number of threads to use. | | `-static-thresh 0` | Motion detection threshold. | | `-max-intra-rate 300` | Maximum i-Frame bitrate (pct) | | `-deadline realtime` | Alternative (legacy) version of `-quality realtime` | | `-lag-in-frames 0` | Maximum number of frames to lag | | `-qmin 4 -qmax 48` | Minimum and maximum values for the quantizer. The values here are merely a suggestion and adjusting this will help increase/decrease video quality at the expense of compression efficiency. | | `-row-mt 1` | Enable row-multithreading. Allows use of up to 2x thread as tile columns. `0` = off, `1` = on. | | `-error-resilient 1` | Enable error resiliency features. | ### Choosing encoding parameters The information below uses constant bitrate (CBR) encoding for live adaptive bitrate streaming (ABR), where each target rate is explicitly set in the packager's manifest. This will result in cleaner "switching" between rates for clients. Variable bitrate (VBR) encoding and [CQ mode](https://developers.google.com/media/bitrate-modes/#cq) are also options if the bitrate can be more flexible or the encoding is being chunked. [Q mode](https://developers.google.com/media/bitrate-modes/#q) will struggle with the realtime encoding required for live video. See [Bitrate Modes](https://developers.google.com/media/bitrate-modes/) for more information. For further details on how to manipulate VP9 it is also worth referring to the accompanying article on [VOD settings](https://developers.google.com/media/settings/vod/), but taking into consideration a focus on CBR. #### Tips and tricks Remember that, when live streaming, everything is constrained to a minimum realtime encoding speed of 1x (FFmpeg reports encoding speed as it progresses). If your encoding speed drops below 1x then the encoding process will not keep up with the input of live video, and users will experience buffering, and breaks in the transmission will render the stream unusable during the live broadcast (although the archive will be generally usable). ## Examples of encoding parameters in action The following shows CPU utilization at 25 fps for various frame sizes on a quad-core i5 3.6Ghz desktop running Linux: | Target Resolution | FFmpeg VP9 Parameters | CPU / Speed (example) | |---|---|---| | 3840x2160 (2160p) | -r 30 -g 90 -s 3840x2160 -quality realtime -speed 5 -threads 16 -row-mt 1 -tile-columns 3 -frame-parallel 1 -qmin 4 -qmax 48 -b:v 7800k | \~88% 0.39x | | 2560x1440 (1440p) | -r 30 -g 90 -s 2560x1440 -quality realtime -speed 5 -threads 16 -row-mt 1 -tile-columns 3 -frame-parallel 1 -qmin 4 -qmax 48 -b:v 6000k | \~86% 0.68x | | 1920x1080 (1080p) | -r 30 -g 90 -s 1920x1080 -quality realtime -speed 5 -threads 8 -row-mt 1 -tile-columns 2 -frame-parallel 1 -qmin 4 -qmax 48 -b:v 4500k | \~82% 1.04x | | 1280x720 (720p) | -r 30 -g 90 -s 1280x720 -quality realtime -speed 5 -threads 8 -row-mt 1 -tile-columns 2 -frame-parallel 1 -qmin 4 -qmax 48 -b:v 3000k | \~78% 1.77x | | 854x480 (480p) | -r 30 -g 90 -s 854x480 -quality realtime -speed 6 -threads 4 -row-mt 1 -tile-columns 1 -frame-parallel 1 -qmin 4 -qmax 48 -b:v 1800k | \~64% 3.51x | | 640x360 (360p) | -r 30 -g 90 -s 640x360 -quality realtime -speed 7 -threads 4 -row-mt 1 -tile-columns 1 -frame-parallel 0 -qmin 4 -qmax 48 -b:v 730k | \~62% 5.27x | | 426x240 (240p) | -r 30 -g 90 -s 426x240 -quality realtime -speed 8 -threads 2 -row-mt 1 -tile-columns 0 -frame-parallel 0 -qmin 4 -qmax 48 -b:v 365k | \~66% 8.27x | An example FFmpeg might look like this: ffmpeg -stream_loop 100 -i /home/id3as/Videos/120s_tears_of_steel_1080p.webm \ -r 30 -g 90 -s 3840x2160 -quality realtime -speed 5 -threads 16 -row-mt 1 \ -tile-columns 3 -frame-parallel 1 -qmin 4 -qmax 48 -b:v 7800k -c:v libvpx-vp9 \ -b:a 128k -c:a libopus -f webm pipe1 #### Tips and tricks - Note that here we are outputting to a FIFO pipe ("pipe1"), which should be created before execution, before running the FFmpeg command. To do this, give the command `mkfifo pipe1` in your working directory. When using [Shaka Packager](https://github.com/google/shaka-packager), it will listen to that pipe as its input source for the given stream. Other packaging models may require a different method. - To ensure `-row-mt` commands are recognized, use the latest stable release of FFmpeg (3.3.3 currently) from ## Example adaptive bitrate set Depending on the power of the machine running the FFmpeg encode it may or may not be possible to deliver all of the following encodings at the same time, so a subset suited to your own available resources and target audiences should be selected from the list. ### FFmpeg full ABR set In an ideal scenario we combine the encoding examples outlined in the preceding section to create a single command that produces them all at the same time: ffmpeg -stream_loop 100 -i lakes1080p.mp4 \ -y -r 25 -g 75 -s 3840x2160 -quality realtime -speed 5 -threads 8 \ -tile-columns 2 -frame-parallel 1 \ -b:v 7800k -c:v libvpx-vp9 -b:a 196k -c:a libopus -f webm pipe1 \ -y -r 25 -g 75 -s 2560x1440 -quality realtime -speed 5 -threads 8 \ -tile-columns 2 -frame-parallel 1 \ -b:v 6000k -c:v libvpx-vp9 -b:a 196k -c:a libopus -f webm pipe2 \ -y -r 25 -g 75 -s 1920x1080 -quality realtime -speed 5 -threads 4 \ -tile-columns 2 -frame-parallel 1 \ -b:v 4500k -c:v libvpx-vp9 -b:a 196k -c:a libopus -f webm pipe3 \ -y -r 25 -g 75 -s 1280x720 -quality realtime -speed 5 -threads 4 \ -tile-columns 2 -frame-parallel 1 \ -b:v 3000k -c:v libvpx-vp9 -b:a 196k -c:a libopus -f webm pipe4 \ -y -r 25 -g 75 -s 854x480 -quality realtime -speed 6 -threads 4 \ -tile-columns 2 -frame-parallel 1 \ -b:v 2000k -c:v libvpx-vp9 -b:a 196k -c:a libopus -f webm pipe5 \ -y -r 25 -g 75 -s 640x360 -quality realtime -speed 7 -threads 2 \ -tile-columns 1 -frame-parallel 0 \ -b:v 730k -c:v libvpx-vp9 -b:a 128k -c:a libopus -f webm pipe6 \ -y -r 25 -g 75 -s 426x240 -quality realtime -speed 8 -threads 2 \ -tile-columns 1 -frame-parallel 0 \ -b:v 365k -c:v libvpx-vp9 -b:a 64k -c:a libopus -f webm pipe7 The above full set will, however, require a very powerful CPU, or possibly support from hardware GPU offload such as some chipsets increasingly provide. Intel Kabylake (and beyond) has a full hardware encoding pipeline. (Note that the Kabylake GPU can do 8-bit VP9 encode, but not 10-bit). ### A practical desktop example using Shaka Packager A more practical example for common desktop machines might use [Shaka Packager](https://github.com/google/shaka-packager). A simple way to setup Shaka is to install it within a Docker container, using Google's DockerHub image. Instructions can be found here: For this example, we used a machine with the following configuration: | | | |---|---| | **System** | Host: obs Kernel: 4.4.0-91-lowlatency x86_64 (64 bit) | | **Desktop** | Xfce 4.12.3 Distro: OS: | | **CPU** | Quad core Intel Core i5-6500 (-MCP-) cache: 6144 KB clock speeds: max: 3600 MHz 1: 800 MHz 2: 800 MHz 3: 800 MHz 4: 800 MHz | | **Graphics Card** | Intel Skylake integrated graphics | | **Memory** | 8GB RAM | In practice this machine could optimally produce the following usable range of ABR encodes, with FFmpeg consistently reporting 1x encode speed: ffmpeg -stream_loop 100 -i 120s_tears_of_steel_1080p.webm \ -y -r 30 -g 90 -s 1920x1080 -quality realtime -speed 7 -threads 8 \ -row-mt 1 -tile-columns 2 -frame-parallel 1 -qmin 4 -qmax 48 \ -b:v 4500k -c:v libvpx-vp9 -b:a 128k -c:a libopus -f webm pipe1 \ -y -r 30 -g 90 -s 1280x720 -quality realtime -speed 8 -threads 6 \ -row-mt 1 -tile-columns 2 -frame-parallel 1 -qmin 4 -qmax 48 \ -b:v 3000k -c:v libvpx-vp9 -b:a 128k -c:a libopus -f webm pipe2 \ -y -r 30 -g 90 -s 640x360 -quality realtime -speed 8 -threads 2 \ -row-mt 1 -tile-columns 1 -frame-parallel 1 -qmin 4 -qmax 48 \ -b:v 730k -c:v libvpx-vp9 -b:a 128k -c:a libopus -f webm pipe3 Note that the `-speed` settings are quite high. These settings were established experimentally and **will** vary from machine to machine. ### Shaka Packager overhead Packaging is not a particularly CPU-intensive activity. Shaka Packager can be set to listen for all the outputs, even if only a subset are being delivered by FFmpeg. These are the packager settings tested on the machine outlined above: packager \ in=pipe1,stream=audio,init_segment=livehd-audio-1080.webm,segment_template=livehd-audio-1080-\$Number\$.webm \ in=pipe1,stream=video,init_segment=livehd-video-1080.webm,template=livehd-video-1080-\$Number\$.webm \ in=pipe2,stream=audio,init_segment=livehd-audio-720.webm,segment_template=livehd-audio-720-\$Number\$.webm \ in=pipe2,stream=video,init_segment=livehd-video-720.webm,template=livehd-video-720-\$Number\$.webm \ in=pipe3,stream=audio,init_segment=livehd-audio-360.webm,segment_template=livehd-audio-360-\$Number\$.webm \ in=pipe3,stream=video,init_segment=livehd-video-360.webm,template=livehd-video-360-\$Number\$.webm \ --mpd_output livehd.mpd --dump_stream_info --min_buffer_time=10 --time_shift_buffer_depth=300 \ --segment_duration=3 --io_block_size 65536