VP9 HDR Encoding

High Dynamic Range (HDR) video is a maturing technology. There are today a number of gradually-harmonizing standards.

For a maintained picture of HDR video evolution see the WikiPedia article High-dynamic-range video.

FFmpeg can be used along with VP9 to both (1) compress HDR sources to Standard Dynamic Range (SDR) outputs or (2) to repackage an HDR source into HDR web-friendly formats.

Input sources

Currently, when encoding HDR with VP9, any high bit depth (10-bit) codecs supported by FFmpeg can be used as input (for example, VP9 Profile 2 and 3, H.264 (10-bit), DNxHR, HEVC, ProRes, et al.). Also see the YouTube support article Upload High Dynamic Range (HDR) videos.

Encoded output reach:

Currently and in general, target devices supported include any device that can decode VP9 Profile 2 and output to an HDR display.

  • Chromecast Ultra + HDR-capable TV
  • Chrome 64 Canary on Windows 10 Fall Creators Update, with HDR flag turned on. (This is a good debugging tool).
  • Any HDR TV with VP9 Profile 2
    • All Samsung 2017 HDR models (flatpanelshd.com list) and 2016 models starting with "KS"
    • All LG 2017 HDR models (also some 2016 LG G6 models)
    • On mobile, devices that support hardware-accelerated VP9 Profile 2 10-bit HDR decode (Meizu Pro 7)
    • UHD Bluray from 2017

FFmpeg command-line settings to enable VP9 Profile 2 and HDR EOTFs

Your FFmpeg will need to have been built with 10-bit (or even 12-bit) support.

In the examples below we use a statically-built ffmpeg with 10-bit support. You may use our build script to make your own, or follow the guidance on the FFmpeg site.

Following is a usable FFmpeg command line for HDR encoding to SMPTE 2084 (PQ EOTF) standards with VP9:

ffmpeg -i strobe_scientist_18Mbps.webm -b:v 18000000 -pass 1 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate 26800000 -minrate 8040000 -profile:v 2 -vcodec libvpx-vp9 /dev/null && \
   ffmpeg -i strobe_scientist_18Mbps.webm -b:v 18000000 -pass 2 \
   -pix_fmt yuv420p10le \
   -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
   -maxrate 26800000 -minrate 8040000 -profile:v 2 -vcodec libvpx-vp9 \
   strobe_scientist_18Mbps.webm

Breaking out the key parameters introduced here:

ffmpeg The static build with 10-bit support
-pass 1 HDR requires 2-pass encoding. In this example, the first pass is output to /dev/null and processed in memory during the second pass.
-pix_fmt yuv420p10le Sets YUV 4:2:0 10-bit pixel format
-color_primaries 9 Sets BT2020 See page 5 and 6 of this document for more detail.
-color_trc 16 16 sets Transfer Characteristics to SMPTE 2084. PQ 18 sets Transfer Characteristics SMPTE 2086 HLG
-colorspace 9 colorspace is the "matrix_coefficients." This should be set consistent with how the content was mastered. (e.g., bt709, bt2020_ncl). In this example, it should be 9 for bt2020_ncl. For further reference see page 12 of this document
-color_range 1 Color range (0 = unspecified, 1 = mpeg/studio/tv, 2 = jpeg/full/pc)
-profile:v 2 For HDR, a profile or 2 or 3 is required.
-vcodec libvpx-vp9 Use the VP9 encoder.

Example encodes

The following examples all use the source file strobe_scientist.mkv

As a general note on bitrates with HDR encoding, a 25-30% higher bitrate than equivalent 8-bit SDR encoding is recommended. The examples typically target 18Mbps. Good HDR results can be achieved with 12Mbps bitrates with optimal tuning.

The encodes were performed on an Ubuntu Linux system with the following specifications:

  • Processor: 4x Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz
  • Memory (RAM): 8060MB (1492MB used)
  • Graphics: Intel HD Graphics 530 (Skylake GT2)
  • OS: Ubuntu 16.04 LTS

Convert to YUV 4:2:0 10-bit PQ

Example A: 6Mbps 4k 2-pass
ffmpeg -y -report -i strobe_scientist.mkv -b:v 6000000 -speed 4 -pass 1 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate 8000000 -minrate 4000000 \
  -profile:v 2 -vcodec libvpx-vp9 -f webm /dev/null && \
  ffmpeg -y -report -i strobe_scientist.mkv -b:v 6000000 -pass 2 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate 8000000 -minrate 4000000 \
  -profile:v 2 -vcodec libvpx-vp9 \
  2pass_HDR_strobe_scientist_6Mbps-static.webm
Example B1: 18Mbps 4k 2-pass
ffmpeg -y -report -i strobe_scientist.mkv -b:v **18000000** -speed 4 -pass 1 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate **20800000** -minrate **15040000** \
  -profile:v 2 -vcodec libvpx-vp9 -f webm /dev/null && \
  ffmpeg -y -report -i strobe_scientist.mkv -b:v 18000000 -pass 2 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate **20800000** -minrate **15040000** \
  -profile:v 2 -vcodec libvpx-vp9 \
  2pass_HDR_strobe_scientist_18Mbps-static.webm
Example B2: 18Mbps 1080p 2-pass
ffmpeg -y -report -i strobe_scientist.mkv -b:v 18000000 -speed 4 -pass 1 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate 20800000 -minrate 15040000 \
  -profile:v 2 **-vf ****scale=-1:1080**** **-vcodec libvpx-vp9 \
  -f webm /dev/null && \
  ffmpeg -y -report -i strobe_scientist.mkv -b:v 18000000 -pass 2 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
  -maxrate 20800000 -minrate 15040000 \
  -profile:v 2 **-vf ****scale=-1:1080**** **-vcodec libvpx-vp9 \
  1080_2pass_HDR_strobe_scientist_18Mbps-static.webm

Convert HDR to SDR with a look-up table (LUT)

Converting HDR to SDR requires the transcoder to understand how to map brightness levels and colors to the smaller BT.709 color space and non-HDR range of brightness. For best results, an LUT describing how to perform that mapping -- usually specific to the content -- can be used. Examples C and D show how to use a LUT. We used bt2020_to_bt709_example.cube

Example C: 18Mbps 4k HDR to SDR with LUT
ffmpeg -i strobe_scientist.mkv -y -b:v 18000000 -speed 4 -pass 1 \
  -pix_fmt yuv420p \
  -color_primaries 1 -color_trc 1 -colorspace 1 -color_range 1 \
  -maxrate 26800000 -minrate 8040000 -profile:v 0 \
  -vf scale=-1:-1:in_color_matrix=bt2020,format=rgb48,lut3d=bt2020_to_bt709_example.cube,scale=-1:-1:out_color_matrix=bt709 \
  -vcodec libvpx-vp9 -f webm /dev/null && \
  ffmpeg -i strobe_scientist.mkv -y -b:v 18000000 -pass 2 \
  -pix_fmt yuv420p \
  -color_primaries 1 -color_trc 1 -colorspace 1 -color_range 1 \
  -maxrate 26800000 -minrate 8040000 -profile:v 0 \
  -vf scale=-1:-1:in_color_matrix=bt2020,format=rgb48,lut3d=bt2020_to_bt709_example.cube,scale=-1:-1:out_color_matrix=bt709 \
  -vcodec libvpx-vp9 -f webm SDR_strobe_scientist_18Mbps-static.webm

Convert HLG to VP9 HLG10 10-bit PQ

Requires an HLG source. We used the input file strobe_scientist_hlg.mkv

Example D: 18Mbps 4k 2-Pass HLG
ffmpeg -y -i strobe_scientist_hlg.mkv -b:v 18000000 -pass 1 -speed 4 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 18 -colorspace 9 -color_range 1 \
  -maxrate 26800000 -minrate 8040000 \
  -profile:v 2 -vcodec libvpx-vp9 -f webm /dev/null && \
  ffmpeg -y -i strobe_scientist_hlg.mkv -b:v 18000000 -pass 2 \
  -pix_fmt yuv420p10le \
  -color_primaries 9 -color_trc 18 -colorspace 9 -color_range 1 \
  -maxrate 26800000 -minrate 8040000 \
  -profile:v 2 -vcodec libvpx-vp9 \
  HLG_HDR_strobe_scientist_18Mbps-static.webm

Summary

The output files are of varying size. The input was 4.3GB.

Example Target Output Size on Disk Encode Time
A 6Mbps 4k Unscaled HDR 51.2MB 142m
B1 18Mbps 4k unscaled HDR 161MB 213m
B2 18Mbps 1080 scaled HDR 160MB 113m
C 18Mbps HDR to SDR unscaled 165MB 124m
D 18Mbps HLG conversion 165MB 168m

Note that 2-pass encoding is recommended for HDR, since 1-pass can vary in certain configurations.

Tools needed to signal HDR in WebM and MP4 output (Matroska command-line tools)

At the time of writing, FFmpeg doesn't provide a mechanism for specifying SMPTE 2086 static metadata in WebM/MKV, although it will propagate it from an input stream when the stream contains it.

The mkvmerge tool, part of mkvtoolnix, can be used to insert or modify this metadata if your application requires it.

Following is an example of inserting HDR metadata into one of the files created above. It is especially suited to preparing HDR content for upload to YouTube.

mkvmerge \
  -o HDR_strobe_scientist_18Mbps.mkv\
  --colour-matrix 0:9 \
  --colour-range 0:1 \
  --colour-transfer-characteristics 0:16 \
  --colour-primaries 0:9 \
  --max-content-light 0:1000 \
  --max-frame-light 0:300 \
  --max-luminance 0:1000 \
  --min-luminance 0:0.01 \
  --chromaticity-coordinates 0:0.68,0.32,0.265,0.690,0.15,0.06 \
  --white-colour-coordinates 0:0.3127,0.3290 \
  HDR_strobe_scientist_18Mbps.webm

This example produces the file HDR_strobe_scientist_18Mbps.mkv, in 0.6 seconds.