## VP9 HDR Encoding

High Dynamic Range (HDR) video is a maturing technology. There are today a
number of gradually-harmonizing standards.

For a maintained picture of HDR video evolution see the WikiPedia article
[**High-dynamic-range video**](https://en.wikipedia.org/wiki/High-dynamic-range_video).

FFmpeg can be used along with VP9 to both (1) compress HDR sources to Standard
Dynamic Range (SDR) outputs or (2) to repackage an HDR source into HDR
web-friendly formats.

### Input sources

Currently, when encoding HDR with VP9, any high bit depth (10-bit) codecs
supported by FFmpeg can be used as input (for example, VP9 Profile 2 and 3,
H.264 (10-bit), DNxHR, HEVC, ProRes, et al.). Also see the YouTube support
article
[**Upload High Dynamic Range (HDR) videos**](https://support.google.com/youtube/answer/7126552).

### Encoded output reach:

Currently and in general, target devices supported include any device that can
decode VP9 Profile 2 and output to an HDR display.

- Chromecast Ultra + HDR-capable TV
- Chrome 64 Canary on Windows 10 Fall Creators Update, with HDR flag turned on. (This is a good debugging tool).
- Any HDR TV with VP9 Profile 2
  - All Samsung 2017 HDR models ([flatpanelshd.com list](http://flatpanelshd.com/article.php?subaction=showfull&id=1493363368)) and 2016 models starting with "KS"
  - All LG 2017 HDR models (also some 2016 LG G6 models)
  - On mobile, devices that support hardware-accelerated VP9 Profile 2 10-bit HDR decode ([Meizu Pro 7](https://www.meizu.com/en/pro7/summary/))
  - UHD Bluray from 2017

> [!NOTE]
> **Note:** More info can be found in the excellent and comprehensive [**YouTube HDR Creators Guide**](https://docs.google.com/document/u/1/d/1OHGOE4Ihv6SKazfiub_DP1lJbR9PdMMPOQYxPJQAES4/pub#h.3v06o2et6qxw), and more info about VP9 Profiles can be found [at the Webm
> Project site.](https://www.webmproject.org/vp9/profiles/)

### FFmpeg command-line settings to enable VP9 Profile 2 and HDR EOTFs

Your FFmpeg will need to have been built with 10-bit (or even 12-bit) support.

In the examples below we use a statically-built `ffmpeg` with 10-bit support.
You may use our [build script](https://github.com/id3as/ffmpeg-libvpx-HDR-static)
to make your own, or follow the
[guidance on the FFmpeg site](https://trac.ffmpeg.org/wiki/Encode/VP9).

Following is a usable FFmpeg command line for HDR encoding to
[SMPTE 2084 (PQ EOTF)](https://www.smpte.org/publications/past-issues/december-2016)
standards with VP9:

    ffmpeg -i strobe_scientist_18Mbps.webm -b:v 18000000 -pass 1 \
      -pix_fmt yuv420p10le \
      -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
      -maxrate 26800000 -minrate 8040000 -profile:v 2 -vcodec libvpx-vp9 /dev/null && \
       ffmpeg -i strobe_scientist_18Mbps.webm -b:v 18000000 -pass 2 \
       -pix_fmt yuv420p10le \
       -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
       -maxrate 26800000 -minrate 8040000 -profile:v 2 -vcodec libvpx-vp9 \
       strobe_scientist_18Mbps.webm

Breaking out the key parameters introduced here:

| ffmpeg | The static build with 10-bit support |
|---|---|
| `-pass 1` | HDR requires 2-pass encoding. In this example, the first pass is output to `/dev/null` and processed in memory during the second pass. |
| `-pix_fmt yuv420p10le` | Sets YUV 4:2:0 10-bit pixel format |
| `-color_primaries 9` | Sets `BT2020` See page 5 and 6 of this document for more detail. |
| `-color_trc 16` | `16` sets Transfer Characteristics to SMPTE 2084. PQ `18` sets Transfer Characteristics SMPTE 2086 HLG |
| `-colorspace 9` | `colorspace` is the "matrix_coefficients." This should be set consistent with how the content was mastered. (e.g., bt709, bt2020_ncl). In this example, it should be `9` for bt2020_ncl. For further reference see page 12 of this document |
| `-color_range 1` | Color range (`0` = unspecified, `1` = mpeg/studio/tv, `2` = jpeg/full/pc) |
| `-profile:v 2` | For HDR, a profile or `2` or `3` is required. |
| `-vcodec libvpx-vp9` | Use the VP9 encoder. |

### Example encodes

The following examples all use the source file [strobe_scientist.mkv](https://storage.googleapis.com/media.webmproject.org/devsite/vp9/hdr-encoding/strobe_scientist.mkv)

As a general note on bitrates with HDR encoding, a 25-30% higher bitrate than
equivalent 8-bit SDR encoding is recommended. The examples typically target
18Mbps. Good HDR results can be achieved with 12Mbps bitrates with optimal
tuning.

The encodes were performed on an Ubuntu Linux system with the following
specifications:

- Processor: 4x Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz
- Memory (RAM): 8060MB (1492MB used)
- Graphics: Intel HD Graphics 530 (Skylake GT2)
- OS: Ubuntu 16.04 LTS

#### Convert to YUV 4:2:0 10-bit PQ

##### Example A: 6Mbps 4k 2-pass

    ffmpeg -y -report -i strobe_scientist.mkv -b:v 6000000 -speed 4 -pass 1 \
      -pix_fmt yuv420p10le \
      -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
      -maxrate 8000000 -minrate 4000000 \
      -profile:v 2 -vcodec libvpx-vp9 -f webm /dev/null && \
      ffmpeg -y -report -i strobe_scientist.mkv -b:v 6000000 -pass 2 \
      -pix_fmt yuv420p10le \
      -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
      -maxrate 8000000 -minrate 4000000 \
      -profile:v 2 -vcodec libvpx-vp9 \
      2pass_HDR_strobe_scientist_6Mbps-static.webm

- Output file: [2pass_HDR_strobe_scientist_6Mbps-static.webm](https://storage.googleapis.com/media.webmproject.org/devsite/vp9/output/hdr/2pass_HDR_strobe_scientist_6Mbps-static.webm)
- Encode time: 142m30s
- File size on disc: 51.2MB

##### Example B1: 18Mbps 4k 2-pass

    ffmpeg -y -report -i strobe_scientist.mkv -b:v **18000000** -speed 4 -pass 1 \
      -pix_fmt yuv420p10le \
      -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
      -maxrate **20800000** -minrate **15040000** \
      -profile:v 2 -vcodec libvpx-vp9 -f webm /dev/null && \
      ffmpeg -y -report -i strobe_scientist.mkv -b:v 18000000 -pass 2 \
      -pix_fmt yuv420p10le \
      -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
      -maxrate **20800000** -minrate **15040000** \
      -profile:v 2 -vcodec libvpx-vp9 \
      2pass_HDR_strobe_scientist_18Mbps-static.webm

- Output file: [2pass_HDR_strobe_scientist_18Mbps-static.webm](https://storage.googleapis.com/media.webmproject.org/devsite/vp9/output/hdr/2pass_HDR_strobe_scientist_18Mbps-static.webm)
- Encode time: 213m32s
- File size on disc: 161MB

##### Example B2: 18Mbps 1080p 2-pass

    ffmpeg -y -report -i strobe_scientist.mkv -b:v 18000000 -speed 4 -pass 1 \
      -pix_fmt yuv420p10le \
      -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
      -maxrate 20800000 -minrate 15040000 \
      -profile:v 2 **-vf ****scale=-1:1080**** **-vcodec libvpx-vp9 \
      -f webm /dev/null && \
      ffmpeg -y -report -i strobe_scientist.mkv -b:v 18000000 -pass 2 \
      -pix_fmt yuv420p10le \
      -color_primaries 9 -color_trc 16 -colorspace 9 -color_range 1 \
      -maxrate 20800000 -minrate 15040000 \
      -profile:v 2 **-vf ****scale=-1:1080**** **-vcodec libvpx-vp9 \
      1080_2pass_HDR_strobe_scientist_18Mbps-static.webm

- Output file: [1080_2pass_HDR_strobe_scientist_18Mbps-static.webm](https://storage.googleapis.com/media.webmproject.org/devsite/vp9/output/hdr/1080_2pass_HDR_strobe_scientist_18Mbps-static.webm)
- Encode time: 113m9s
- File size on disc: 160MB

#### Convert HDR to SDR with a look-up table (LUT)

Converting HDR to SDR requires the transcoder to understand how to map
brightness levels and colors to the smaller BT.709 color space and non-HDR
range of brightness. For best results, an LUT describing how to perform that
mapping -- usually specific to the content -- can be used. Examples C and D show
how to use a LUT. We used
[bt2020_to_bt709_example.cube](https://storage.googleapis.com/media.webmproject.org/devsite/vp9/input/hdr/bt2020_to_bt709_example.cube)

> [!NOTE]
> **Note:** The LUT above is provided as an example of how to use an LUT with FFmpeg and is not optimized for the content in the example. LUTs are nearly always optimized for specific content.

##### Example C: 18Mbps 4k HDR to SDR with LUT

    ffmpeg -i strobe_scientist.mkv -y -b:v 18000000 -speed 4 -pass 1 \
      -pix_fmt yuv420p \
      -color_primaries 1 -color_trc 1 -colorspace 1 -color_range 1 \
      -maxrate 26800000 -minrate 8040000 -profile:v 0 \
      -vf scale=-1:-1:in_color_matrix=bt2020,format=rgb48,lut3d=bt2020_to_bt709_example.cube,scale=-1:-1:out_color_matrix=bt709 \
      -vcodec libvpx-vp9 -f webm /dev/null && \
      ffmpeg -i strobe_scientist.mkv -y -b:v 18000000 -pass 2 \
      -pix_fmt yuv420p \
      -color_primaries 1 -color_trc 1 -colorspace 1 -color_range 1 \
      -maxrate 26800000 -minrate 8040000 -profile:v 0 \
      -vf scale=-1:-1:in_color_matrix=bt2020,format=rgb48,lut3d=bt2020_to_bt709_example.cube,scale=-1:-1:out_color_matrix=bt709 \
      -vcodec libvpx-vp9 -f webm SDR_strobe_scientist_18Mbps-static.webm

> [!NOTE]
> **Note:** `-profile` has been set to `0` (4:2:0) and `-pix_fmt` changed to `yuv420p`.

- Output file: [SDR_strobe_scientist_18Mbps-static.webm](https://storage.googleapis.com/media.webmproject.org/devsite/vp9/output/hdr/SDR_strobe_scientist_18Mbps-static.webm)
- Encode time: 124m23s
- File size on disc: 165MB

#### Convert HLG to VP9 HLG10 10-bit PQ

Requires an HLG source. We used the input file
[strobe_scientist_hlg.mkv](https://storage.googleapis.com/media.webmproject.org/devsite/vp9/input/hdr/strobe_scientist_hlg.mkv)

##### Example D: 18Mbps 4k 2-Pass HLG

    ffmpeg -y -i strobe_scientist_hlg.mkv -b:v 18000000 -pass 1 -speed 4 \
      -pix_fmt yuv420p10le \
      -color_primaries 9 -color_trc 18 -colorspace 9 -color_range 1 \
      -maxrate 26800000 -minrate 8040000 \
      -profile:v 2 -vcodec libvpx-vp9 -f webm /dev/null && \
      ffmpeg -y -i strobe_scientist_hlg.mkv -b:v 18000000 -pass 2 \
      -pix_fmt yuv420p10le \
      -color_primaries 9 -color_trc 18 -colorspace 9 -color_range 1 \
      -maxrate 26800000 -minrate 8040000 \
      -profile:v 2 -vcodec libvpx-vp9 \
      HLG_HDR_strobe_scientist_18Mbps-static.webm

> [!NOTE]
> **Note:** `-color_trc` has been set to `18`, to signal HLG.

- Output file: [HLG_HDR_strobe_scientist_18Mbps-static.webm](https://storage.googleapis.com/media.webmproject.org/devsite/vp9/output/hdr/HLG_HDR_strobe_scientist_18Mbps-static.webm)
- Encode time: 186m10s
- File size on disc: 165MB

### Summary

The output files are of varying size. The input was 4.3GB.

| Example | Target Output | Size on Disk | Encode Time |
|---|---|---|---|
| A | 6Mbps 4k Unscaled HDR | 51.2MB | 142m |
| B1 | 18Mbps 4k unscaled HDR | 161MB | 213m |
| B2 | 18Mbps 1080 scaled HDR | 160MB | 113m |
| C | 18Mbps HDR to SDR unscaled | 165MB | 124m |
| D | 18Mbps HLG conversion | 165MB | 168m |

Note that 2-pass encoding is recommended for HDR, since 1-pass can vary in
certain configurations.

### Tools needed to signal HDR in WebM and MP4 output (Matroska command-line tools)

At the time of writing, FFmpeg doesn't provide a mechanism for specifying SMPTE
2086 static metadata in WebM/MKV, although it will propagate it from an input
stream when the stream contains it.

The [mkvmerge](https://mkvtoolnix.download/doc/mkvmerge.html) tool, part of
[mkvtoolnix](https://mkvtoolnix.download/), can be used to insert or modify this
metadata if your application requires it.

Following is an example of inserting HDR metadata into one of the files created
above. It is especially suited to
[preparing HDR content for upload to YouTube](https://github.com/YouTubeHDR/hdr_metadata).

    mkvmerge \
      -o HDR_strobe_scientist_18Mbps.mkv\
      --colour-matrix 0:9 \
      --colour-range 0:1 \
      --colour-transfer-characteristics 0:16 \
      --colour-primaries 0:9 \
      --max-content-light 0:1000 \
      --max-frame-light 0:300 \
      --max-luminance 0:1000 \
      --min-luminance 0:0.01 \
      --chromaticity-coordinates 0:0.68,0.32,0.265,0.690,0.15,0.06 \
      --white-colour-coordinates 0:0.3127,0.3290 \
      HDR_strobe_scientist_18Mbps.webm

This example produces the file
[HDR_strobe_scientist_18Mbps.mkv](https://storage.googleapis.com/media.webmproject.org/devsite/vp9/output/hdr/HDR_strobe_scientist_18Mbps.mkv),
in 0.6 seconds.