NetFlix open source movie, Meridian

[sophisticles]

I wasn't sure which sub-forum would be best, but to me this does fall under "Latest Video News".

NetFlix has created a 12 minute short movie and released the assets under a Creative Commons License:

http://variety.com/2016/digital/news/netflix-meridian-imf-tools-open-source-1201859416/

This film is unique in that it's purposely crafted in such a way as to be a torture test for encoding algorithms, hardware and work flows.

I downloaded the 90gb test file and it lives up to its purpose, one interesting thing I learned, opening the file with Media Info reveals the following:

MERIDIAN.mxf
Format : MXF
Format version : 1.3
Format profile : OP-1a
Format settings : Closed / Complete
File size : 88.5 GiB
Duration : 11 min 58 s
Overall bit rate : 1 058 Mb/s
Encoded date : 2016-09-09 11:29:39.012
Writing application : Colorfront Transkoder 2016.0.0.29813.0
Writing library : Microsoft Windows 7 Professional Service Pack 1 (Build 7601) 2.7.3.0.1

Video
ID : 2
Format : JPEG 2000
Format profile : 206
Format settings, wrapping mode : Frame
Codec ID : 0D010301020C0100-040102020301017F
Duration : 11 min 58 s
Bit rate : 1 058 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate : 59.940 (60000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 10 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 2.128
Stream size : 88.5 GiB (100%)
Title : Picture
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Other #1
ID : 1-Material
Type : Time code
Format : MXF TC
Time code of first frame : 00:00:17;12
Time code settings : Material Package
Time code, striped : Yes

Other #2
ID : 1-Source
Type : Time code
Format : MXF TC
Time code of first frame : 00:00:17;12
Time code settings : Source Package
Time code, striped : Yes

So, NetFlix uses Win 7 pc's and they use something called Colorfront Transkoder, googling it brings us to the following:

http://www.colorfront.com/index.php?page=SOFTWARE&spage=Transkoder

It looks like they use HP based workstations, and reading through the docs it seems evident that they rely on gpu acceleration because they claim 100+ fps 4k hevc encoding and they claim that their solution makes use of multi-gpu setups.

As an interesting side note, I've been playing around with NVENC on Ubuntu via a self-compiled ffmpeg build and while reading through all the docs on NVIDIA's website on NVENC I came across a very interesting bit of info: NVENC, is not entirely based on a dedicated hardware block, it's actually partially powered by CUDA!

Additionally, since some of the complexity estimation is performed in CUDA, there may be some performance impact when temporal AQ is
enabled:

http://developer.download.nvidia.com/designworks/video-codec-sdk/secure/7.1/01/NVENC_V..._ProgGuide.pdf

You can find it under the Temporal AQ section.

Even more interesting, I found the following under "Motion-Estimation-Only Mode":

NVENC can be used as a hardware accelerator to perform motion search and generate motion vectors and mode information. The resulting motion vectors or mode decisions can be used, for example, in motion compensated filtering or for supporting other codecs not fully supported by NVENC or simply as motion vector hints for a custom encoder. The procedure to use the feature is explained below.

So, if one where so inclined, one could create a xvid, x264, or x265 fork where the software motion vector algorithms were removed and instead used a NVENC as a brute force approach.

[transporterfan]

Very nice. For anyone wanting short video test material, the main directory holds 'Derf's Test Media Collection'. A selection in various flavors.
https://media.xiph.org/video/derf/

[poisondeathray]

As an interesting side note, I've been playing around with NVENC on Ubuntu via a self-compiled ffmpeg build and while reading through all the docs on NVIDIA's website on NVENC I came across a very interesting bit of info: NVENC, is not entirely based on a dedicated hardware block, it's actually partially powered by CUDA!

Additionally, since some of the complexity estimation is performed in CUDA, there may be some performance impact when temporal AQ is
enabled:

http://developer.download.nvidia.com/designworks/video-codec-sdk/secure/7.1/01/NVENC_V..._ProgGuide.pdf

You can find it under the Temporal AQ section.

Even more interesting, I found the following under "Motion-Estimation-Only Mode":

NVENC can be used as a hardware accelerator to perform motion search and generate motion vectors and mode information. The resulting motion vectors or mode decisions can be used, for example, in motion compensated filtering or for supporting other codecs not fully supported by NVENC or simply as motion vector hints for a custom encoder. The procedure to use the feature is explained below.

So, if one where so inclined, one could create a xvid, x264, or x265 fork where the software motion vector algorithms were removed and instead used a NVENC as a brute force approach.

That is interesting...

Your pdf link doesn't work for me (maybe limited by session id) , and the first few google entries had the wrong older version. The one directly from searching the site is dated Nov 2016. Anyways here is the relevant excerpt

Although the core video encoder hardware on GPU is completely independent of CUDA
cores or graphics engine on the GPU, following encoder features internally use CUDA
for hardware acceleration. Note that the impact of enabling these features on overall
CUDA or graphics performance is minimal and this list is provided purely for
information purposes.

Two-pass rate control modes for high quality presets.
Look-ahead
All adaptive quantization modes.
Encoding with inputs in RGB formats.

And that would be awesome if someone could program some ME filters using NVEnc , or if you could utilize the vectors in other applications . I don't know how "good" the ME component is or what algorithms are used, but it still could be useful

[sophisticles]

I'm attaching it here.

[KarMa]

It's 4K yet still BT.709 and not BT.2020. Interesting. And with the JPEG 2000 it looks like it's designed to play in a digital movie theater. Nice share, thanks.