Adobe Premiere - MainConcept's H.264 vs x264 open source encoder

[jagabo]

http://fourcc.org/fccyvrgb.php

[poisondeathray]

http://files.x264.nl/x264_patches

There might be a newer one hosted somewhere else, I would ask @ D9 when you get registered

There is talk about re-introducing slice support soon. The development of x264 really is at an incredible pace if you look over the last 2 years, there is a minor patch almost every day. I just want to warn you to be prepared because most GUI's and interfaces need changes every month or so to reflect the newest changes...

For the RGB equations, the guys you want to talk to are clsid, madshi and a few others in the software playback forum @ D9

[creamyhorror]

An initial slicing patch was done today, as PDR implies. We'll probably see it committed to the main build in a matter of weeks or even days, if the insane rate of progress recently is anything to go by.

rallymax, you might want to PM Lord Mulder (the guy who builds libx264). I think his builds may already include the NAL HRD patch. At the very least, he seems to be very helpful and quite knowledgeable.

Thanks for your hard work on this project. It's no doubt going to make powerful encoding much easier for a whole bunch of Premiere users.

[rallymax]

The presets are listed in this thread, which is slightly outdated. They are (as of this posting):

ultrafast (fastest / lowest quality)
veryfast
faster
fast
medium
slow
slower
veryslow (as slow as you should go / best quality)
placebo (pointlessly slow / bestest quality)

To use them in x264 CLI, you use the --preset switch, e.g. --preset slower. I assume there's an analogous way to call libx264.

There are also the --tune and --profile switches, which will probably be important to include:

--tune name where name = film, animation, grain, and a few other possibilities.

--tune film should be used for movies and other film sources, and may also work well for video sources. --tune animation is self-explanatory. --tune grain maximizes grain retention on source that have a lot of it.

--profile name where name = baseline, main or high.

This allows users to ensure compliance with H.264 profiles, and overrides everything else. It's generally only necessary when you're targeting playback on specific devices or less capable decoders (e.g. when streaming HD maybe).

The priority of these settings is:

Presets < Tunings < User settings < Profiles

This means that any clashing options will take precedence in that order. User settings will override any conflicting options from presets and tunings, while profiles override all the rest.

For playback on devices, it's important to have the VBV switches, which limit bitrate to ensure the device doesn't choke. In future, a --device switch may be created to make things even simpler.

You can find a reference to all of these options on this page at the MeGUI wiki.

Sorry if this is confusing; just ask whenever you have doubts.

Thanks. that info really helped.
I found how all those settings are parsed in the code and have dropped it into my code.
My plan is to present a tiered gui that visually shows the heirachy of choices as you described.

'getting close!

[creamyhorror]

Fantastic. Can't wait to see the first beta

[poisondeathray]

You might have already covered these, but for blu-ray / AVCHD some important settings are not reflected in the built in x264 presets:

1) GOP size (--min-keyint 2 --keyint 24) ; some other GUI's like ripbot have a larger GOP size, but on some standalones, you will notice pausing on FF/RW . Pro encoders like Cinevision also uses these sizes and they are considered to be the most compatible

2) b-pyramid screws up playback on most SAPs and mediaboxes

3) buffer size is different for AVCHD on DVD media vs. true blu-ray on BD-R 25/50 media. This has to do with the rotation speed and density on the media. (--vbv-bufsize 30000 --vbv-maxrate 40000 is what works for me and used in most blu-ray profiles for BD25/50)

Nice job rallymax! If you need beta tester, I'm available

[rallymax]

seeing as the patch rate seems to be so high at the moment and NAL HRD seems to be in the very near future I'm going to leave it out for now and then when it is committed I'll drop it in.
I have some time to spend on the GUI triggering the same behaviour as the CLI --preset, --tune and --profile do as well as getting around this RGB requirement so I can add the NAL HRD later (quite eaisily) - I'm not changing any of the libx264 so it should be fine.

[rallymax]

From the Adobe docs:

Premiere supports various depths of BGRA, VUYA (uncompressed YUV with alpha), and ARGB
(After Effects’ native format). It supports compressed NTSC DV25, compressed PAL DV25, V210,
720p and 1080i DV100, 4:2:2 YUY2, 4:2:2 UYVY, and 4:2:0 YUV MPEG2. All pixel formats use
the ITU-R Recommendation BT.601 color space unless noted.

[rallymax]

You might have already covered these, but for blu-ray / AVCHD some important settings are not reflected in the built in x264 presets:

1) GOP size (--min-keyint 2 --keyint 24) ; some other GUI's like ripbot have a larger GOP size, but on some standalones, you will notice pausing on FF/RW . Pro encoders like Cinevision also uses these sizes and they are considered to be the most compatible

2) b-pyramid screws up playback on most SAPs and mediaboxes

3) buffer size is different for AVCHD on DVD media vs. true blu-ray on BD-R 25/50 media. This has to do with the rotation speed and density on the media. (--vbv-bufsize 30000 --vbv-maxrate 40000 is what works for me and used in most blu-ray profiles for BD25/50)

Nice job rallymax! If you need beta tester, I'm available

Can you elaborate on that?
What should be the
GOP
b-pyramid
vbv values?

Are you saying I should have drop down box for "DEFAULT", "AVCHD-DVD" and "BluRay" that sets those values.
If so, what are the parameters I need to add for the latter two?

[creamyhorror]

You could consult the x264 Quicktime Codec source code to see how/if it handles RGB input.

The page jagabo linked has what looks to be some interesting but thorny information: you have to do a conversion with a different color matrix for HD video than you do for SD video. Premiere uses BT.601, which is the SD one. It's probably not something to worry about yet, though.

Actually, doesn't Premiere have the capability to export YUV/YV12 directly? It would stand to reason that it can, given that YV12 is the colorspace of all common consumer video and encoders may require it. (Sorry, quite clueless about the details, but I remember Premiere Pro CS3 being able to import and export in YUV...)

[rallymax]

unfortunately it requires that you support RGBA (8888). I can _ask_ for YUV but there is no guarantee you'll get that back.
The only time it can do YUV end to end is if the importer brought in YUV and the filters applied to it can process in YUV. Chances are that isn't going to happen most of the time since at min filters have to support RGBA and thus will be written for that color space first. After Effects filters are ARGB so all those won't fall through as YUV.
No biggie, I'll just do the conversions if needed. I won't bother until I start debugging and Premiere hands me a RGBA frame. ie I'll just assume a YUV frame for now.

[rallymax]

The page jagabo linked has what looks to be some interesting but thorny information: you have to do a conversion with a different color matrix for HD video than you do for SD video. Premiere uses BT.601, which is the SD one. It's probably not something to worry about yet, though.

Can I do the following choice:
If height > 480 or width > 720 then it's BT.709 otherwise it's BT.601?

[poisondeathray]

Can you elaborate on that?
What should be the
GOP
b-pyramid
vbv values?

Are you saying I should have drop down box for "DEFAULT", "AVCHD-DVD" and "BluRay" that sets those values.
If so, what are the parameters I need to add for the latter two?

Yes, I think you will need to differentiate between them. I don't know how difficult this would be to implement (and it kinda defeats the purpose of a GUI), but a command line box to pass thru extra commands is sometimes nice - this is definitely beneficial for CLI GUI builds as there are so frequent changes to the commands and switches

Settings for AVCHD on DVD5/9 are different than blu-ray. I'm not sure how it works in the libx264 version but the command line for the x264cli version for keyframe interval limits is the same for AVCHD and blu-ray for GOP size
--min-keyint 2 --keyint 24

As I alluded to earlier, some players have difficulty on ff/rwd with larger GOP sizes (such as those from ripbot264, which has --keyint 250 for the max key interval). I think it's safer to go with 2 and 24

--b-pyramid is the switch that enables b-pyramid, but on some x264 builds (like the recent ones with mbtree) it's turned off by default I think. On some builds, it might be enabled by default, on others disabled. It's much safer to have it OFF for AVCHD, blu-ray, DXVA purposes.

--vbv-bufsize 30000 --vbv-maxrate 40000 is the commonly accepted buffer values for BD25/50.

There are inconsistencies on what works/doesn't work for AVCHD (BD5/9). For example megui uses --vbv-bufsize 16500 --vbv-maxrate 16500 , but I go a little higher personally --vbv-bufsize 20000 --vbv-maxrate 25000

Also the stream muxing is different (e.g tsmuxer sets different values for blu-ray on bd25/50 vs. bd5/9). I'm not very well versed on the intricacies of the stream, but there are differences for sure. deank from D9 is the author of multiavchd (or roman for tsmuxer) and they knows this stuff very well, they would be the people to talk to if you are integrating a stream muxer in this

[rallymax]

bingo...
http://www.martinreddy.net/gfx/faqs/colorconv.faq
All you ever wanted to know about color spaces....

...snip...
[6.4] ITU.BT-601 Y'CbCr

This is the international standard for digital coding of TV pictures at 525 
and 625 line rates. It is independent of the scanning standard and the 
system primaries, therefore there are no chromaticity coordinates, no CIE 
XYZ matrices, and no assumptions about white point or CRT gamma. It deals 
only with the digital representation of R'G'B' signals in Y'CbCr form. 
The non-linear coding matrices are:

    Y'= 0.299*R' + 0.587*G' + 0.114*B'
    Cb=-0.169*R' - 0.331*G' + 0.500*B'
    Cr= 0.500*R' - 0.419*G' - 0.081*B'

    R'= Y' + 0.000*U' + 1.403*V'
    G'= Y' - 0.344*U' - 0.714*V'
    B'= Y' + 1.773*U' + 0.000*V'


[6.5] ITU.BT-709 HDTV studio production in Y'CbCr

This is a recent standard, defined only as an interim standard for HDTV 
studio production. It was defined by the CCIR (now the ITU) in 1988, but is 
not yet recommended for use in broadcasting. The primaries are the R and B 
from the EBU, and a G which is midway between SMPTE-C and EBU. The CRT gamma 
law is assumed to be 2.2. White is D65. The chromaticities are:
    R:      xr=0.64       yr=0.33
    G:      xg=0.30       yg=0.60
    B:      xb=0.15       yb=0.06
    White:  xn=0.312713   yn=0.329016

The conversion equations for linear signals are:
    X = 0.412*R + 0.358*G + 0.180*B
    Y = 0.213*R + 0.715*G + 0.072*B
    Z = 0.019*R + 0.119*G + 0.950*B

    R = 3.241*X - 1.537*Y - 0.499*Z
    G =-0.969*X + 1.876*Y + 0.042*Z
    B = 0.056*X - 0.204*Y + 1.057*Z

The coding equations for non-linear signals are:
    Y'= 0.2215*R' + 0.7154*G' + 0.0721*B'
    Cb=-0.1145*R' - 0.3855*G' + 0.5000*B'
    Cr= 0.5016*R' - 0.4556*G' - 0.0459*B'

    R'= Y' + 0.0000*Cb + 1.5701*Cr
    G'= Y' - 0.1870*Cb - 0.4664*Cr
    B'= Y' - 1.8556*Cb + 0.0000*Cr

The conversion equations between linear 709 RGB signals and EBU RGB signals 
are:
    Re= 0.9578*R7 + 0.0422*G7 + 0.0000*B7
    Ge= 0.0000*R7 + 1.0000*G7 + 0.0000*B7
    Be= 0.0000*R7 + 0.0118*G7 + 0.9882*B7

    R7= 1.0440*Re - 0.0440*Ge + 0.0000*Be
    G7= 0.0000*Re + 1.0000*G7 + 0.0000*Be
    B7= 0.0000*Re - 0.0119*Ge + 1.0119*Be

...snip...

[creamyhorror]

Hope you find a good, simple way to handle the RGB conversion. It might be good to come onto the x264 IRC channel, where you can ask for help about implementation issues. irc://irc.freenode.net/#x264

Can I do the following choice:
If height > 480 or width > 720 then it's BT.709 otherwise it's BT.601?

HD only starts at 1280x720, so you probably want to check if width > 1280. I suggest putting in a dropbox/radio button for the user to select if necessary - but use the condition to autoselect the right matrix by default.

edit: going to bed now, good luck

[rallymax]

good idea

[rallymax]

You could consult the x264 Quicktime Codec source code to see how/if it handles RGB input.

Bummer, it only does YUV2 -> YUV420

copy_2vuy_to_planar_YUV420( )

... that would've been handy.

[jagabo]

You have to be able to select between rec601 and rec709. It's not only HD that uses rec709. I've seen DVDs with both. You might need pc601 and pc709 too.

[poisondeathray]

Well look at that progress; r1243 added today supports slices

Multi-slice encoding support
Slicing support is available through three methods (which can be mixed):
--slices sets a number of slices per frame and ensures rectangular slices (required for Blu-ray).Overridden by either of the following options:
--slice-max-mbs sets a maximum number of macroblocks per slice.
--slice-max-size sets a maximum slice size, in bytes (includes NAL overhead).
Implement macroblock re-encoding support to allow highly accurate slice size limitation.Might be useful for other things in the future, too.

[rallymax]

RRR! I just merged it all. oh well guess I'll have to merge it again

[creamyhorror]

Well look at that progress; r1243 added today supports slices

Okay, that beat even my most optimistic estimations. DS really is trying to get it out quick - I wonder if it's so that Lyris can produce a replicated BD from it?

rallymax, did you eventually ask anyone on Doom9 or IRC? I can ask for you about a simple way to do the RGB conversion if you want.

[rallymax]

havn't asked yet. the x264 src is very well commented/layed out so it's not hard to work out how it all works. I was abl to "borrow" from the CLI code heavily so it's not too hard to plug it together.
turns out that the premiere example has functions for color space changes.
spent most of the evening trying to get the dialog box working that replaces the CLI arguments. gave up on that for now since I don't need it for initial test. have just got it to the right color space ready to call the encoder in the libx264 library. got a ways to go but good progress.

[creamyhorror]

Wow, good stuff. The arguments can always come later.

[rallymax]

ok i'll add a button to override after autodectecting based on height. (color space choice)

[rallymax]

ok. help! I can't work out how to change packed YUV into planar YUV 420.
I see from googling that there is some code in AVS but I don't have it.
can someone point me to something?

[rallymax]

or..... is there some way to push packed YUV into x264?

[poisondeathray]

ok. help! I can't work out how to change packed YUV into planar YUV 420.
I see from googling that there is some code in AVS but I don't have it.
can someone point me to something?

Do you mean AVS as in avisynth?

In avisynth , you would normally use
ConvertToYV12()

I'm not a programmer, so I don't know what the underlying equations or rounding values go in to the transformation. The folks in the avisynth subforum at Doom9 would know

[jagabo]

YV12 is pretty simple. For a progressive NxM image: an NxM Y plane, followed by an N/2xM/2 U plane, followed by an N/2xM/2 V plane. I believe AviSynth averages pairs of U and pairs of V values (from YUY2) to generate the less dense YV12 data.

<edit>
Oops, got the order wrong. It's Y then V then U.
</edit>

[rallymax]

as I understand it, the picture is a standard full WxH array of Y values then the next W/2 x H/2 values is a 2x2 subsample of the U then again of the V
is that right? - coz I can do the simple math for that (sum 0,0 0,1 1,0 1,1 /4)

[rallymax]

sum([0,0] [0,1] [1,0] [1,1]) /4)