Adobe Premiere - MainConcept's H.264 vs x264 open source encoder

[poisondeathray]

I believe it's LPCM

For the audio technical stuff, I would ask in the Doom9 audio subforum. There are little intricacies about byte headers and stuff I really really don't understand

EDIT: I can hear your 2nd test video fine, but I'm not familiar with that chart and the significance of the colors etc..

You can make a video of a bellenuit test chart if you want to check levels / color matrices for your export. There's instructions on the site that tell you how to distinguish rec.601 vs. rec.709. Basically you look at the luma of the 601 vs 709 patches , and if they are the same
http://www.belle-nuit.com/testchart.html

I made one here for 1280x720p60 rec.709, but it's static

There was a well known bug (not sure of it's current status) about Adobe (including AE) converting everything to Rec.601 internally. It was documented on the Adobe forums and several others


720p60%20rec709.m2ts

[jagabo]

Isn't the black level way off?

is it?
pls explain.
Premiere gives out BGRA8:8:8:8 in BT601.
I'm yet to find a library that will change BT601 to BT709 so it's the same for now.
is that why the black is "off"

Just looking at the video the image is washed out. The darker grays are rather light (not the black border but on the card), the colors unsaturated. If that's what your source looked like it's ok. Either way, it's not a 601 vs 709 problem. It might be a rec.601/709 vs PC.601/709 problem. Ie, whether Y=16-235 <--> RGB=0-255, or Y=0-255 <--> RGB=0-255.

[rallymax]

You can make a video of a bellenuit test chart if you want to check levels / color matrices for your export. There's instructions on the site that tell you how to distinguish rec.601 vs. rec.709. Basically you look at the luma of the 601 vs 709 patches , and if they are the same
http://www.belle-nuit.com/testchart.html

Awesome. That's the one I was looking for!

[rallymax]

It might be a rec.601/709 vs PC.601/709 problem. Ie, whether Y=16-235 <--> RGB=0-255, or Y=0-255 <--> RGB=0-255.

sorry for throwing you off with they grey border of the macbeth chart. as you've probably already read I found a better chart and I just now have the exact one I'm after.
This post was useful though. - I didn't know what the difference of rec. and pc. were. I'm now assuming that rec. is in YUV and pc. is in RGB.

[rallymax]

There was a well known bug (not sure of it's current status) about Adobe (including AE) converting everything to Rec.601 internally. It was documented on the Adobe forums and several others

that makes sense. the native flow inside Adobe is rec.601 (well pc.601 if indeed "pc." is when it's in RGB).
What I do wonder is - if the source is 709 is information lost?
I saw that all the blacks dissapear if I dropped a 709 into the workflow.
I'll keep an eye on that.

[poisondeathray]

PPCS4 (at least a while back when I tested for myself, and confirmed on several forums) clamps all non native YUV formats (ie. formats it does NOT have a preset for, unlike DV-AVI, XDCAM which it has presets for etc...) , and RGB formats (so this would include .png, .avs import, lagarith , huffyuv) to 16-235. You will see the YC waveform monitor truncated or clamped to 0-100 IRE with no overshoots above 100, but importing the exact same file, but in uncompressed format will allow the overshoots. Importing and exporting uncompressed used to be the only way around this, and then using colormatrix in avisynth to "fix" for Rec.709; not sure of the current status or if the recent updates fixed it yet. There were a whole bunch of bugs or quirks that have been reported to the Adobe team, but it looks like they are concentrating on CS5 more, rather than "fixing" CS4

[rallymax]

doesn't seem to be suffering from color loss...


myoutput.mp4

[poisondeathray]

define "color loss"...

what did you import? the still or a video? premiere could treat it differently depending on a number of factors (or at least it used to), because the still will be RGB, the video (might be YUV)

Your most recent sample is rec.601

to encode rec709 in the sample video, I had to use specify rec709 when converting the still to yv12 to encode to a YUV video (the sample I posted above). You can check the luma of the patches in aviynsth by using videoscope()

[poisondeathray]

This is a bit of a rant, so sorry for hijacking your thread....

This is yet another Adobe PPCS4 bug, and has been confirmed on several forums. It's decoder interprets the AVCHD as interlaced chroma = very bad.

Attached is a zip file of (point resized zoom) a green screen footage shot with an HMC150 in native progressive 24pN AVCHD mode (no pulldown) decoded using Premiere, and using libavcodec; and an example of the "fake AVCHD" rec.709 video I posted earlier. Notice the jaggies on the red/green border of the fan. You can also see this on the lagarith export, look at the green arrowheads

The good news is the Rec709 in, Rec601 out bug seems to be ok now with the latest patches (appears to be fixed)




screenshots.zip

[jagabo]

Pretty sad.

By the way, the levels are wrong in your graphs. The zero bar in the blacker-than-black should be all the way down to the bottom of the graph. And the 255 bar should be all the way at the top of the graph.

[poisondeathray]

Pretty sad.

It certainly is, for expensive retail software. I can confirm these bugs (the interlaced chroma AVCHD decoding, and clamping issues) do not exist with vegas.

They have been made aware of dozens of quirks / bugs but seem ridiculously slow to respond. Most their efforts seem dedicated to the CS5 release. Hopefully all these bugs will be sorted out

By the way, the levels are wrong in your graphs. The zero bar in the blacker-than-black should be all the way down to the bottom of the graph. And the 255 bar should be all the way at the top of the graph.

I did the inital source video with rec.709, not pc709

[jagabo]

By the way, the levels are wrong in your graphs. The zero bar in the blacker-than-black should be all the way down to the bottom of the graph. And the 255 bar should be all the way at the top of the graph.

I did the inital source video with rec.709, not pc709

The point of the belle nuit chart is to have the 255,251,239,231 area whiter-than-white, and the 20,12,4,0 area blacker-than-black. What's in the video file should always have black (not blacker-than-black) at Y=16, and white (not whiter-than-white) at 235. Your video has black at Y~=32, and white at Y~=. It should look like this:



I realize that the point of your post was the interlaced chroma problem, not levels. I just thought I would point it out so others don't get confused.

[poisondeathray]

So what avisynth script did you / would you use to generate a video or that image ?

This is what I had used, but obviously its wrong

ImageSource()
ConvertToYUY2(matrix="Rec709") #or converttoYV12(matrix="Rec709") for the video

[jagabo]

I used:

ImageSource()
ConvertToYUY2(matrix="pc.709")

[poisondeathray]

So I should have used PC.709 ?

I'm getting something different than your screen above, the blacks look crushed with pc.709 compared to yours ? I thought to use Rec.709 so you could distinguish the small overshoots and blacker than black ? i.e 0 RGB is mapped to 16 Y' or RGB [0,255] <-> YUV [16,235] for normal video ?


pc709_vs_rec709.zip

[jagabo]

So I should have used PC.709 ?

Yes.

I'm getting something different than your screen above, the blacks look crushed with pc.709 compared to yours ? I thought to use Rec.709 so you could distinguish the small overshoots and blacker than black ? i.e 0 RGB is mapped to 16 Y' or RGB [0,255] <-> YUV [16,235] for normal video ?

Before posting the screen cap I converted back to RGB with ConvertToRGB(matrix="PC.709"):

ImageSource("Belle Nuit 1280x720.bmp", start=0, end=23, fps=23.976)
ConvertToYUY2(matrix="pc.709")
VideoScope()
ConvertToRGB(matrix="PC.709")

I then used VirtualDub to open the AVS script and make the sample image.

The reason your video has crushed blacks and whites is because the software you used to to prepare the cap (Virtualdub?) used a rec601 matrix to convert to RGB.

[poisondeathray]

Thanks jagabo, I think you explained it to me before in another thread (probably two or three times...), but it didn't sink in then

[rallymax]

This is a bit of a rant, so sorry for hijacking your thread....

This is yet another Adobe PPCS4 bug, and has been confirmed on several forums. It's decoder interprets the AVCHD as interlaced chroma = very bad.

I can better you on this one. If I try to drop a AVCHD movie (like the one posted) into CS3 it crashes in the decoding dll.

Looks like I'll have to use the same libs to make a better importer too
'probably should get the export working first though...

[rallymax]

ok I feel stupid and confused now.
Can someone dumb it down and start from the start about what rec.601, rec.709, pc.601 and pc.709 are and what they are for?
all I know is that 601 is SD and 709 is HD.
I guess also, do I need to know? Currently I'm getting RGBA8888 from Adobe - which presumably would mean that the Adobe importer is used to do YUV->RGB and would thus inject the interlaced chroma bug (yuk!).
But... I do have the option of asking Adobe for YUV in many forms. If one of those forms is treated right by Adobe all the way through then I don't need to do a colorspace change right? - it may just do 709 all the way through. - although this does mean that every filter used MUST be able to work in that color mode (which is highly unlikely). -As soon as you hit a filter than doesn't understand it, Adobe will do the <whatever> mode to RGB and inject the interlaced chroma and clamping mistakes.
Where does that leave this? well I don't know. Perhaps if I change the import engine to use libavcodec for pretty much every file type and pass it to the workflow as properly constructed RGB then everything will be good.

[poisondeathray]

rec vs. pc -

rec is 16-235 (tv levels) ,
RGB [0,255] <-> YUV [16,235]

pc is 0-255 (full range)
RGB [0,255] <-> YUV [0,255]

http://avisynth.org/mediawiki/ConvertBackToYUY2

The AVCHD interlaced chroma issue appears to be from AVCHD decoding of native progressive footage, so if you use .avs import plugin with a proper decoder it's gone. Note most consumer camcorders don't shoot native progressive footage, it's usually telecined (24p in 30i container)

[rallymax]

a long time ago...

Y is unsigned (0 to 255). For rec 601 and 709 black is a Y=16, full white at Y=235. Values outside that range are valid though.

what about the chroma. what is its range for 601 and 709?
If I detect that Adobe is given me 0..255 do I clamp or scale it to make it rec. ?

[rallymax]

never mind... pc709_vs_rec709.zip shows that it's compressed within the range. I'll check for that and fix if necessary.

[poisondeathray]

never mind... pc709_vs_rec709.zip shows that it's compressed within the range. I'll check for that and fix if necessary.

Those don't mean anything. As jagabo pointed out, they were viewed with rec601 matrix.

(whenever you have YUV<=>RGB conversions, a matrix has to be specified, like PC/Rec 601/709)

So the YUV image form video gets converted to RGB when taking a screenshot; and it's appearance will differ depending how you take the screenshot and what matrix you specify

[rallymax]

so the display on the right and bottom aren't indicative of the 0..255 values?

[poisondeathray]

so the display on the right and bottom aren't indicative of the 0..255 values?

Exactly how are you doing this? in avisynth or on the still? What is the input format? YUV video or RGB still ? What is the output format?

[rallymax]

I guess I confused things.
Base don your pc709_vs_rec709.zip pictures I saw that the scopes of each show on the bottom (which I'm assuming is luma info) that the range is 0..255 or 16..235.
Am I understanding that wrong?
My thinking is - if I see 0..255 from Adobe (ie it's in pc. mode not rec.) then I need to rescale it from 0..255 -> 16..235
so... value = (value * 235 / 255) + 16

[poisondeathray]

Yes that's what it means, but you have to use the proper context. The screenshots were taken with Rec601 , so they are a bit off. Compare with jagabo's screenshot which was PC.709 and screenshot taken with PC.709

Remember Adobe treats RGB source differently than YUV sources (or at least it used to). If you import a still (like a .png, .bmp, that is RGB). If you import AVCHD video, DV-AVI, that is YUV video

Sometimes Adobe can stay in YUV the whole time, eg. with DV-AVI . Sometimes it does an internal RGB conversion (and I think it uses Rec601). I think whenever filters like color correction are used, it does RGB conversion (Rec601)

I think it's important to clarify what the import source is, and the export format to keep things straight.

For example, it treats native DV-AVI slightly differently than a lossless YUY2 lagarith encode of that exact same DV-AVI. You can look at the YC waveform in Premiere to confirm this - it clamps the YUY2 lagarith, but not the native DV-AVI

[rallymax]

i see now. I bet if you applied a non DV-AVI capable filter it would show the YC clamped since it will convert to BGRA. (and yes according to the dev manual it's rec.601)

Premiere supports various depths of BGRA, VUYA (uncompressed YUV with alpha), and ARGB
(After Effects’ native format). It supports compressed NTSC DV25, compressed PAL DV25, V210,
720p and 1080i DV100, 4:2:2 YUY2, 4:2:2 UYVY, and 4:2:0 YUV MPEG2. All pixel formats use
the ITU-R Recommendation BT.601 color space unless noted.

[poisondeathray]

The problem comes, when you deal with HD sources & projects which are usually BT.709. Although I didn't use any filters, that test earlier on the YV12 Rec.709 video passed out as Rec.709 when encoded to lagarith RGB, not Rec.601 (albeit with the interlaced chroma bug) . The old behaviour was that it converted to Rec.601

Too bad jagabo doesn't use PP , he could help sort this out quickly. I can only provide observations

[rallymax]

Here's the table of what Adobe can natively handle...
It is mandated that the top one must be supported by every plugin
It seems that if you want ot work in 709 then the source must be one of the 709 modes below AND there has to be a 709 workflow option in the \Plug-ins\en_US\Editing Modes\Adobe Editing Modes.xml file AND every filter has to support that mode too. If any of those are not satisfied the image will be converted to BGRA rec.601 and if the source is YUV progressive is then exposed to the interlaced chroma bug too.
right?

PrPixelFormat Bits  Format Additional Details
              / Channel 
BGRA_4444_8u    8     BGRA Formerly BGRA32
VUYA_4444_8u    8     VUYA Formerly VUYA32
ARGB_4444_8u    8     ARGB Formerly ARGB32
BGRA_4444_16u  16     BGRA
ARGB_4444_16u  16     ARGB
BGRA_4444_32f   32   BGRA
VUYA_4444_32f   32   VUYA
ARGB_4444_32f   32   ARGB

YUYV_422_8u_601 8   4:2:2 YUY2
YUYV_422_8u_709 8   4:2:2 YUY2 Rec. 709 color space
UYVY_422_8u_601 8   4:2:2 UYVY
UYVY_422_8u_709 8   4:2:2 UYVY Rec. 709 color space

V210_422_10u_601 10 V210
V210_422_10u_709 10 V210 Rec. 709 color space

720pDV100            DV100 720p
1080iDV100           DV100 1080i

YUV_420_MPEG2_FRAME_
PICTURE_PLANAR_8u         4:2:0 YUV For MPEG-2 progressive footage
YUV_420_MPEG2_FIELD_
PICTURE_PLANAR_8u         4:2:0 YUV For MPEG-2 interlaced footage

Raw                             Raw, opaque data, with no rowbytes or height