Premiere Frameserves SD in Rec 709?

[koberulz]

You adjust it to how you want it.

I want it to look the way it did when it aired, but it's hard to tell which that is.

[poisondeathray]

You adjust it to how you want it.

I want it to look the way it did when it aired, but it's hard to tell which that is.

But you're going to get different results depending on how the MJPEG handled by different decoders / software - MJPEG is that inconsistent. And if you inadvertently had a colorspace conversion filter in the capture chain (it happens with BM, depending on how you set it up)... good luck trying to figure it all out. In the future use bars/test patterns before capture

You should have recorded v210 if you wanted color consistency - but you probably didn't have enough HDD space , and you need at least 2xHDD Raid-0 array or SSD's for reliable capture the bandwidth is much higher than the crappy BM MJPEG

[koberulz]

Most TV broadcasts aren't nice enough to include color bars.

I tried one of the uncompressed formats, but yeah, dropped frames.

Could you talk me through that mp4? I don't know what 'intra, crf 12' means, or much of the command line you quoted.

[poisondeathray]

Most TV broadcasts aren't nice enough to include color bars.

You can do it on your end, to test your capture setup

Could you talk me through that mp4? I don't know what 'intra, crf 12' means, or much of the command line you quoted.

Intra means I-frame only. Each frame is individually encoded, and relies only on itself. Conversely, "Long GOP" means temporal compression, also know as "inter" , not "intra". There are P and B frames that rely on data from other frames. Temporal compression improves compression ratio, but makes seek times (navigation, especially when editing) worse. So it's a trade off. Typically for editing intermediates, intra is used, because quick seeking trumps reduction in filesize. Typically people have multiple HDD's when editing and filesize isn't as important, but it's up to you. Dozens of configurable options for x264. For example a 1-2 sec max GOP would probably cut filesize down to about 50-60% with the same quality. But it will "feel" slower on the timeline

CRF is a rough estimate of "quality" . Smaller values result in larger filesizes, higher quality. If that example was too low in quality for you , try 6-10 for example. Or if you wanted smaller filesizes but are ok with lower quality try 12-16 . At "0" it's mathematically lossless, but PP cannot decode it properly. So at "1" it's very very close to lossless and probably good enough even for demanding professional scenarios. The quality would be higher than prores XQ or cineform filmscan2

To be clear, that MP4 range in YUV is full (it's spread out 0-255, same as the output from the decoder). If you wanted it scaled to legal range, you actually just let ffmpeg do it's thing without forcing the range. Since it decodes mjpeg as full range, it will automatically clamp the output. One problem is the clamping occurs in 8bit and you can get additional banding. If you look at a waveform for example, you will see notching. It's a "dumb" clamp where everything gets scaled, not selectively. Everything will be in "legal range" but it's not necessarily ideal because it's done in a "dumb" fashion. The black level will be slightly too elevated in some shots for example. The way I would do it is the 1st way, and make proper adjustments using your scopes and yuv filters. The fast way is the 2nd way without the -vf scale arguments, but it's up to you


Once you are happy with the settings, you can batch encode. e.g. a folder of mjpeg AVI's will be converted to the MP4's . You can set paths or environment variables, but an easy way to batch for beginners is

1) copy & paste the text to a notepad file in the folder of files to be converted. Make the adjustments you want (e.g. change the CRF, or take out the -vf scale=in_range=tvut_range=tv if you wanted it clamped automatically) . Save it.

2) rename the text file from .txt to .bat,

3) place a copy of ffmpeg.exe in the folder

4) double click the .bat file

for %%a in ("*.avi") do ffmpeg -i "%%a" -vf scale=in_range=tv:out_range=tv -c:v libx264 -pix_fmt yuv422p -x264opts tff=1:keyint=1:colorprim=bt709:transfer=bt709:colormatrix=bt709:force-cfr -crf 12 -an "%%~na.mp4"
pause

This is the MP4 with auto scaling; ie. mjpeg decoded at full range, clamped automatically by ffmpeg (just erase the -vf scale=in_range=tvut_range=tv)

[koberulz]

You can do it on your end, to test your capture setup

Test disc into a BD player, run a short capture?

I tried running the batch file, but the Command Processor almost immediately crashes.

[poisondeathray]

You can do it on your end, to test your capture setup

Test disc into a BD player, run a short capture?

Sure, but you ahve to try to emulate what your capture setup would be. For example if you were capturing off a cable box that might be slightly different than bd output . What connections are you using and what software setup. One culprit sometimes is the software inserts a colorspace filter. But even a BD capture with a test disc will provide useful information

I tried running the batch file, but the Command Processor almost immediately crashes.

Is this on your OS drive? If so, you might need admin privileges . Try running the .bat as administrator

Or maybe a copy/paste issue. Download the zip file and unzip it. Do a short test first. Like 2-3 mjpeg AVI's. Maybe even the same small sample you uploaded, or a few copies of the same file

[koberulz]

Sure, but you ahve to try to emulate what your capture setup would be. For example if you were capturing off a cable box that might be slightly different than bd output .

But there's no way to get color bars from the cable box, so...

What connections are you using and what software setup.

Cable box ->HDMI cable -> HDMI splitter -> BlackMagic -> USB3.0 -> Whatever the software is that comes with the BlackMagic.

Is this on your OS drive?

No. My OS drive holds my OS and nothing else. Although, actually, ffmpeg is probably on the OS drive. Not sure where it is, I set it up as a global so I didn't have to worry about it.

3 files is all that needs doing, but they're a couple hours each. It crashes running the sample though. Running as administrator...appears to work, but there's no mp4 file.

[poisondeathray]

post the console output . It should stay open because of the "pause"

[koberulz]

It didn't, it just ran through and closed. Tried yours, and it crashes whether I run as admin or not.

[poisondeathray]

The "pause" should keep it open. I don't know what is going on with your computer. Or maybe a problem with your ffmpeg binary

If you only have 3 videos, a "batch" really isn't necessary

Can you successfully encode just that small sample individually ?

e.g

ffmpeg -i sample.avi -vf scale=in_range=tv:out_range=tv -c:v libx264 -pix_fmt yuv422p -x264opts tff=1:keyint=1:colorprim=bt709:transfer=bt709:colormatrix=bt709:force-cfr -crf 12 -an output.mp4

[koberulz]

Okay, so it doesn't appear it's actually doing anything. When I run the batch file, it has the path of the current folder followed by the exact code from the batch file. New "paragraph", and it lists the current folder followed by just the ffmpeg version, and with "sample" replacing "%%a". And then it just keeps repeating those two entries until it either closes or crashes.

If I manually enter just the ffmpeg portion...it does exactly the same thing. Alternates between the ffmpeg version and the batch version, even though the batch wasn't used.

Batching isn't really necessary for three files, no, but it would be helpful in a push-a-button-and-go-to-bed sense.

[koberulz]

Okay, so it doesn't appear it's actually doing anything. When I run the batch file, it has the path of the current folder followed by the exact code from the batch file. New "paragraph", and it lists the current folder followed by just the ffmpeg version, and with "sample" replacing "%%a". And then it just keeps repeating those two entries until it either closes or crashes.

If I manually enter just the ffmpeg portion...it does exactly the same thing. Alternates between the ffmpeg version and the batch version, even though the batch wasn't used.

Batching isn't really necessary for three files, no, but it would be helpful in a push-a-button-and-go-to-bed sense.

EDIT: Even if I just try

ffmpeg -i sample.avi -vcodec copy -acodec copy output.avi

it runs through the batch version and resulting version of the previous code until it crashes.

[poisondeathray]

I'm not sure if I can rely on your description. If the console box is open, copy/paste the contents

Exactly how did you setup your ffmpeg as "global" ? If you set it up as an environment variable, you can call it from anywhere. Do me a favour , download a recent ffmpeg version and just place a copy in the directory with sample.avi . This way you are certain it's not a ffmpeg or path issue

[koberulz]

If the console box is open, copy/paste the contents

It's visible, but not usable. Greyed out and "not responding". The behaviour persists after a reboot, even when running the basic copy command without having run the batch.

If you set it up as an environment variable, you can call it from anywhere.

That might have been it. I can't remember the exact process but that was the end result.

[poisondeathray]

not responding as in "hanging" ? I think you're having serious computer issues . Maybe a motherboard / PSU issue, or HDD dying issue, maybe overheating. It's probably behind the UT Video issues and even lagarith issues

I would run some hardware stress tests, memory integrity tests

[koberulz]

Downloaded a new ffmpeg, dropped the avi and bat into the bin folder, removed the environment variable, it worked.

Added the environment variable pointing to the new ffmpeg, put the avi back where it was...failed the same way. So the issue is how I've set up the environment variable.

[koberulz]

Downloaded a new ffmpeg, dropped the avi and bat into the bin folder, removed the environment variable, it worked.

Added the environment variable pointing to the new ffmpeg, put the avi back where it was...failed the same way. So the issue is how I've set up the environment variable.


EDIT: Tried plugging my Blu-Ray player in where I've plugged my cable box in, but it just shows a blank screen in the BlackMagic software.

[koberulz]

Burned a five-minute disc from the MJPEG, and the same five-minute disc from the MP4. One in my Blu-Ray player, the other in the PS4. Found the same shot on each, and on my cable box, then flicked between the three.

On contrast, the MJPEG is correct. The black color of the jersey of the player under the basket in my sample is far more accurate.

To get the Blu-Rays:
1. Frameserved through AVS as YUY2.
2. Into an AviSynth script:

AVISource("signpost.avi").AssumeFPS(25,1)
ColorMatrix(mode="rec.601->rec.709", interlaced=true)
ConvertToYV12(interlaced=true)

3. Created .264 file with MeGUI.
4. Authored with DoStudio.

I tried running the same process without the matrix conversion, but it still looks orange. Different, not sure if it's better or worse, definitely still wrong.

EDIT: I'm getting this now when I try to open After Effects:

---------------------------
After Effects
---------------------------
After Effects warning: The following plugins have failed to load.  Please reinstall these plugins:
C:\Program Files\Adobe\Common\Plug-ins\CS6\MediaCore\BMDDeviceControl.prm
C:\Program Files\Adobe\Common\Plug-ins\CS6\MediaCore\BMDExport.prm
C:\Program Files\Adobe\Common\Plug-ins\CS6\MediaCore\BMDRecord.prm
C:\Program Files\Adobe\Common\Plug-ins\CS6\MediaCore\BMDTransmit.prm



---------------------------
OK   
---------------------------

I'm guessing it's related to the failed Black Magic install, and everything still seems to work, but not sure what to do about it.

[poisondeathray]

On contrast, the MJPEG is correct. The black color of the jersey of the player under the basket in my sample is far more accurate.

If you're decoding mjpeg as full range clamped, it's definitely "more" correct, but still not optimal . Black won't be "black" . Check your scopes.

If you're decoding as full range - that is definitely wrong if you just left it like that - but you're supposed to adjust it. If you took full range then adjusted it properly, it will look better for sure. But it's more work.

But if the mjpeg decoder outputs limited range, then it might be correct - check your scopes. But you might have the field swap issue depending on which decoder is used

Don't assume that a broadcast chain never makes mistakes, or 2nd generation capture is always "correct" . Use your scopes and your eyes and make adjustments

[koberulz]

What about the shift towards orange in the red shirts? That's present in both versions.

Attached screenshots of the MJPEG and MP4 with the YC waveform. Black is at 0.3, and the MJPEG comes pretty damn close. The MP4 clips a ton of the signal.

[koberulz]

What about the shift towards orange in the red shirts? That's present in both versions.

Attached screenshots of the MJPEG and MP4 with the YC waveform. Black is at 0.3, and the MJPEG comes pretty damn close. The MP4 clips a ton of the signal.

[poisondeathray]

red shift towards orange is usually a Rec601/709 issue . Might be a CS6 issue . Might be a decoder issue (some mjpeg decoders output as 601)

An AVC MP4 shouldn't clip if it was derived from the mjpeg and was decoded full range and treated as YUV

If you're happy with the mjpeg directly and it doesn't have the field issue in PP, then why not go with that ? If you have to, you can make final adjustments later in avisynth . You can use scopes in avisynth too with histogram (really a Y' waveform) and other scopes . You should be doing that, because a bunch of things can go wrong when you output from PP (as you've seen with frameserver 601)

[poisondeathray]

Remember, when you're looking at the premiere preview, it' s not necessarily representative of what's on the timeline. The preview is in RGB, but the timeline/sequence could be in YUV (should be if you're using it properly). Older versions of PP (I think maybe CS4 or earlier) used to always use 601 for the preview, except for v210 where it used 709. Nowadays, some formats use 601, others use 709. Handling is better and more consistent in CC. But the actual data isn't converted to RGB on a YUV timeline. It's still in YUV. You're just looking at a temporary RGB representation which might or might not be correct

Some formats that are untouched and "perfect" are UYVY for 8bit422, and v210 for 10bit422 . So if you export UYVY directly from the timeline, that's exactly what is on a YUV timeline (but if you used RGB filters, or other manipulations which cause red render bar), all bets are off. When you incur an actual RGB conversion (sequence data has actually been converted), then the preview is accurate.

The end result is the most important. You should be checking at each stage where things are going wrong.

[koberulz]

Just once I'd like to be able to make a single post...even when only one shows up it's because the 'you can't post again yet' thing kicked in.

I'd be rolling with the MJPEGs and happy with it were it not for this damn red shift. But I don't know where it's coming from or how to fix it. I'm using the MP4s for AE, because that does have the field issue, but I'm just creating a matte so it doesn't matter what happens with the colors.

Reds shifting toward orange is why I had the matrix conversion in the AviSynth script for MeGUI in the first place. Not sure why it's cropped up again. Presumably PP is reading the MJPEG as 601? But it's also an issue with the MP4, so it's happening with ffmpeg even with all the specification of color format...

[poisondeathray]

Just once I'd like to be able to make a single post...even when only one shows up it's because the 'you can't post again yet' thing kicked in.

maybe you type too fast ? I think you need to wait 30 sec

I'd be rolling with the MJPEGs and happy with it were it not for this damn red shift. But I don't know where it's coming from or how to fix it. I'm using the MP4s for AE, because that does have the field issue, but I'm just creating a matte so it doesn't matter what happens with the colors.

Reds shifting toward orange is why I had the matrix conversion in the AviSynth script for MeGUI in the first place. Not sure why it's cropped up again. Presumably PP is reading the MJPEG as 601? But it's also an issue with the MP4, so it's happening with ffmpeg even with all the specification of color format...

Can't you just fix the issue after? As long as you have everything consistent within a program you can use colormatrix or whatever to fix it later at or near the very end

For sure, at least in CC, the frameserver shifted the colors . So how are you determining the "shift" ? Is it the PP preview, or looking at frameserver output ? And how are you taking screenshots or controlling the RGB conversion ?

It's difficult to compare notes "on the same page", because of so many inconsistencies with mjpeg handling, which decoder, which PP version handling, where the RGB conversion is occurring and how - so you have to be very specific

[koberulz]

maybe you type too fast ? I think you need to wait 30 sec

No, I mean, my internet is obviously screwy. It keeps submitting posts twice. Most of the time it gets caught by the timer, so only one shows up, but other times there are two copies of the exact same post.

Can't you just fix the issue after? As long as you have everything consistent within a program you can use colormatrix or whatever to fix it later at or near the very end

I've tried converting to 709, and not converting to 709...neither fixes it.

So how are you determining the "shift" ?

Frameserved out of PP into an AviSynth script which went through MeGUI to create a .264 file. Authored a Blu-Ray disc. Popped it into my player, went to a point during the earlier AVI sample. Changed the source to my cable box, found approximately the same point, flicked back and forth. Both the cable box and the BD player run through a receiver, so it's the same input on the TV. Reds were shifted towards orange whether or not I included a 601-709 conversion in the AviSynth script.

[poisondeathray]

Do you have a reference image of the team jersey ? What team is it ?

If I look at your sample directly, 601 makes it more "red" , 709 more "orange" . So if "red" was supposed to be correct, it might supposed to be handled as 601. Difficult to say because your capture chain, and mjpeg handling - there might be other stuff or filters in your capture that had altered the colors for example

[poisondeathray]

Can't you just fix the issue after? As long as you have everything consistent within a program you can use colormatrix or whatever to fix it later at or near the very end

I've tried converting to 709, and not converting to 709...neither fixes it.

You can convert 709 to 601 in YUV too, using colormatrix

ColorMatrix(mode="rec.709->rec.601", interlaced=true, clamp=0)

So lets say everything was mishandled as 709 (hypothetically lets say it was supposed to be 601 for whatever reason), that should fix it if frameserver didn't do anything extra

[koberulz]

Minus the clamp, that's exactly what was in the AviSynth script.

EDIT: Wait, no, it's the reverse. I need to learn to read. What does the 'clamp' do?

[poisondeathray]

What does the 'clamp' do?

for colormatrix, clamp actually "clips" data outside of Y=16-235. Setting it to zero or false (depending on version) disables the clipping. It should be default behaviour IMO.