Jitter in encoded SD MPEG-2 from MBAFF H.264 1080i file - why?

[Killer3737]

As per the title I encoded an HD Source to SD for DVD using AVISynth into CCE SP3 as usual. The input file specs:

Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Codec ID : 27
Duration : 1h 4mn
Bit rate mode : Variable
Bit rate : 28.7 Mbps
Maximum bit rate : 30.8 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : MBAFF
Scan order : Top Field First
Bits/(Pixel*Frame) : 0.554
Stream size : 12.9 GiB (94%)
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

So I've been told before AVCSource is no good for MBAFF and not to use DSS either, so I used FFMS2. This is the rest of the script, I think. I say I think because I experimented with a few and not 100% which I used for the final encode. It was either this or just one with Yadif itself, mode=1, order=1.

Load_Stdcall_Plugin("E:\Video & Audio Editing Programs\MeGUI\tools\yadif\yadif.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\ffms2.dll")
ffmpegsource2("FooFighters2014-09-14 PRO 1.ts")
AssumeFPS(25)
AssumeTFF()
QTGMC("Fast")
Spline36Resize(720,576)
SeparateFields()
SelectEvery(4,0,3)
Weave()
ColorMatrix(mode="Rec.709->Rec.601", clamp=0)
ConvertToYUY2(interlaced=true)

The resulting file is full of jitters every now and again, here is a sample - https://mega.co.nz/#!4MNwHY4L!o2oRgn1AMT06-yZiIw-gqqyq6cuNmh74cKE-6A_1RsI It's most apparent on the panning shots, like the start.

So, any idea what might have caused the problem? How should I tackle this sort of file?

[poisondeathray]

Haven't look at your video, but it's probably ffms2 filter decoding frames out of order.

AVCSource has problems with PAFF, not MBAFF, so it's worth a try

Other options that will work most likely are DGNVTools (if you nave Nvidia card and license - not free) , or deocding it with ffmbc to a lossless intermediate first

[Killer3737]

Ran it through DGAVCIndex and encoded a sample, result:




So I guess it doesn't like that either, at least not with that script combo. I don't have an Nvidia card, no. Haven't used ffmbc before, will take a look.

[poisondeathray]

Yes, DGAVCDec is out of the question there . It does have other problems, not just the documentated PAFF issue

You can use ffv1 with ffmbc as a lossless codec

ffmbc -i "FooFighters2014-09-14 PRO 1.ts" -c:v ffv1 -an output.avi

AviSource("output.avi")
.
.
.
whatever your script is

[Killer3737]

What lossless compression scheme would that be using, how big is the output going to be?

[poisondeathray]

FFV1 is the codec

It's going to be about 5-10x larger on average (depends on content complexity)

[poisondeathray]

If you just want to do a quick test first to see if it works or not (to not waste time if it doesn't work; but I think it should work because these issues are very common with interlace transport streams and this is a verified workaround), and limit it to say the first minute, add -t 00:01:00 to the command line (you mentioned there was a glitch in the beginning)

ffmbc -i "FooFighters2014-09-14 PRO 1.ts" -vcodec ffv1 -t 00:01:00 -an output.avi

The other option in avisynth is L-Smash Source, but in my experience it has problems similar to ffms2 with these types of interlaced AVC transport streams

And for ffmbc, it should be "-vcodec", it doesn't use "-c:v codec" that ffmpeg migrated to, sorry about that

[Killer3737]

edit: posted before your above message, just a moment...

[poisondeathray]

haha it was a "ninja" edit ...., I added the -vcodec commented at the end

About the duration discrepancy, what is the lineage of this recording? What what source it recorded from, what device, how was it connected, was it straight recording or edited in any way (e.g. maybe cut out commercials) ?

[Killer3737]

It's a capture via satellite, I would assume it was trimmed at the start and end before I got it.

Created the avi, 1.05GB for the minute so not too bad. Just encoded it with HCEnc, result looks good to me although there is one issue that I've been seeing in a few of my recent encodes. The following image is from the M2V in Vdub with a deinterlace filter.




I'm referring to the red horizontal lines under his ear, off his shoulder and to the right of his thumb. Is it just the red lighting on stage that causes those lines?

Here is the output file - https://mega.co.nz/#!pEs1CCbY!EI8YqKVh93_grxG8dvd9tPHMxrE_cB38iBhC5CRu_cc

Less noticable in motion but I do see it occasionally.

[poisondeathray]

I'll take a closer look , but I the problem is your method of screnshot taking - vdub doesn't handle interlaced 4:2:0 very well. It upsamples it as progressive .

If you take a screenshot with avisynth (e.g. avspmod, or even through vdub with avs script), what does it look like (interlaced =true flag for the RGB conversion)

mpeg2source()
converttorgb24(interlaced=true)


Looks ok here, normal (not deinterlaced)



Simple bob()

[Killer3737]

Ok, did not know that. Tried that and yep, no problem now. Thanks.

[poisondeathray]

Something that bothers me is your m2v is BFF, but your original re-interlacing script suggest the output should have been TFF. Did you use something different?

Also, earlier you mentioned (and deleted) a comment about a time discrepancy in ffmbc , want to expand on that and post the log ?

[Killer3737]

Is it? I did use that script and mediainfo says it's TFF, are you sure it's BFF? I just quickly ran it through HCEnc which was also set on TFF as far as I know.

The discrepancy in ffmbc was when I loaded the full length clip, without the first minute trim. It said this:

I:\HD\2014\Foo Fighters - 2014-09-14 Invictus Games Closing Concert [HD 1080i]>f
fmbc -i "FooFighters2014-09-14 PRO 1.ts" -vcodec ffv1 -an houtput.avi
FFmbc version 0.7-rc8
Copyright (c) 2008-2013 Baptiste Coudurier and the FFmpeg developers
Input #0, mpegts, from 'FooFighters2014-09-14 PRO 1.ts':
Duration: 00:08:21.36, bitrate: 28659 kb/s
Program 1
Stream #0.0[0x1011](und): Video: h264 (High), yuv420p, 1920x1080i tff [PAR 1
:1 DAR 16:9], 25.00 fps
Stream #0.1[0x1100](eng): Audio: mp2, 48000 Hz, stereo, s16, 256 kb/s
Output #0, avi, to 'houtput.avi':
Metadata:
ISFT: FFmbc 0.7
Stream #0.0(und): Video: ffv1, yuv420p, 1920x1080p [PAR 1:1 DAR 16:9], vbr,
lossless, 25.00 fps
Stream mapping:
Stream #0.0 -> #0.0
Press [q] to stop, [?] for help
frame= 3 fps= 10 q=0.0 size= 2631kB time=00:00:00.12 bitrate=179624.4kbits
frame= 6 fps= 11 q=0.0 size= 5294kB time=00:00:00.24 bitrate=180686.8kbits
frame= 9 fps= 10 q=0.0 size= 8016kB time=00:00:00.36 bitrate=182399.3kbits
frame= 12 fps= 10 q=0.0 size= 10777kB time=00:00:00.48 bitrate=183924.2kbits
frame= 15 fps= 11 q=0.0 size= 13402kB time=00:00:00.60 bitrate=182985.8kbits
frame= 18 fps= 11 q=0.0 size= 16065kB time=00:00:00.72 bitrate=182783.8kbits
frame= 21 fps= 10 q=0.0 size= 18781kB time=00:00:00.84 bitrate=183157.2kbits
frame= 24 fps= 10 q=0.0 size= 21534kB time=00:00:00.96 bitrate=183757.7kbits
frame= 27 fps= 10 q=0.0 size= 24295kB time=00:00:01.08 bitrate=184280.0kbits
frame= 30 fps= 10 q=0.0 size= 27095kB time=00:00:01.20 bitrate=184967.5kbits
frame= 33 fps= 10 q=0.0 size= 29957kB time=00:00:01.32 bitrate=185913.7kbits

Actual duration is 65 minutes.

[Brad]

Is it normal for DGIndex to switch between "Interlaced" and "Progressive" frame type for a clip like this when Preview is used?

I was also going to comment on the BFF issue. Since it's flagged as TFF, VLC and anything else that follows the flag to deinterlace is going to give back-and-forth jerkiness.

[poisondeathray]

hmmm the duration might be problematic when you do the entire thing, not sure. Sometimes if video is edited, the timestamps can get messed up and cause problems (it might be reading the original time from the original longer recording). Or it might be ok, you'd have to try it out

The fallback option of course is to use DirectShowSource(). It should be ok with straight linear encoding, even with QTGMC in the mix.




The content, without a doubt, is BFF in "newnew.m2v". So double check your steps

[Killer3737]

Hmm, that is odd. Script was as follows:

Load_Stdcall_Plugin("E:\Video & Audio Editing Programs\MeGUI\tools\yadif\yadif.dll")
AviSource("output.avi")
Yadif(mode=1, order=1)
LanczosResize(720,576)
ColorMatrix(mode="Rec.709->Rec.601", clamp=0)
SeparateFields()
SelectEvery(4,0,3)
Weave()

As I say, into HCEnc. Except the obvious bitrate/aspect ratio etc what should the settings in there be? On 'Settings 1' I've got it as auto detect and TFF.

[poisondeathray]

You forgot to explicitly add AssumeTFF() before SeparateFields(). Avisynth "assumes" BFF by default (but sometimes the source filter can pass info like field order, but AVISource does not). In the original script, you had an AssumeTFF() before QTGMC, but are using yadif's (order=1) in this recent version. yadif's internal order doesn't carry through to other filters

[Killer3737]

Ah, not sure where that went, I didn't notice obviously. I've got so many scripts around for encoding I must've copied/edited the wrong one. Thanks.

I'll have to see if that timecode thing proves an issue, will report back.

So, does this file look right now? https://mega.co.nz/#!pA9jUQoD!W7fVNIhBx61DzRRqxr_33NJ3hAVatQmFPwD1Vwr9Duo

[poisondeathray]

Yes that last version has the correct (matching) field order

[Killer3737]

Encode with ffmbc went ok, the ETA was still based on 8 minutes and so said 0 to go for ages but it's encoded the whole thing by the looks of it. Only issue is right at the very end, it gave this error:

/s eta=00:00:00.00
[h264 @ 01c3b720] missing picture in access unit
[h264 @ 01c3b720] error fetching slice type, discarding
frame=96270 fps= 9 q=0.0 Lsize=86289840kB time=01:04:10.80 bitrate=183568.7kbit
s/s
video:86288257kB audio:0kB global headers:0kB muxing overhead 0.001835%

Anything to cause concern?

[poisondeathray]

Maybe, but probably not.

This type of error occurs frequently in transmitted transport streams , which often have errors. When you play video, the decoder and video player encounter them too , but you just don't see the log

I would still have a quick look at it, scrub through it and look for problems