help ! Frames being returned out of order

[halsboss]

Yikes ! I've noticed that some HCenc encodes had been a bit stuttery, and have determined something's not right with the order of frames being returned.

In the attached 350k xvid .AVI - direct download http://rapidshare.com/files/191837705/zzz-problem-frames-34-35.avi - there's a scene change between frames 34 and 35 (base 0 numbering, ie VirtualDub frame numbering). This script will show the undesirable effect only under certain circumstances :-

avisource("G:\DVD\problem\zzz-problem-frames-34-35.avi",audio=false)
AssumeFPS(25)

1. If you open that .AVS and single-step sequently from frame 0 to frame 40 (strictly, no jumping ahead) you'll notice that frame 34 is "previous scene" and frame 35 is "next scene". If you then back-single-step subsequently over frame 35 and 34 you'll notice the same thing. All OK here !

2. Exit VirtualDub and open the .AVS again. Click directly past frame 35 without playing or going over it - say click onto frame 42. Then single-step backwards to past frames 36,35,34,33,32. Notice that NOW frame 34 is "next scene" and slightly out of sequence to frame 35 !! You can go back and forth as many times as you like and it stays like that.

3. If I don't use filters which go back and forth between frames (ie which compare previous, current, next) then all's OK and the output seems fine. However with some combinations of filters (eg Deblock_QED_MT2, convolution3d, LimitedSharpenFaster, setmtmode=2) the bad frame behaviour appears and the "odd" frame appears to make the final output "jerk" in places. Most disconcerting.

4. If you open the .AVI directly in VirtualDub and go through the same steps as in either 1 or 2 above, the "bad behaviour" appears regardless.

It happens on 2 PCs, both XP-SP3 fully updated, late ffdshow codecs (one with xvid 1.2.1 codec used instead of ffdshow), MT-0.7 based on avisynth 2.5.7.

Can anyone else reproduce this behaviour and/or suggest solutions ?

I'd think it was the PC but like I said, it happens on 2 and one is a fresh fully patched clean XP-sp3 build with very little on it.

[poisondeathray]

It might be the way you cut it (the sample you uploaded)

This seems to "fix" it for me:

Open your .avi, checkmark extended options in the dialog box.
Checkmark re-derive keyframes

Direct stream copy audio/video => save new avi

Make new .avs AVISource("newfile.avi")

I tested only the .avi, no filter scenarios

[halsboss]

Thanks I'll try that. It happens all throughout the larger original source, that clip was just one instance.

[halsboss]

Edit: warning, no it didn't work, see below.

Hey, that worked when opening the .avi directly ! Thanks !

It must be something to do with the positioning of keyframes inside the .avi then ?

Is there a VirtualDub or ffmpeg commandline to do this - I need it for batch files ? I tried ffmpeg with -vcopy and that didn't fix it.

[poisondeathray]

I'm not sure why, I assume it has to do with the keyframes. What is the source of the .avi ? The errors might have been introduced when making that .avi

I had a similar problem a long time ago, and that seemed to fix it for my problem

I don't know how to implement that process in a commandline...so hopefully someone else knows....

[jagabo]

VirtualDub needs packed bitstream in order to decode the frames in the correct order. So does AviSynth's AviSource() since it is based on VirtualDub's AVI parser.

This problem arises from the fact that VFW is based on a one-frame-in-one-frame-out model. When a B frame references a future frame this model breaks. Ie, the B frame can't be decoded until the future frame is decoded. So you get pauses, frames out of order, backwards jumps, etc. Packed bistream fixes this by packaging the B frames and the future frames into what looks like one frame to VFW. The decoder buffers and decodes the entire packed sequence then returns one frame at a time to the editor.

If you use Mpeg4Modifier to pack the bitstream the problems should go away. Also note there is a frame in the wrong order at the very start of the clip.

[halsboss]

Thanks, just tried to download Mpeg4Modifier and it's broken - the zip downloads but the app won't run after I extract it.

[jagabo]

Mpeg4Modifier requires Microsoft .Net Framework.

[halsboss]

Hm, just tried with

directshowsource("G:\DVD\problem\zzz-problem-frames-34-35.avi",audio=false)
AssumeFPS(25)

No VFW, I'd thought.

Per post 1 test #1, it looks OK.

Per post 1 test #2, Click directly past frame 35 without playing or going over it - say click onto frame 42. Then single-step backwards to past frames 36,35,34,33,32. Notice that NOW frame 34 is completely black for the duration of the vdub edit !! You can go back and forth as many times as you like and it stays like that.

Is there something more fundamental going on ?

OK, I haven't .net so can't try the mpeg4modifier packed bitstream fix, although I do know the vdub fix (re-derive keyframes and direct stream copy) works.

I wonder Does the vdub "re-derive keyframes and direct stream copy" also re-pack into "packed bitstream" or just re-calculate the index ?

[jagabo]

I think all your problems stem from the same issue -- the difficulty of finding the right frame when frames are encoded out of order.

You can use GSpot's Visual GOP Structure viewer to see the GOP structure.

VirtualDub's rederive keyframe option seems to have removed the keyframe at frame 34 from the index. If you use Shift+Arrow you will see that it no longer stops at frame 34. The frame is still a keyframe of course, it's just no longer in the keyframe index. It does not convert unpacked to packed bitstream.

Here's the sample clip after packing with Mpeg4Modifier:

packed.avi

[poisondeathray]

Going with jagabo's VFW explanation, and halsboss's no VFW experiment, the problem with DirectShowSource() is that it's not frame accurate and will depend on your filter chain/setup on your PC. For example, I didn't get the black frame insertion, instead I get a repeated frame on 34, 35.

DSS2("zzz-problem-frames-34-35.avi")

Using DirectShowSource2 seems to work for me, the transition is at 34, but no jerkiness like when using AVISource(), and it appears to be in order without repeated frames. I'm using ffdshow libavcodec for the decoder r2639, and graphstudio shows I'm using the regular windows avi splitter quartz.dll

[jagabo]

Using DirectShowSource2...

What's that?

[poisondeathray]

Using DirectShowSource2...

What's that?

It's a frame accurate directshow source filter to replace the original DirectShowSource()

It comes in the haali media splitter package as "avss.dll" , without documentation (just put it into the avisynth/plugins folder)

It also automatically converts vfr to cfr

DSS2()

more info (site is down atm)
http://forum.doom9.org/showthread.php?t=134275

[poisondeathray]

FFMpegSource("zzz-problem-frames-34-35.avi")

also seems to work as DSS2 - i.e. no dup frames or stuttering and transition at #34.

(The FFMpegSource I'm using is the 1st version, there is a FFMS2 now, but both are frame accurate)

Can you guys replicate these results?

[jagabo]

Thanks for the DSS2() tip.

Here's a version of the video with 15 frame GOPS, frame numbers stamped on each frame, unpacked:

unpacked.avi

Here's the same video after packing with Mpeg4Modifier:

repacked.avi

With frame numbers on the frames it's very obvious that the unpacked file often returns the wrong frame.

[poisondeathray]

Nice demonstration jagabo

VFW sux

[halsboss]

Wow !

It's disturbing seeing frame 37 pop up out of sequence when back-single-stepping through the unpacked video !!

So, as an ongoing "fix" I could (edit: refer to posts below this one, for the final actual solution)
1. process such streams with VirtualDub or Mpeg4Modifier to create another interim "fixed" file to process [at a cpu/elapsed-time/disk-space cost]
2. Use DirectShowSource2(...) on the "bad" file [dss2 comes in the haali media splitter package as "avss.dll" , without documentation (just put it into the avisynth/plugins folder) - I don't have haali installed]
3. Use FFMpegSource(...) or FFMpegSource2(...) [I'll need to locate where these are, but I do use ffdshow and ffmpeg]

Thankyou VERY much for your insight, would never have resolved it without you.

I wonder how many people are coming across these without knowing it ? And how these things can be produced in the 1st place. Oh well.

[jagabo]

And how these things can be produced in the 1st place. Oh well.

Just save with Xvid, with B frames, and without packed bitstream. Some set top boxes have problems with Xvid, multiple B frames, and packed bitstream. So some people don't use packed bitstream.

[halsboss]

Have asked a query on how to use the new ffmpegsource over at http://forum.doom9.org/showthread.php?p=1244222

Interestingly it works "after a fashion". If you open the .avs in Virtualdub

loadplugin("C:\Program Files\AviSynth 2.5\plugins-zzz\FFMS2.dll")
FFVideoSource("G:\DVD\problem\zzz-problem-frames-34-35.avi")
AssumeFPS(25)

and single-step through it all the way past frames 32,33,34,35,36,37 then you'll see the "new scene" begin on frame 33 for the duration of that session.

If you close Virtualdub and open the same .avs again and then click directly on frame 42 and single-step backwards you'll see the "new scene" begin on frame 34 for the duration of that session.

Hmm.

[jagabo]

Did you verify there were no duplicate and skipped frames? Use the unpacked.avi file I uploaded, note the frame numbers, compare to VirtualDub's frame numbers. The file has a GOP size of 15 so the problem shows up more often in the clip.

Are you guys having trouble with VirtualDub's Rederive Keyframes option? I've tried a half dozen 22 to 90 minute AVI files and it has crashed on every one of them.

I would use Mpeg4Modifier() to pack the file. The problems will go away.

[poisondeathray]

halsboss - I am using ffmpegsource1, you are using v2. I tested it a dozen times, reopening the .avs files with vdub single stepping and skipping and it's consistently the same with no dupes, and in order - both with FFMpegSource() and DSS2()

jagabo - I only used the vdub rederive keyframes on the short sample. I have used it in the past on a similar problem a long time ago , which is why I brought it up here. I haven't tried it on a full length movie

EDIT: I just tried FFMS2, and it acts exactly as you described, ie. differently than DSS2() or FFMpegSource()

DSS2() and FFMpegSource() consistently show the switch at 34, regardless of single stepping or skipping first etc...

[halsboss]

Did you verify there were no duplicate and skipped frames? Use the unpacked.avi file I uploaded

I only found as I posted all seemed OK. Apparently the FFMS2 result seems to be one "skip" when moving sequentially forward , but not when jump-ahead-and-singlestep-backward ? If I have that right. I tried only on the original on the basis that's the source I need fixed.

Are you guys having trouble with VirtualDub's Rederive Keyframes option? I've tried a half dozen 22 to 90 minute AVI files and it has crashed on every one of them.

Oh, I only tried on the short sample, will try on the original source and let you know. Thanks.

I would use Mpeg4Modifier() to pack the file. The problems will go away.

Seems like the answer, although being a dinosaur I have always been loath to install the .Net framework and have got away with it until now.

halsboss - I am using ffmpegsource1, you are using v2. I tested it a dozen times, reopening the .avs files with vdub single stepping and skipping and it's consistently the same with no dupes, and in order - both with FFMpegSource() and DSS2()

Wow, your experience is different to mine. Freshly installed XP-sp3 all fully latest patched and ffdshow jan2009. Will re-boot and try again, but I did see what I posted re latest FFMS2 in terms of forward-skipping yielded different result than skip-to-42-firstoff-then-single-step-backward (tried it 3 times, couldn't believe it). ffmpegsource's thread says V1 is "unsupported" and V2 is built with later ffmpeg libraries and can handle more formats "better". As you said earlier, the DSS2 is also "undocumented"... What's your FF script (did you index separately or force it to?)

Thanks again !!!

[poisondeathray]

I have no idea why the behaviour is different from FFMpegSource v1 and v2, but the pattern is repeatable.

Give DSS2 or FFMpegSource() a try and see if it works on your system

For FFMpegSource(), you need FFMpegSource.dll in the plugins directory:

FFMpegSource("zzz-problem-frames-34-35.avi")

For DSS2, you need avss.dll in the plugins directory (and haali media splitter installed I think)

DSS2("zzz-problem-frames-34-35.avi")

I didn't need to load the plugin for either. Neither load audio, so you have to do it separately and use AudioDub(). The other difference is I didn't use AssumeFPS()

[halsboss]

Are you guys having trouble with VirtualDub's Rederive Keyframes option? I've tried a half dozen 22 to 90 minute AVI files and it has crashed on every one of them.

Well, trying it out on a larger source, VirtualDub V1.8.8 's "Rederive Keyframes", it's effectively unusable. For a large source of 85726 frames with the above clip at frame 15388/15389 ...

For the Virtualdub "open" with option set to "re-derive keyframes", the process "Rekeying video stream" takes a long time (minutes) but completes OK. The subsequent "saving" with direct-stream-copy only took about 40 secs.

original .avi file steping forward
old-scene at 15388
new-scene ar 15389
original .avi file stepping back from 15400
old-scene at 15387
new-scene ar 15388

rekeyed .avi file stepping forward
old-scene at 15388
new-scene ar 15389

rekeyed .avi file stepping backward from 15400 .. stepping backward takes a LONG time on each and every frame as every single-step says "preloading frame..." and runs through a lot of numbers from 0 to the frame number in question. It's unusable.

[halsboss]

Finally, something gives a consistent result, right or wrong FFMpegSource V2.00beta4 from http://forum.doom9.org/showthread.php?t=127037 is consistent and usable (and is a "latest" current version BTW, with all the ffmpeg-based compatibility/features that implies).

loadplugin("C:\Program Files\AviSynth 2.5\plugins-zzz\FFMS2.dll")
FFIndex("G:\DVD\problem\problem-full.avi", indexmask=7, dumpmask=0, overwrite=true)
FFVideoSource("G:\DVD\problem\problem-full.avi")
AssumeFPS(25)

The FFIndex(..., overwrite=true) forces re-creation of a full index with the same filename/extension plus appended extension ".ffindex". In this case a 500Mb xvid .avi results in only about an 8Mb ".ffindex" file. The FFVideoSource(...) seems to detect the ".ffindex" and use it. It takes about a minute for the FFIndex(...) to do it's thing which is acceptable in the scheme of things to get a reliable source for avisynth to use.

I guess the only thing to be aware of is a 2-pass HCEnc may force re-creation of a new index. If you use HCEnc, I guess the "lossless" function is handy - it's better than great for slow scripts in any case !

A HUGE thankyou to poisondeathray and jagabo without whose direction a usable solution would never have arisen.

[jagabo]

Are you guys having trouble with VirtualDub's Rederive Keyframes option? I've tried a half dozen 22 to 90 minute AVI files and it has crashed on every one of them.

Well, trying it out on a larger source, VirtualDub V1.8.8 's "Rederive Keyframes"... stepping backward takes a LONG time on each and every frame as every single-step says "preloading frame..." and runs through a lot of numbers from 0 to the frame number in question. It's unusable.

I suspected this would happen. Because only the very first keyframe is listed in the output file! All the other keyframes are left out of the index. I tried using DivFix afterward. It rebuilt the keyframe index but the resulting file displays the same problems as the original.

[halsboss]

Ah. Good pick, then. Like you I noticed keyframe stepping went haywire after the Vdub thing. I also tried the v2 of DivFix, before all this and on the original source, with no change in result either.

The quickest and least disk space option, assuming the user wants to do avisynth processing on the .AVI, is to use FFMpegSource V2 per the post above. It seems you can even get it to do postprocessing with the pp option to tidy it up for you, eg

FFVideoSource("C:\folder\src.avi",pp="vb/hb/dr") # for horiz/vert deblocking and deringing

Edit: I got odd results with deblocking and deringing like that, not sure if the problem was me or it thought ...

[halsboss]

Well, it was too good to be true. It did something all funny to some frames. It DID say it was a beta.

Refer clipped source 200k http://rapidshare.com/files/193348627/clipped-original.avi and funny output from HC (recompressed to avi) 130k http://rapidshare.com/files/193346274/clipped-destroyed.avi

[jagabo]

Both your links point to the same file.

[halsboss]

corrected ink. a gentle thought, if you want to participate or sit back and watch, the doom9 thread is probably the place to go now as the developer is over there. Everyone here has been fantastic, so thankyou.

mpeg4modifier might be the way to go now for the time being (cost a more cpu and disk to unpack and use that unpacked avi but seems like it'd work). Can it do commandline things ?


edit: Useful links:
http://forum.digital-digest.com/showpost.php?p=174145&postcount=11
http://itsjustonesandzeros.blogspot.com/2007/01/what-is-packed-bitstream.html