VideoHelp Forum

Our website is made possible by displaying online advertisements to our visitors.
Please consider supporting us by disabling your ad blocker or buy PlayOn and record Netflix! :)
+ Reply to Thread
Page 1 of 2
1 2 LastLast
Results 1 to 30 of 36
Thread
  1. Member
    Join Date
    Jan 2004
    Location
    Australia
    Search Comp PM
    Yikes ! I've noticed that some HCenc encodes had been a bit stuttery, and have determined something's not right with the order of frames being returned.

    In the attached 350k xvid .AVI - direct download http://rapidshare.com/files/191837705/zzz-problem-frames-34-35.avi - there's a scene change between frames 34 and 35 (base 0 numbering, ie VirtualDub frame numbering). This script will show the undesirable effect only under certain circumstances :-
    Code:
    avisource("G:\DVD\problem\zzz-problem-frames-34-35.avi",audio=false)
    AssumeFPS(25)
    1. If you open that .AVS and single-step sequently from frame 0 to frame 40 (strictly, no jumping ahead) you'll notice that frame 34 is "previous scene" and frame 35 is "next scene". If you then back-single-step subsequently over frame 35 and 34 you'll notice the same thing. All OK here !

    2. Exit VirtualDub and open the .AVS again. Click directly past frame 35 without playing or going over it - say click onto frame 42. Then single-step backwards to past frames 36,35,34,33,32. Notice that NOW frame 34 is "next scene" and slightly out of sequence to frame 35 !! You can go back and forth as many times as you like and it stays like that.

    3. If I don't use filters which go back and forth between frames (ie which compare previous, current, next) then all's OK and the output seems fine. However with some combinations of filters (eg Deblock_QED_MT2, convolution3d, LimitedSharpenFaster, setmtmode=2) the bad frame behaviour appears and the "odd" frame appears to make the final output "jerk" in places. Most disconcerting.

    4. If you open the .AVI directly in VirtualDub and go through the same steps as in either 1 or 2 above, the "bad behaviour" appears regardless.

    It happens on 2 PCs, both XP-SP3 fully updated, late ffdshow codecs (one with xvid 1.2.1 codec used instead of ffdshow), MT-0.7 based on avisynth 2.5.7.

    Can anyone else reproduce this behaviour and/or suggest solutions ?

    I'd think it was the PC but like I said, it happens on 2 and one is a fresh fully patched clean XP-sp3 build with very little on it.
    Quote Quote  
  2. It might be the way you cut it (the sample you uploaded)

    This seems to "fix" it for me:

    Open your .avi, checkmark extended options in the dialog box.
    Checkmark re-derive keyframes

    Direct stream copy audio/video => save new avi

    Make new .avs AVISource("newfile.avi")

    I tested only the .avi, no filter scenarios
    Quote Quote  
  3. Member
    Join Date
    Jan 2004
    Location
    Australia
    Search Comp PM
    Thanks I'll try that. It happens all throughout the larger original source, that clip was just one instance.
    Quote Quote  
  4. Member
    Join Date
    Jan 2004
    Location
    Australia
    Search Comp PM
    Edit: warning, no it didn't work, see below.

    Hey, that worked when opening the .avi directly ! Thanks !

    It must be something to do with the positioning of keyframes inside the .avi then ?

    Is there a VirtualDub or ffmpeg commandline to do this - I need it for batch files ? I tried ffmpeg with -vcopy and that didn't fix it.
    Quote Quote  
  5. I'm not sure why, I assume it has to do with the keyframes. What is the source of the .avi ? The errors might have been introduced when making that .avi

    I had a similar problem a long time ago, and that seemed to fix it for my problem

    I don't know how to implement that process in a commandline...so hopefully someone else knows....
    Quote Quote  
  6. VirtualDub needs packed bitstream in order to decode the frames in the correct order. So does AviSynth's AviSource() since it is based on VirtualDub's AVI parser.

    This problem arises from the fact that VFW is based on a one-frame-in-one-frame-out model. When a B frame references a future frame this model breaks. Ie, the B frame can't be decoded until the future frame is decoded. So you get pauses, frames out of order, backwards jumps, etc. Packed bistream fixes this by packaging the B frames and the future frames into what looks like one frame to VFW. The decoder buffers and decodes the entire packed sequence then returns one frame at a time to the editor.

    If you use Mpeg4Modifier to pack the bitstream the problems should go away. Also note there is a frame in the wrong order at the very start of the clip.
    Quote Quote  
  7. Member
    Join Date
    Jan 2004
    Location
    Australia
    Search Comp PM
    Thanks, just tried to download Mpeg4Modifier and it's broken - the zip downloads but the app won't run after I extract it.
    Quote Quote  
  8. Mpeg4Modifier requires Microsoft .Net Framework.
    Quote Quote  
  9. Member
    Join Date
    Jan 2004
    Location
    Australia
    Search Comp PM
    Hm, just tried with
    Code:
    directshowsource("G:\DVD\problem\zzz-problem-frames-34-35.avi",audio=false)
    AssumeFPS(25)
    No VFW, I'd thought.

    Per post 1 test #1, it looks OK.

    Per post 1 test #2, Click directly past frame 35 without playing or going over it - say click onto frame 42. Then single-step backwards to past frames 36,35,34,33,32. Notice that NOW frame 34 is completely black for the duration of the vdub edit !! You can go back and forth as many times as you like and it stays like that.

    Is there something more fundamental going on ?

    OK, I haven't .net so can't try the mpeg4modifier packed bitstream fix, although I do know the vdub fix (re-derive keyframes and direct stream copy) works.

    I wonder Does the vdub "re-derive keyframes and direct stream copy" also re-pack into "packed bitstream" or just re-calculate the index ?
    Quote Quote  
  10. I think all your problems stem from the same issue -- the difficulty of finding the right frame when frames are encoded out of order.

    You can use GSpot's Visual GOP Structure viewer to see the GOP structure.

    VirtualDub's rederive keyframe option seems to have removed the keyframe at frame 34 from the index. If you use Shift+Arrow you will see that it no longer stops at frame 34. The frame is still a keyframe of course, it's just no longer in the keyframe index. It does not convert unpacked to packed bitstream.

    Here's the sample clip after packing with Mpeg4Modifier:

    packed.avi
    Quote Quote  
  11. Going with jagabo's VFW explanation, and halsboss's no VFW experiment, the problem with DirectShowSource() is that it's not frame accurate and will depend on your filter chain/setup on your PC. For example, I didn't get the black frame insertion, instead I get a repeated frame on 34, 35.

    DSS2("zzz-problem-frames-34-35.avi")

    Using DirectShowSource2 seems to work for me, the transition is at 34, but no jerkiness like when using AVISource(), and it appears to be in order without repeated frames. I'm using ffdshow libavcodec for the decoder r2639, and graphstudio shows I'm using the regular windows avi splitter quartz.dll
    Quote Quote  
  12. Originally Posted by poisondeathray
    Using DirectShowSource2...
    What's that?
    Quote Quote  
  13. Originally Posted by jagabo
    Originally Posted by poisondeathray
    Using DirectShowSource2...
    What's that?
    It's a frame accurate directshow source filter to replace the original DirectShowSource()

    It comes in the haali media splitter package as "avss.dll" , without documentation (just put it into the avisynth/plugins folder)

    It also automatically converts vfr to cfr

    DSS2()

    more info (site is down atm)
    http://forum.doom9.org/showthread.php?t=134275
    Quote Quote  
  14. FFMpegSource("zzz-problem-frames-34-35.avi")

    also seems to work as DSS2 - i.e. no dup frames or stuttering and transition at #34.

    (The FFMpegSource I'm using is the 1st version, there is a FFMS2 now, but both are frame accurate)

    Can you guys replicate these results?
    Quote Quote  
  15. Thanks for the DSS2() tip.

    Here's a version of the video with 15 frame GOPS, frame numbers stamped on each frame, unpacked:

    unpacked.avi

    Here's the same video after packing with Mpeg4Modifier:

    repacked.avi

    With frame numbers on the frames it's very obvious that the unpacked file often returns the wrong frame.
    Quote Quote  
  16. Nice demonstration jagabo

    VFW sux
    Quote Quote  
  17. Member
    Join Date
    Jan 2004
    Location
    Australia
    Search Comp PM
    Wow !

    It's disturbing seeing frame 37 pop up out of sequence when back-single-stepping through the unpacked video !!

    So, as an ongoing "fix" I could (edit: refer to posts below this one, for the final actual solution)
    1. process such streams with VirtualDub or Mpeg4Modifier to create another interim "fixed" file to process [at a cpu/elapsed-time/disk-space cost]
    2. Use DirectShowSource2(...) on the "bad" file [dss2 comes in the haali media splitter package as "avss.dll" , without documentation (just put it into the avisynth/plugins folder) - I don't have haali installed]
    3. Use FFMpegSource(...) or FFMpegSource2(...) [I'll need to locate where these are, but I do use ffdshow and ffmpeg]

    Thankyou VERY much for your insight, would never have resolved it without you.

    I wonder how many people are coming across these without knowing it ? And how these things can be produced in the 1st place. Oh well.
    Quote Quote  
  18. Originally Posted by halsboss
    And how these things can be produced in the 1st place. Oh well.
    Just save with Xvid, with B frames, and without packed bitstream. Some set top boxes have problems with Xvid, multiple B frames, and packed bitstream. So some people don't use packed bitstream.
    Quote Quote  
  19. Member
    Join Date
    Jan 2004
    Location
    Australia
    Search Comp PM
    Have asked a query on how to use the new ffmpegsource over at http://forum.doom9.org/showthread.php?p=1244222

    Interestingly it works "after a fashion". If you open the .avs in Virtualdub
    Code:
    loadplugin("C:\Program Files\AviSynth 2.5\plugins-zzz\FFMS2.dll")
    FFVideoSource("G:\DVD\problem\zzz-problem-frames-34-35.avi")
    AssumeFPS(25)
    and single-step through it all the way past frames 32,33,34,35,36,37 then you'll see the "new scene" begin on frame 33 for the duration of that session.

    If you close Virtualdub and open the same .avs again and then click directly on frame 42 and single-step backwards you'll see the "new scene" begin on frame 34 for the duration of that session.

    Hmm.
    Quote Quote  
  20. Did you verify there were no duplicate and skipped frames? Use the unpacked.avi file I uploaded, note the frame numbers, compare to VirtualDub's frame numbers. The file has a GOP size of 15 so the problem shows up more often in the clip.

    Are you guys having trouble with VirtualDub's Rederive Keyframes option? I've tried a half dozen 22 to 90 minute AVI files and it has crashed on every one of them.

    I would use Mpeg4Modifier() to pack the file. The problems will go away.
    Quote Quote  
  21. halsboss - I am using ffmpegsource1, you are using v2. I tested it a dozen times, reopening the .avs files with vdub single stepping and skipping and it's consistently the same with no dupes, and in order - both with FFMpegSource() and DSS2()

    jagabo - I only used the vdub rederive keyframes on the short sample. I have used it in the past on a similar problem a long time ago , which is why I brought it up here. I haven't tried it on a full length movie

    EDIT: I just tried FFMS2, and it acts exactly as you described, ie. differently than DSS2() or FFMpegSource()

    DSS2() and FFMpegSource() consistently show the switch at 34, regardless of single stepping or skipping first etc...
    Quote Quote  
  22. Member
    Join Date
    Jan 2004
    Location
    Australia
    Search Comp PM
    Originally Posted by jagabo
    Did you verify there were no duplicate and skipped frames? Use the unpacked.avi file I uploaded
    I only found as I posted all seemed OK. Apparently the FFMS2 result seems to be one "skip" when moving sequentially forward , but not when jump-ahead-and-singlestep-backward ? If I have that right. I tried only on the original on the basis that's the source I need fixed.

    Originally Posted by jagabo
    Are you guys having trouble with VirtualDub's Rederive Keyframes option? I've tried a half dozen 22 to 90 minute AVI files and it has crashed on every one of them.
    Oh, I only tried on the short sample, will try on the original source and let you know. Thanks.

    Originally Posted by jagabo
    I would use Mpeg4Modifier() to pack the file. The problems will go away.
    Seems like the answer, although being a dinosaur I have always been loath to install the .Net framework and have got away with it until now.

    Originally Posted by poisondeathray
    halsboss - I am using ffmpegsource1, you are using v2. I tested it a dozen times, reopening the .avs files with vdub single stepping and skipping and it's consistently the same with no dupes, and in order - both with FFMpegSource() and DSS2()
    Wow, your experience is different to mine. Freshly installed XP-sp3 all fully latest patched and ffdshow jan2009. Will re-boot and try again, but I did see what I posted re latest FFMS2 in terms of forward-skipping yielded different result than skip-to-42-firstoff-then-single-step-backward (tried it 3 times, couldn't believe it). ffmpegsource's thread says V1 is "unsupported" and V2 is built with later ffmpeg libraries and can handle more formats "better". As you said earlier, the DSS2 is also "undocumented"... What's your FF script (did you index separately or force it to?)

    Thanks again !!!
    Quote Quote  
  23. I have no idea why the behaviour is different from FFMpegSource v1 and v2, but the pattern is repeatable.

    Give DSS2 or FFMpegSource() a try and see if it works on your system

    For FFMpegSource(), you need FFMpegSource.dll in the plugins directory:
    Code:
    FFMpegSource("zzz-problem-frames-34-35.avi")
    For DSS2, you need avss.dll in the plugins directory (and haali media splitter installed I think)
    Code:
    DSS2("zzz-problem-frames-34-35.avi")
    I didn't need to load the plugin for either. Neither load audio, so you have to do it separately and use AudioDub(). The other difference is I didn't use AssumeFPS()
    Quote Quote  
  24. Member
    Join Date
    Jan 2004
    Location
    Australia
    Search Comp PM
    Originally Posted by jagabo
    Are you guys having trouble with VirtualDub's Rederive Keyframes option? I've tried a half dozen 22 to 90 minute AVI files and it has crashed on every one of them.
    Well, trying it out on a larger source, VirtualDub V1.8.8 's "Rederive Keyframes", it's effectively unusable. For a large source of 85726 frames with the above clip at frame 15388/15389 ...

    For the Virtualdub "open" with option set to "re-derive keyframes", the process "Rekeying video stream" takes a long time (minutes) but completes OK. The subsequent "saving" with direct-stream-copy only took about 40 secs.

    original .avi file steping forward
    old-scene at 15388
    new-scene ar 15389
    original .avi file stepping back from 15400
    old-scene at 15387
    new-scene ar 15388

    rekeyed .avi file stepping forward
    old-scene at 15388
    new-scene ar 15389

    rekeyed .avi file stepping backward from 15400 .. stepping backward takes a LONG time on each and every frame as every single-step says "preloading frame..." and runs through a lot of numbers from 0 to the frame number in question. It's unusable.
    Quote Quote  
  25. Member
    Join Date
    Jan 2004
    Location
    Australia
    Search Comp PM
    Finally, something gives a consistent result, right or wrong FFMpegSource V2.00beta4 from http://forum.doom9.org/showthread.php?t=127037 is consistent and usable (and is a "latest" current version BTW, with all the ffmpeg-based compatibility/features that implies).
    Code:
    loadplugin("C:\Program Files\AviSynth 2.5\plugins-zzz\FFMS2.dll")
    FFIndex("G:\DVD\problem\problem-full.avi", indexmask=7, dumpmask=0, overwrite=true)
    FFVideoSource("G:\DVD\problem\problem-full.avi")
    AssumeFPS(25)
    The FFIndex(..., overwrite=true) forces re-creation of a full index with the same filename/extension plus appended extension ".ffindex". In this case a 500Mb xvid .avi results in only about an 8Mb ".ffindex" file. The FFVideoSource(...) seems to detect the ".ffindex" and use it. It takes about a minute for the FFIndex(...) to do it's thing which is acceptable in the scheme of things to get a reliable source for avisynth to use.

    I guess the only thing to be aware of is a 2-pass HCEnc may force re-creation of a new index. If you use HCEnc, I guess the "lossless" function is handy - it's better than great for slow scripts in any case !

    A HUGE thankyou to poisondeathray and jagabo without whose direction a usable solution would never have arisen.
    Quote Quote  
  26. Originally Posted by halsboss
    Originally Posted by jagabo
    Are you guys having trouble with VirtualDub's Rederive Keyframes option? I've tried a half dozen 22 to 90 minute AVI files and it has crashed on every one of them.
    Well, trying it out on a larger source, VirtualDub V1.8.8 's "Rederive Keyframes"... stepping backward takes a LONG time on each and every frame as every single-step says "preloading frame..." and runs through a lot of numbers from 0 to the frame number in question. It's unusable.
    I suspected this would happen. Because only the very first keyframe is listed in the output file! All the other keyframes are left out of the index. I tried using DivFix afterward. It rebuilt the keyframe index but the resulting file displays the same problems as the original.
    Quote Quote  
  27. Member
    Join Date
    Jan 2004
    Location
    Australia
    Search Comp PM
    Ah. Good pick, then. Like you I noticed keyframe stepping went haywire after the Vdub thing. I also tried the v2 of DivFix, before all this and on the original source, with no change in result either.

    The quickest and least disk space option, assuming the user wants to do avisynth processing on the .AVI, is to use FFMpegSource V2 per the post above. It seems you can even get it to do postprocessing with the pp option to tidy it up for you, eg
    Code:
    FFVideoSource("C:\folder\src.avi",pp="vb/hb/dr") # for horiz/vert deblocking and deringing
    Edit: I got odd results with deblocking and deringing like that, not sure if the problem was me or it thought ...
    Quote Quote  
  28. Member
    Join Date
    Jan 2004
    Location
    Australia
    Search Comp PM
    Well, it was too good to be true. It did something all funny to some frames. It DID say it was a beta.

    Refer clipped source 200k http://rapidshare.com/files/193348627/clipped-original.avi and funny output from HC (recompressed to avi) 130k http://rapidshare.com/files/193346274/clipped-destroyed.avi
    Quote Quote  
  29. Both your links point to the same file.
    Quote Quote  
  30. Member
    Join Date
    Jan 2004
    Location
    Australia
    Search Comp PM
    corrected ink. a gentle thought, if you want to participate or sit back and watch, the doom9 thread is probably the place to go now as the developer is over there. Everyone here has been fantastic, so thankyou.

    mpeg4modifier might be the way to go now for the time being (cost a more cpu and disk to unpack and use that unpacked avi but seems like it'd work). Can it do commandline things ?


    edit: Useful links:
    http://forum.digital-digest.com/showpost.php?p=174145&postcount=11
    http://itsjustonesandzeros.blogspot.com/2007/01/what-is-packed-bitstream.html
    Quote Quote  



Similar Threads