x264 encode messed up

[SameSelf]

I just did an x264 encode using the following:

x264.exe --preset veryslow --level 4.1 --ref 4 --tune grain --crf 12 --vbv-maxrate 40000 --vbv-bufsize 40000 -o "video.264" "signpost.avs"

It usually works flawlessly. However, this time about 2 minutes into the video, it inserted a random frame from farther in the video which I determined after pulling the encode back into my NLE and looking for what was causing the strange flash during viewing. Anyone else seen this sort of behavior?

I am wondering if maybe it is just a consequence of the fact that my machine is not hardened with a Xeon chip/ECC ram and now I have to reencode.

EDIT: The more I think about this, I am starting to think the problem is that DMFS served the wrong frame from PPro for some reason.

[LigH.de]

Indeed, the reason is most probably outside x264, which completely depends on the frames it gets served from the video source. I already experienced shuffled contents in AviSynth scripts, depending on the source loaded there (especially in MT environments or multi-source scripts), but when the output of AviSynth is in correct order, x264 should not be able to mess it up afterwards.

[SameSelf]

Jeepers, as they say. I have been been following this workflow for a while, and this is the first time I have seen any frames get shuffled. I have seen some frames get "garbled" before (equally annoying) but not shuffled. And only a single frame out of ~20,000 was misplaced from what I could see. But it is encouraging on a surreal level that I am not the only one who has experienced this problem. Maybe I am just pushing the chain too far? Or, maybe it is just to be expected that one frame out of ~100,000 frames will get garbled/shuffled/etc.? My full workflow is:

AE > Dynamic Link to PPro > DMFS > Avisynth > x264

So in reality, it is AE that is doing the heavy lifting while PPro just has typical fades and titling.

Anyway, I bit the bullet and kicked off the encode again. If no frames are shuffled this time, then I guess it could be the MT, who knows. But again, I wonder if it is due to the fact that my machine is not hardened with a Xeon and ECC RAM. Fortunately, time is not money for what I am doing. But if it was, I would definitely be in the market for an HP Z800.

[jagabo]

I wonder if it is due to the fact that my machine is not hardened with a Xeon and ECC RAM.

Not likely.

Getting out of order frames is very common with highly compressed sources (MPEG 2, MPEG 4, etc.) where frames are stored out of order to begin with. Many source filters are very bad at locating the exact frame on random seeks. Avoid using DirectShowSource(). When using other source filters be sure to use SetMtMode(5,x) before the source filter so only one instance of the filter is used. When using ffVideoSource() use seekmode=0, threads=1. Etc.

[SameSelf]

Not likely.

Getting out of order frames is very common with highly compressed sources (MPEG 2, MPEG 4, etc.) where frames are stored out of order to begin with. Many source filters are very bad at locating the exact frame on random seeks. Avoid using DirectShowSource(). When using other source filters be sure to use SetMtMode(5,x) before the source filter so only one instance of the filter is used. When using ffVideoSource() use seekmode=0, threads=1. Etc.

Thank you. However, my source footage is actually ProRes HQ. Avisynth (I need to recheck the code) is merely frameserving the signpost.avi file from DMFS to x264 but does a ConvertToYUV to prevent any x264 conversion messages. I haven't configured Avisynth for MT. As much as I would love to avoid using DirectShowSource(), AVISource() does not work. I am guessing because even though the signpost.avi is using the AVI container, the codec is something else?

[poisondeathray]

Probably AE. Always clear the media cache before exporting , especially on older versions.

[SameSelf]

That sounds like great advice pdr. How does one do that exactly? Thanks.

[poisondeathray]

I haven't seen it occur in CC yet (it does a much better job of handling memory & disk usage), but on older versions it definitely occurs sometimes where frames get mixed up from cached renders, even with I-frame sources and image sequences (so nothing to do with long GOP). Always the recommendation was to clear the cache. It's edit=>purge=> memory and disk cache or something like that. You can also go edit=>preferences=>memory and disk handling or something like that and setup the custom settings . There is a disk cache specific to AE, and one common to Adobe products (used for PP too)

The newer version is much better because it remembers what instance and what version under each specific condition is used. This can speed up renders and editing considerably . For example, in the old versions, if you disable an effect or layer it would have to re-render when you enabled it. This new version remembers every instance and combination, either in ram preview or disk cache. I'm probably not describing it very well, but it's light years ahead in terms of memory and disk cache management.

[SameSelf]

Thanks so much. I didn't even know that existed in AE. I am such a newb! I am not a user of CC because I hate SaaS, sorry Adobe, you'll never see any of my dollars again!

Anyway, I re-encoded and no problems this time. Of course, this was after a full reboot but before I knew about the clear cache. In the future, I think I will just do a reboot before kicking off a ten hour encode. Better safe than sorry.

[hello_hello]

I've seen DirectShowSource put what appears to be random frames in odd places before.

[SameSelf]

My problem is that AVISource does not work for some reason. Can anyone please tell me what I need to do to get it working? I really wish I didn't have to use DirectShowSource, but I don't know what else to use. Thanks.

EDIT: Actually what exactly should I use? The master footage is ProRes HQ which is graded, denoised, etc. in AE then dynamically linked in PP then frameserved to Avisynth using DMFS. What are my other choices? Thanks

[poisondeathray]

What is the error message with AVISource() ?

If the problem was AE, it would affect normal renders (not frameserved) . But it sounds like you probably only use frameserver

For some people advanced frameserver works better, and for others none of them work - either one problem or another. The last (mostly) working version frameserver for everyone was CS5

[SameSelf]

Finally got around to inspecting the avs script. Turns out I was using AVISource() after all. Doh! So the problem was not due to DirectShowSource().

However, I got to thinking, based on jagabo's comments above, that maybe I could speed up things a little by inserting some MT lines into the script. It is like special sauce when de-interlacing with QTGMC. And every little bit helps whenever I am de-noising.

After a clean reboot, I tried a test 15 sec clip. About half way through the encode, the frame count in PP started doing the weirdest thing. Advancing a couple of frames then going back a few frames. And it just kept doing this over and over like it was stuck in some loop. I finally just killed it because it looked like it would never finish. Even if it did, who knows what what kind of shuffled mess would have been the result.

So now I am convinced that the problem lies somewhere in Avisynth. DMFS works like a dream for me. I am so grateful for the developer of this nifty little tool. Obviously, MT in Avisynth is a no go.

Don't get me wrong. I am not complaining. In fact, I am amazed that this workflow even works, period. There are more moving parts here than should be legal.

[jagabo]

I use QTGMC() with Avisynth MT all the time.

SetMtMode(5,4)
WhateverSource()
SetMtMode(2)
QTGMC()
etc()

The only time I have problems is when combining QTGMC() with other mvtools based filters like TemporalDegrain() or RemoveSpotsMC(). And that's usually running out of memory and crashing issues, not random frames from out of nowhere appearing in the output.

[poisondeathray]

In the order of probability of the cause of problems, it's Adobe's mixing up cache, DMFS. Avisynth chance of causing this is very very low with AVISource and single treaded mode, and x264 even lower.

DMFS is not very stable in >CS5. For some reason you got it working, I have no idea how. It's very very buggy for most people. (or maybe it's showing it's true colors for you now)

Or you can do what everyone else does - use a lossless or near lossless intermediate. If you see mixed up frames there - it's likely the cache issue since you're using I-frame sources

Yes, MT is a bad bad idea with frameserving from PP . It will end up thrashing and you will increase the chance of errors exponentially and actually be slower.

[SameSelf]

In the order of probability of the cause of problems, it's Adobe's mixing up cache, DMFS. Avisynth chance of causing this is very very low with AVISource and single treaded mode, and x264 even lower.

DMFS is not very stable in >CS5. For some reason you got it working, I have no idea how. It's very very buggy for most people. (or maybe it's showing it's true colors for you now)

Or you can do what everyone else does - use a lossless or near lossless intermediate. If you see mixed up frames there - it's likely the cache issue since you're using I-frame sources

Yes, MT is a bad bad idea with frameserving from PP . It will end up thrashing and you will increase the chance of errors exponentially and actually be slower.

That makes two of us not knowing how I got DMFS working. No one told me going in that it was buggy. Ignorance is bliss I suppose until it all comes crashing down. I have entertained inserting a lossless intermediate into the workflow. It's just, me and 300 GB files don't really mix well and I hate having to encode more than once.

Your comments on MT being a problem in PP frameserving (and my experience last night watching it trash around) has me thinking though. I have the MT version of Avisynth installed on my system. I wonder if that is causing problems even if I don't put any MT lines in the script. Nearly all of my footage is progressive these days, so my need for QTGMC is quite low (the only reason I have the MT version installed). I may revert back to a standard build of Avisynth if I start having problems again.

[poisondeathray]

Your comments on MT being a problem in PP frameserving (and my experience last night watching it trash around) has me thinking though. I have the MT version of Avisynth installed on my system. I wonder if that is causing problems even if I don't put any MT lines in the script. Nearly all of my footage is progressive these days, so my need for QTGMC is quite low (the only reason I have the MT version installed). I may revert back to a standard build of Avisynth if I start having problems again.

No - if you don't have any SetMTMode or MT arguments, it's the same as "vanilla" avisynth

[SameSelf]

No - if you don't have any SetMTMode or MT arguments, it's the same as "vanilla" avisynth

Bummer, I was hoping I was onto something.

[jagabo]

No - if you don't have any SetMTMode or MT arguments, it's the same as "vanilla" avisynth

With some scripts I have to have at least SetMtMode() or AviSynth will crash with a "cache" error. mvtools seems to be at the root but I haven't verified that.

I don't think this is related to SameSelf's problem with PP.

[poisondeathray]

No - if you don't have any SetMTMode or MT arguments, it's the same as "vanilla" avisynth

With some scripts I have to have at least SetMtMode() or AviSynth will crash with a "cache" error. mvtools seems to be at the root but I haven't verified that.

I don't think this is related to SameSelf's problem with PP.

For the cache error, it's usually related to masktools2 . Likely you're using masktools2 alpha48. Download masktools2 beta1 which fixes it for avisynth 2.6 and mt . (The "mt_masktools-26.dll" in alpha48 was actually compiled for avisynth 2.5)

But you might have other issues with mvtools also... mvtools2 compiled for the 2.6 branch should work. But some scripts use the old mvtools for avisynth 2.5 . They should be re-written for 2.6 using mvtools2

Basically I'm guessing that you have a bunch of 2.5 plugins and are using 2.6 MT - because I went through the same problems earlier

[SameSelf]

Ran an 11-hour encode last night. It was ~16,000 frames at ~0.4 fps. No out of order frames this time. Phew. Now I can deliver the final encode to our web guy for uploading to YT.

On a side note, I am putting together a plan for authoring a blu-ray. All my source footage is 1920x1080p29.97. But I plan to downsample to 1280x720p29.97 (my Encore doesn't play nice with 1080p30 content). I am guessing that the best way to do this is to frameserve from PP to Avisynth, use Spline36 or a similar resizer, then encode in x264?

IOW, what do you gurus recommend?

[jagabo]

Neither 1920x1080p29.97 nor 1280x720p29.97 is Blu-ray compliant. You have to author as 1920x1080i29.97 or 1280x720p59.94.

[LigH.de]

I would prefer avoiding AfterEffects/PremierePro if there are only editing features you could do with AviSynth alone; "too many cooks spoil the broth"... your workflow appears long, slow, and weak to me. Regarding technical accuracy and speed, I would rather trust AviSynth than most commercial tools (any tool preferring RGB as work color space is not optimal for YUV 4:2:0 based target video formats).

But if there is no alternative for AE, then I would agree to your plan. Just ensure that the source is not interlaced; scaling interlaced material is not trivial, but AviSynth will handle that well if you know what to do. If you are certain that the source is in fact progressive (by content, not just by encoding and header flags), you don't need to worry.
_

P.S.: jagabo seems to be correct, Blu-ray video allows 23.976 or 59.94 fps but not 29.97 fps with 720p resolution... so check whether you have to "bob" (really interlaced content) or "double" (progressive content) while downscaling. 1080 resolution would have to be encoded interlaced, no matter if the content is interlaced; x264 offers a "fake interlaced mode" for Blu-ray compatibility of in-fact 1080p material (it flags MBAFF encoding which might be interlaced in slices per frame, but all slices will eventually be encoded in progressive mode).

[SameSelf]

None of my content is interlaced. The main reason I am targeting 1280x720p29.97 is based on this link

x264 --bitrate XXXXX --preset veryslow --tune film --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 --keyint 30 --open-gop --slices 4 --pulldown double --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 1 -o out.264 input.file

x264 --bitrate XXXXX --preset veryslow --tune film --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 --keyint 30 --open-gop --slices 4 --pulldown double --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 2 -o out.264 input.file

And the whitepaper basically states 30p content need not apply. 24p is fine and so is 59.94p if it is 1280x720. So my plan is to downscale in Avisynth and use the --pulldown double flag.

I use AE because I do a lot of grading to my footage. I don't know how else to assemble a timeline without PP. Never used any other NLE/frameserver.

[poisondeathray]

It won't work for you if you're using Encore to author - it will choke on the --pulldown double . You would need to hardcode duplicates if using Encore

[jagabo]

Apparently x264's --fake-interlaced works for 1080p30 to Blu-ray too. I don't know about AE.

[poisondeathray]

Yes --fake-interlaced works for 1920x1080p29.97

AE works in RGB, but PP can work in YUV. It has YUV filters

[LigH.de]

Doubling the frame rate and number for 720p60 probably won't save you much bitrate compared to 1080i30; even though encoding perfect duplicates with B frames should be quite efficient, practical tests revealed that double rate content still requires considerably more bitrate than single rate content (IIRC, it was tested in the German doom9/Gleitz board once; but try for yourself).

[SameSelf]

I am getting a little befuddled here. Maybe someone can provide a guide for this question:

I have 1920x1080p29.97 ProRes HQ content that I would like to author to blu-ray. My workflow is thus:

AE (grading) - Dynamic Link to PP (editing) - DMFS - Avisynth - x264

My thinking originally was to just insert a Spline36 resize, or similar scaler, in the Avisynth script to make it 1280x720, then use the --pulldown double flag to increase the frame rate to 59.94p.

I am loathe to using the fake interlace flag because my tests in the past show Encore chokes on it pretty hard.

Does anyone have a better workflow they can suggest? If need be, I can post this in a new thread. Thanks.

[poisondeathray]

Encore is incompatible with --pulldown double . --fake interlace works, are you sure it didn't "choke" for another reason? I'm sure I've used it before. They work for sure with other authoring tools like dostudio, scenarist

The other option is to encode duplicates (720p29.97 with doubled frames, to give you 720p59.94) , or to encode PAFF or MBAFF (MBAFF in x264's case) as 1080i59.94 (fields/s) .