Prepare Fast 60 fps Gameplay Footage For Youtube w/ Avisynth

[Kitin]

Hi! I've been looking for ways to make my speedrun footage of rage look better for youtube the last few days when i stumbled upon avisynth. I don't want to waste the extra frames in my footage by just dropping them so i've looked at some ways to "sample" all frames to create motionblur. The best i've come up with so far is:

AVISource("footage.avi", audio=true).AssumeFPS(60.000).converttoYV12

super = MSuper(pel=4, sharp=2, rfilter=4)
backward_vectors = MAnalyse(super, overlap=4, isb = true, search=3)
forward_vectors = MAnalyse(super, overlap=4, isb = false, search=3)
MFlowBlur(super, backward_vectors, forward_vectors, blur=30, prec=1)

I'm not really satisfied with this and i haven't found any other good way of doing it, 60 fps seems to be to low for this kind of fast footage with every method i've tried.

I also want to use something like OLMsmoother (morphological-antialiasing) but for avisynth. This seems to work well since it looks at each frame individually and not at combined frames wich doesen't seem to work well with this kind of footage. I also want to use a good sharpening filter on top of it since it makes textures in the game look much better. A good color correction filter would also be great.


Basically what I want is something that works like OLMsmoother, a good sharpening filter on top of that, a good color correction filter on top of that and finally a good way to make the footage smoother with motion blur on top of all that. Other tips to make my footage look nicer for youtube is appreciated as well.


Link to 60 mb compressed sample footage

[jagabo]

You can use Santaig() or AAA() to antialias.

For simple frame rate reduction you can use Merge(SelectEven(),SelectOdd()). But that looks like double exposures when motions are large. You can use motion interpolation to generate very high frame rates (like 8x) then several Merge(SelectEven(),SelectOdd()) calls to get better motion blur.

[Kitin]

You can use Santaig() or AAA() to antialias.

For simple frame rate reduction you can use Merge(SelectEven(),SelectOdd()). But that looks like double exposures when motions are large. You can use motion interpolation to generate very high frame rates (like 8x) then several Merge(SelectEven(),SelectOdd()) calls to get better motion blur.

Will try Santaig() and AAA().

Do you mean something like this?

AVISource("footage.avi", audio=true).AssumeFPS(60.000).converttoYV12


super = MSuper(pel=2)
backward_vectors = MAnalyse(super, isb = true)
forward_vectors = MAnalyse(super, isb = false)
MFlowFPS(super, backward_vectors, forward_vectors, num=480).TemporalSoften(3,255,255,0,2).selectevery(16,1)

Just TemporalSoften instead of merging evens and odds? In that case I've already tried it and it didn't look very good, to much ghosting and/or artifacts. I have tried InterFrame() instead of MFlowFPS() as well.


Or do you mean some other motion interpolation? I also don't really understand "several Merge(SelectEven(),SelectOdd())", could you provide an example?

[jagabo]

SmoothFPS2(480,1) # 60 to 480
Merge(SelectEven(), SelectOdd()) # 240
Merge(SelectEven(), SelectOdd()) # 120
Merge(SelectEven(), SelectOdd()) # 60 
Merge(SelectEven(), SelectOdd()) #30

You can find SmoothFPS2() in these forums.

Yes, you will see ghosting (multiple exposures) when motions are large. But it's not so obvious in realtime playback. You can reduce the ghosting by going to even higher frame rates and merging more.

If that's too much blur you can selectively decimate:

SmoothFPS2(480,1) # 60 to 480
Merge(SelectEven(), SelectOdd()) # 240
Merge(SelectEven(), SelectOdd()) # 120
SelectEven() # 60
SelectEven() # 30

None of the motion interpolations work well when motions are too large or too complex.

[Kitin]

You can find SmoothFPS2() in these forums.

Yes, you will see ghosting (multiple exposures) when motions are large. But it's not so obvious in realtime playback. You can reduce the ghosting by going to even higher frame rates and merging more.

If that's too much blur you can selectively decimate:

None of the motion interpolations work well when motions are too large or too complex.

Thanks, I can't find SmoothFPS2() plugin in this forum though (if it is a plugin). Read something about svpflow1+2.dll and mvtools.dll and i have booth of those, yet is says "there's no function named SmoothFPS2" when i load the script in VDub.

Edit: Found the code for SmoothFPS2 now, thanks again!

[jagabo]

I don't think the end results will work much better than what you were doing. It's just the limitations of the motion interpolation techniques. When motions are too large or too complex they can't detect them and end up doing nothing or just weighted pixel interpolation. And sometimes you just get weird artifacts from misidentified motions.

[Kitin]

I don't think the end results will work much better than what you were doing. It's just the limitations of the motion interpolation techniques. When motions are too large or too complex they can't detect them and end up doing nothing or just weighted pixel interpolation. And sometimes you just get weird artifacts from misidentified motions.

That's what I expected when nothing worked but I thought that I might've missed something . Pretty much every method i've seen prior uses some ridiculously high framerate for this kind of footage, which isn't possible with this game.

This method allows for more controll, looks better and renders faster than the one i used prior though so that's great.

[jagabo]

Here's an example where motion interpolation really screws up:
https://forum.videohelp.com/threads/352741-Frame-interpolation?p=2215502&viewfull=1#post2215502

[Kitin]

Here's an example where motion interpolation really screws up:
https://forum.videohelp.com/threads/352741-Frame-interpolation?p=2215502&viewfull=1#post2215502

Yeah, I don't believe my footage suffers from that problem though. It's just to much difference between each frame which prevents me from getting really good motion interpolation I guess . I just rendered this with motion blur:

It might not be the motion blur that causes it, but something with 60 fps 1080p lossless source footage shouldn't end up looking like that on youtube.

[jagabo]

The problems were especially visible in the Tron sample but I can see the same type of problem many places in your video. Just increase the frame rate and step through the video frame by frame. You'll see all the problems.

[Kitin]

The problems were especially visible in the Tron sample but I can see the same type of problem many places in your video. Just increase the frame rate and step through the video frame by frame. You'll see all the problems.

Yeah I can se the artifacts on thin/small objects that moves very fast. Should I try some specific settings to try and get rid of them or is just too hard to interpolate the motion of my footage?

[jagabo]

Remember, this type of motion interpolation looks for blocks of pixels that have moved between frames. That works very well for simple panning (except at the edges of the frame) but not so well with zooms, objects moving over a complex background, perspective changes, rotations, etc. You can try playing around with the settings in MSuper and MAnalyze but I don't think you'll get much improvement.

[Kitin]

Remember, this type of motion interpolation looks for blocks of pixels that have moved between frames. That works very well for simple panning (except at the edges of the frame) but not so well with zooms, objects moving over a complex background, perspective changes, rotations, etc. You can try playing around with the settings in MSuper and MAnalyze but I don't think you'll get much improvement.

Ok, I found a sweetspot with little to no artifacts that still has some motionblur left so that will have to do. Thanks for your help!

[manono]

QTGMC has pretty good motion blur ability. From the doc:

Shutter Speed Motion Blur / Frame Rate
Key Points: Can output video at single or double-frame rate. For smoother playback of single-rate output there are settings to add light motion blur to simulate different shutter angles (shutter speeds)

By default QTGMC outputs video at "double-rate", twice the frame rate of the source. This is because there are two separate images (fields) in every frame, which the deinterlacing process restores in full. By setting FPSDivisor to 2, every second frame is dropped and the output frame rate is the same as the source ("single-rate")

Single-rate output may look a little stuttery, depending on how the source was filmed/created. Adding motion blur to each frame can help with this, smoothing the feel of the slower rate playback. This is done by setting ShutterBlur to 1,2 or 3, then specifying the shutter angle that you wish the output to simulate. However, as there may be some motion blur already in the source you also need to specify/estimate the shutter angle used in the source. Shutter angles range from 0 to 360, with 0 being perfectly sharp and 360 being fully blurred from one frame to next (artificial output shutter angles > 360 are supported to a very limited degree). Motion-blur can also be applied to double-rate output, but the effect is less noticeable.

As motion analysis is block-based, motion blur of an object can "bleed" into surrounding, more static areas. This can be reduced using the slower ShutterBlur modes or by increasing the SBlurLimit setting (which is enabled and set to 4 by default). Both repair methods may reduce the level of motion blur a little though. In situations of complex motion the motion analysis can be incorrect, which can lead to inappropriate blurring.

It also gives several examples of how to use it.

[jagabo]

Ok, I found a sweetspot with little to no artifacts that still has some motionblur left so that will have to do. Thanks for your help!

So what were the parameters you finally used? Others may benefit in the future.

[Kitin]

Ok, I found a sweetspot with little to no artifacts that still has some motionblur left so that will have to do. Thanks for your help!

So what were the parameters you finally used? Others may benefit in the future.

SmoothFPS2(1920,1) # 60 to 1920
Merge(SelectEven(), SelectOdd()) # 960
Merge(SelectEven(), SelectOdd()) # 480
SelectEven() # 240
SelectEven() # 120
SelectEven() # 60
SelectEven() # 30

This generate little blur though and it's hardly noticable after youtube compression. The script I posted in the topic might work better for this than merging frames because it creates more blur with smaller "warpings" (there's more warpings but they don't get as stretched out).


Will probably keep testing different stuff so I'll likely come back here with the optimal solution later.

Here's two screenshots when both are turned up pretty high to show what I mean:


Merged from 1920 to 120fps:


Script in topic with blur=60

[jagabo]

Thanks for that update. By the way, SelectEvery(16,0) should get you the same thing as four SelectEven().

[Kitin]

Thanks for that update. By the way, SelectEvery(16,0) should get you the same thing as four SelectEven().

Oh, yeah that makes sense.

Here's one with SmoothFPS2() (without the MBlockFps() part) and MFlowBlur(super, backward_3, forward_3, blur=60, prec=1), more stretched out than with my settings but the warps is smoother and will probably stand out less when the video runs at full speed.

[jagabo]

Yes, that looks better.

[Kitin]

TemporalSoften seems to work way better than merge:


Like this:

SmoothFPS2(3840,1).TemporalSoften(5,255,255,0,2)
SelectEven().TemporalSoften(5,255,255,0,2) # 1920
Selectevery(64,0)

Might look better with just stronger TemporalSoften when it's at 3840fps but I can't find any good documentation on the parameters. Where can I find that? Like the limitations for radius, because when i crank that up to like 30 nothing seems to happen.

[jagabo]

Here's my analysis of what's going on: When the motion detector fails to find motions it temporally blends pixels rather than interpolating motion. And when motions are misidentified, or where the motion of a block is properly identified but the motions of surrounding blocks are not, you get warping artifacts in the interpolated frames. Those frames nearest (temporally) the original frames show the least of these problems, those frame farthest away from the original frames show the most. So when you are selecting a subset of those frames to blend together to produce the final motion blurred frames, what you see depends on which frames are blended together. If you select frames nearest the original frames you get the fewest artifacts. If you select frames farthest away you get more artifacts.

MFlowBlur is essentially blending frames nearest the original frames internally. So it gives fewer artifacts and less blending, but when it fails to find motion vectors you get little motion blur.

[Kitin]

So i've started exprementing with multi-threaded avisynth (downloaded everything on this page essentially http://www.svp-team.com/wiki/Download )

This is the codes i've tried:

SetMemoryMax(2048)


pel=2


SetMTMode(5,4)
AVISource("Footage.avi", audio=true).AssumeFPS(60.000).ConvertToYV12()
ConvertToYV12()
SetMTMode(2)


super=MSuper(pel=2)
finest=pel==1 ? super : MFinest(super)


backward_vec=MAnalyse(super, isb=true, blksize=16, overlap=4)
forward_vec=MAnalyse(super, isb=false, blksize=16, overlap=4)
backward_vec =  MRecalculate(super, backward_vec, blksize=8, overlap=2)
forward_vec  =  MRecalculate(super, forward_vec, blksize=8, overlap=2)


MFlowBlur(super, backward_vec, forward_vec, blur=50, prec=1)


selectevery(2,1)

setmtmode(5)
AVISource("footage.avi", audio=true).AssumeFPS(60.000).converttoYV12
setmtmode(2)
super = MSuper(pel=4, sharp=2, rfilter=4)
backward_vectors = MAnalyse(super, overlap=4, isb = true, search=3)
forward_vectors = MAnalyse(super, overlap=4, isb = false, search=3)
MFlowBlur(super, backward_vectors, forward_vectors, blur=50, prec=1)
SelectEvery(2,1)

Both loads in VDub and looks as they should when i jump in the timeline, and when I start render it renders at >5 fps instead of <1.5 as it did before. A couple of seconds later I get the message: "An exception occurred in module 'KERNELBASE'." and then VDub crashes.


Any ideas of what causes this? I'm guessing it has to do with "SetMTmode". I have an Intel i5 2500k @ 4.5GHz, 8gb ddr3 @ 1600MHz and a gtx 670 slightly factory overclocked if that matters.

Edit: FIXED, just needed to set a lower value for setmaxmemory(), < 1500mb seems to work.

Now I'd appreciate some help with a script that uses the features of SVPflow to produce motion blur, i'm a little lost on the syntax and in which ways I can use it alongside with MVtools2. Would like to try GPU-acceleration.

[manono]

Now I'd appreciate some help with a script that uses the features of SVPflow to produce motion blur...

You don't think it might be at least worth a try using the motion blur feature of QTGMC?

[Kitin]

Now I'd appreciate some help with a script that uses the features of SVPflow to produce motion blur...

You don't think it might be at least worth a try using the motion blur feature of QTGMC?

I've tried it before and it creates weird blocks that's completely clean when the motion is high.



This is with the best settings i've come up with.

[manono]

OK then, but don't you get the same using other motion-blur techniques? And are these artifacts
visible while watching the video, as opposed to going over it frame-by-frame? Using SVP may be
slightly better, but it also produces weird artifacts during high motion.

[Kitin]

OK then, but don't you get the same using other motion-blur techniques? And are these artifacts
visible while watching the video, as opposed to going over it frame-by-frame? Using SVP may be
slightly better, but it also produces weird artifacts during high motion.

No not the same kind of artifacts, I posted som examples above in this thread. It's hard to see the artifacts in full speed but those were 3 consecutive frames with pretty bad artifacts in each so it definitely appears less smooth than the other options in full speed.

[jagabo]

Some SVPFlow examples and problems:
https://forum.videohelp.com/threads/352741-Frame-interpolation

[Kitin]

Some SVPFlow examples and problems:
https://forum.videohelp.com/threads/352741-Frame-interpolation

Thanks, will try once my problems (that i thought was resloved by lowering setmemorymax) is resolved.

There's some weird stuff going on:

setmemorymax(1024)
setmtmode(5,4)
AVISource("Footage.avi", audio=true).AssumeFPS(60.000).converttoYV12
setmtmode(2)
super = MSuper(pel=4, sharp=2, rfilter=4)
backward_vectors = MAnalyse(super, overlap=4, isb = true, search=3)
forward_vectors = MAnalyse(super, overlap=4, isb = false, search=3)
MFlowBlur(super, backward_vectors, forward_vectors, blur=50, prec=1)
SelectEvery(2,1)

These frames are not only weird looking, but they also appeared randomly in the video, completely out of the correct frame sequence order.



Tried this today:

setmemorymax(1300)
setmtmode(3,4)
AVISource("Footage.avi", audio=true).AssumeFPS(60.000).converttoYV12
setmtmode(2)
super = MSuper(pel=4, sharp=2, rfilter=4)
backward_vectors = MAnalyse(super, overlap=4, isb = true, search=3)
forward_vectors = MAnalyse(super, overlap=4, isb = false, search=3)
MFlowBlur(super, backward_vectors, forward_vectors, blur=50, prec=1)
SelectEvery(2,1)

When this happened:
.m4v video file @ 4mb (Warning: Unpleasant/loud sound)


VDub still crashes sometimes when i jump around in the timeline and lowering the memorymax to 512 made it crash upon rendering as well, KERNELBASE.dll error every time. What's wrong here? What excactly do I need in order to use multithreaded Avisynth with MVtools2?

[poisondeathray]

Simple : don't use MT with temporal filters. Often you get get messed up or out of place frames . Some people swear it's ok, but in my experience it's not worth the problems . MT crashes a lot too . MT just ins't implemented very well in avisnyth. Once/If vapoursynth gets developed more, MT should work better

I spent days on different setups and systems trying to get it working reliably and consistently, testing different dll combinations and avisynth builds. It's just not possible with temporal filters and the current state of avisynth.

[jagabo]

Yes, I often have problem with AviSynthMT -- including some of the scripts I was working with with this video. But this morning I just updated some of my filters (svpflow, and mvtools from that package) and I haven't had a crash since. The scripts in post #28 gave no problems. Time will tell...