Avisynth - What's Wrong with this Script. (FPS Convert/Motion Estimation)

[RAB78]

I was wondering if some of you experts would take a look at a script and tell me what's wrong and how to fix it?

I've got a huge holiday video to edit (over 8 hours of footage) with multiple cameras. (My family films everything. Lol.)

I'm working in PAL but i've got footage shot in 60fps, 30fps and 24fps as well as 25fps.
While serching online for the most effective way to convert to 25fps without dropping frames or blending them. I came across this video... https://www.youtube.com/watch?v=7uaYLtKUgps
I tried the morph method and it worked great on the 60 and 24fps videos but when i tried it on the 30fps video, it just slowed it down but retained the running time which cut off the end.

Here's the script.....

DirectShowSource("J:\Scotland 2014\4K Files\20140801_193207.mp4")
#ConvertToYUY2(matrix="PC.601") #For RGB sources
super = MSuper(pel=1)
#pel=1 can also be pel=2 or pel=4
backward_vec = MAnalyse(super, isb = true)
forward_vec = MAnalyse(super, isb = false)
MFlowFps(super, backward_vec, forward_vec, num=25,den=1, ml=100)
Lanczos4Resize(1920,1080)

I can't understand why it works for 60 and 24 but not 30.
Btw, I'm not that knowlegable on Avisynth. I normally just search for scripts online and try them.

Hope someone can help as i really want to get on with this project.
Many thanks!

[jagabo]

DirectShowSource() probably got the frame rate and count wrong. Try just:

DirectShowSource("J:\Scotland 2014\4K Files\20140801_193207.mp4")
Info()

And see what it says. You're source is also probably interlaced and needs to be deinterlaced before motion interpolating and scaling.

[newpball]

So your sources are 4k and you downscale it to HD?

[hello_hello]

So your sources are 4k and you downscale it to HD?

I guess the OP didn't realise he first needs to explain why he wants to resize a video, seek and obtain your permission, and only then ask for help regarding the actual problem.

[RAB78]

@jagabo
Thanks, I'll try that and see what it says.
The source is progressive so no need to deinterlace.

@everyone
Thanks for taking an interest in my problem.
A little about the source....
The file is from an LG G3 phone. Shot in 4k, I need to resize it for a 25i Bluray edit.
The framrate is 30fps variable.

@hello_hello
I didn't think it was important to explain why I need to resize as framerate is the issue.
(Edit: This was written first thing in the morning and my humor sensors were still asleep. Lol.)
Should I resize before interpolating? Would that speed it up?
I'm frameserving the script to Avidemux for an mp4 output and the conversion is very slow. Processing at 2-7fps.

[newpball]

Should I resize before interpolating?

If you downscale do the interpolation before if you upscale do it after.

If it is slow it means it does do a good job!
You cannot do a fast motion interpolation.

The alternative to using freeware is using Twixtor to do the job, it works with every major NLE.

Make sure when you downscale you take advantage of extra bit depth and croma information if you do additional post processing. Not sure if Avisynth can handle that flawlessly.

[Skiller]

Here is what I would suggest: convert everything to 50 fps in the following way so that everything can be thrown into the same timeline.


60 fps source footage: motion compensated frame rate conversion to 50 fps.

30 fps source footage: pulldown to 50 fps using ChangeFPS(50) (or do it motion compensated, but pulldown is probably better due to the lack of artifacts, and it keeps the more stuttery 30 fps look).

24 fps source footage: speedup to 25 fps then duplicate every frame for 50 fps, or do a pulldown with ChangeFPS(50) (I would prefer a speedup).


For 25i distribution you can easily interlace the final rendered 50 fps timeline to 25i. It's certainly better to do all the work in 50 fps and then do the conversion to 25i as the last step but it's not a must.

[smrpix]

@RAB78, what NLE are you using to combine all this footage? All the decent ones allow you to mix and match sources.

If you want to actually finish this project, do your edit first and then, if it still seems necessary, go back and modify only the pieces you need with the methods suggested in this thread.

[poisondeathray]

The problem is likely the VFR nature of the clip. It's "missing" frames compared to a CFR equivalent version, so the average framerate in avisynth will be less than 30. You can verify with info() as jagabo suggested. And likely it will be out of sync.

To get it working, you should convert it to CFR using convertfps=true

e.g. if it was a base frame rate of 30.0 FPS (Not 29.97)

DirectShowSource("J:\Scotland 2014\4K Files\20140801_193207.mp4", fps=30, convertfps=true)
#ConvertToYUY2(matrix="PC.601") #For RGB sources
super = MSuper(pel=1)
#pel=1 can also be pel=2 or pel=4
backward_vec = MAnalyse(super, isb = true)
forward_vec = MAnalyse(super, isb = false)
MFlowFps(super, backward_vec, forward_vec, num=25,den=1, ml=100)
Lanczos4Resize(1920,1080)


If you are exclusively using a 1920x1080 timeline, you could probably put the resize first, right after DirectShowSource, it will speed up the script

Directshowsource() can sometimes produce problems with frames out of order, or temporal issues - especially when temporal filters are added in. So some people would use the DirectShowSource() line and convert that to a lossless I-frame intermediate first, before applying the interpolation or any temporal filters

[RAB78]

@Jagabo

DirectShowSource() probably got the frame rate and count wrong.

I tried your Info() idea and you were spot on.
It did indeed tell me that DirectShow thought the framerate was 25fps. I guess because of the variable framerate.
I added ,fps=30 to my script and it converted the video properly. Unfortunately, it took 45 mins to encode a 1m30s video. I have 141 video clips to convert so i'll give up with the motion estimation and try the other options given here.

@newpball
Thanks for explaining when i should interpolate when resizing.
I've heard of Twixtor being used for framerate conversion but unfortunately it's too expensive for what i'd use it for.

@Skiller

convert everything to 50 fps in the following way so that everything can be thrown into the same timeline.

What is the reason for converting all to 50 if 25 is my project target? Why not leave it at 25?

30 fps source footage: pulldown to 50 fps using ChangeFPS(50)

I tried ChangeFPS(50) and the resulting 50p video was more juddery than the 30p video. Not very nice to watch.

24 fps source footage: speedup to 25 fps then duplicate every frame for 50 fps, or do a pulldown with ChangeFPS(50) (I would prefer a speedup).

I did motion compensation on both the 60 and 24fps videos as both looked fine to me. Both of which were a constant framerate i might add.
I'm not sure i want to be speeding up footage as sometimes there is more than one camera rolling at the same time and i will be cutting between them. If one camera is running faster than the others, things could get out of sync.
Why do you prefer to convert the video to 50fps after it's already sped up to 25fps PAL?
Thanks for sharing your method by the way. Great to hear peoples workflow.

@smrpix
I'm using Sony Vegas Movie Studio Platinum 12 and although it does accept all different framerates in one timeline, what it does in the render based on your project properties is not to my liking. It will resample the frames to match the project by blending. This causes motion blurring which i don't like when i know the original video is nice and clear during panning. Turning off resampling causes dropped frames which is equally as ugly during panning.
I like to prepare all the files before i start editing. That way, i know all will be good and it's one less thing to worry about.

[newpball]

I've heard of Twixtor being used for framerate conversion but unfortunately it's too expensive for what i'd use it for.

Well family pictures are priceless!

But ok, if you preserve the originals you can always do that much better, cheaper and faster in the future!

[RAB78]

@Poisondeathray
You snuck in there while i was putting together my post. Lol.
Thanks a lot for your input. I'll try your suggestion and get back to you.

Btw, here's what Media Info has to say about the file i'm working with....

General
CompleteName                     : J:\Scotland 2014\4K Files\20140801_193207.mp4
Format                           : MPEG-4
Format_Profile                   : Base Media
CodecID                          : isom
FileSize/String                  : 326 MiB
Duration/String                  : 1mn 30s
OverallBitRate/String            : 30.1 Mbps
Performer                        : LGE
Encoded_Date                     : UTC 2014-08-01 18:33:39
Tagged_Date                      : UTC 2014-08-01 18:33:39

Video
ID/String                        : 1
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format_Profile                   : High@L5.1
Format_Settings_CABAC/String     : Yes
Format_Settings_RefFrames/String : 1 frame
Format_Settings_GOP              : M=1, N=30
CodecID                          : avc1
CodecID/Info                     : Advanced Video Coding
Duration/String                  : 1mn 30s
BitRate/String                   : 30.0 Mbps
Width/String                     : 3 840 pixels
Height/String                    : 2 160 pixels
DisplayAspectRatio/String        : 16:9
FrameRate_Mode/String            : Variable
FrameRate/String                 : 30.000 fps
FrameRate_Minimum/String         : 29.489 fps
FrameRate_Maximum/String         : 30.529 fps
ColorSpace                       : YUV
ChromaSubsampling                : 4:2:0
BitDepth/String                  : 8 bits
ScanType/String                  : Progressive
Bits-(Pixel*Frame)               : 0.121
StreamSize/String                : 324 MiB (99%)
Title                            : VideoHandle
Language/String                  : English
Encoded_Date                     : UTC 2014-08-01 18:33:39
Tagged_Date                      : UTC 2014-08-01 18:33:39
mdhd_Duration                    : 90646

Audio
ID/String                        : 2
Format                           : AAC
Format/Info                      : Advanced Audio Codec
Format_Profile                   : LC
CodecID                          : 40
Duration/String                  : 1mn 30s
Source_Duration/String           : 1mn 30s
BitRate_Mode/String              : Constant
BitRate/String                   : 156 Kbps
BitRate_Nominal/String           : 96.0 Kbps
Channel(s)/String                : 2 channels
ChannelPositions                 : Front: L R
SamplingRate/String              : 48.0 KHz
Compression_Mode/String          : Lossy
StreamSize/String                : 1.69 MiB (1%)
Source_StreamSize/String         : 1.69 MiB (1%)
Title                            : SoundHandle
Language/String                  : English
Encoded_Date                     : UTC 2014-08-01 18:33:39
Tagged_Date                      : UTC 2014-08-01 18:33:39
mdhd_Duration                    : 90900

[RAB78]

@newpball

Well family pictures are priceless!

But ok, if you preserve the originals you can always do that much better, cheaper and faster in the future!

Lol. You're right.... Family pictures are priceless. That's why i always have everything backed up on a second hard drive. You can never be too careful.

[newpball]

What is the reason for converting all to 50 if 25 is my project target? Why not leave it at 25?

Because you need 50 fields (e.g. half frames) per second to get 25i. Going from 50p to 25i will give you the best results for what you want.

And, just checking, you must put this on a BD?

After all the world did not stop after optical disks:



If you are not tied to BD you can simply avoid all those old age interlace shenanigans!

[RAB78]

@newpball

Because you need 50 fields (e.g. half frames) per second to get 25i. Going from 50p to 25i will give you the best results for what you want.

Thanks. That's good to know.
When i render out in Vegas MS, i normally pick a 25i Sony AVC template and let Vegas do the interlacing. So would the Vegas interlacing benefit from having 50p source files?

And, just checking, you must put this on a BD?

Yeah, it's going on a BD for family members to watch and have on their shelf. Plus i like doing the whole package with menus, front cover and disc photos.

If you are not tied to BD you can simply avoid all those old age interlace shenanigans!

I know what you mean about interlacing. I couldn't believe it when i read that 25p or 50p wasn't supported on BD. What's that all about? Lol.

[newpball]

I know what you mean about interlacing. I couldn't believe it when i read that 25p or 50p wasn't supported on BD. What's that all about? Lol.

Luddism, myopia, stuffy engineering and job security!

But I am sure some members on this forum will completely disagree and think that the BD standards should make it in the Guinness World Records book as the best thing since sliced bread.

[newpball]

When i render out in Vegas MS, i normally pick a 25i Sony AVC template and let Vegas do the interlacing. So would the Vegas interlacing benefit from having 50p source files?

Don't know about Vegas I am a Premiere user.

I would not count on an NLE to do those things right, I would advice to use Avisynth for this or a specialized plugin.

[RAB78]

If i'm picking a template to render and that template is telling Vegas how to render the file, how would anything other than Vegas interlace the video?
Are you talking about rendering outside of Vegas using something like DebugMode to frameserve?

[newpball]

If i'm picking a template to render and that template is telling Vegas how to render the file, how would anything other than Vegas interlace the video?
Are you talking about rendering outside of Vegas using something like DebugMode to frameserve?

Your safest strategy if you have various clips of different frame rates is to convert and (in this case) interlace them first (with for instance Avisynth) and encode them with a good editable codec, for instance a wavelet based codec Then edit the results in your favorite NLE. Not only is this mistake proof it will also improve editing speed!

Alternatively if you throw everything mix and match on the timeline and want deal with conversions later piece by piece you have a chance of making mistakes or omitting things or the NLE might actually mess things up as well.

[RAB78]

Alternatively if you throw everything mix and match on the timeline and want deal with conversions later piece by piece you have a chance of making mistakes or omitting things or the NLE might actually mess things up as well.

Very true. Editing this project will be complicated enough without anything else going wrong.

I've just downloaded the Cineform codec as i've heard good things about it. Are you familiar with it? Do i really need to choose the highest quality HD setting to get a 1:1 conversion?
Is there a way of getting Avisynth scripts into MPEG StreamClip? I'd like to try the DNxHD codec too.

So if the source files are interlaced first, then the NLE won't try to interlace them a second time under the chosen template?

[newpball]

I've just downloaded the Cineform codec as i've heard good things about it. Are you familiar with it?

Yes, that's a great codec!

Do i really need to choose the highest quality HD setting to get a 1:1 conversion?

If you can then why not.

But, important, since this is 4K you would want to see if your HD can handle it!

Is there a way of getting Avisynth scripts into MPEG StreamClip?

Perhaps but why add another unknown? Command line and bat files are king when it comes to Avisynth!

I'd like to try the DNxHD codec too.

That's pretty good as well.

So if the source files are interlaced first, then the NLE won't try to interlace them a second time under the chosen template?

No it won't.

[Skiller]

What is the reason for converting all to 50 if 25 is my project target? Why not leave it at 25?

Because

For 25i distribution you can easily interlace the final rendered 50 fps timeline to 25i. It's certainly better to do all the work in 50 fps and then do the conversion to 25i as the last step but it's not a must.

Converting everything to the same frame rate and frame size before editing ensures your NLE doesn't need to do any of these things on it's own, which is good because NLE's aren't particular good (and intelligent) at these things. I use Vegas myself and I would never let it do a frame rate conversion because it sucks at this.

I tried ChangeFPS(50) and the resulting 50p video was more juddery than the 30p video. Not very nice to watch.

You probably watched the 50 fps video on a computer LCD monitor running at 60 Hz, right? Well, if you watch it on a TV the 30 fps to 50 fps ChangeFPS pulldown will look quite good, I have done this myself a few times and was surprised how well 30 fps spread over 50 fps (despite, mathematically, it looks like a bad match)

Why do you prefer to convert the video to 50fps after it's already sped up to 25fps PAL?

It's not a must, you can disable "intelligent resampling" in Vegas so it will do the simple frame duplicating 25 -> 50 fps by itself. If you do it beforehand you're just making 100% sure nothing will be screwed up by Vegas because the clips already match the project properties. That's my intention behind it all: make everything you throw at your NLE match the project properties so that you get predictable and high quality output.

I know what you mean about interlacing. I couldn't believe it when i read that 25p or 50p wasn't supported on BD.

50p is supported on BD for 720p.

[RAB78]

@poisondeathray

To get it working, you should convert it to CFR using convertfps=true

Using the script as you posted it (moving the resizing under DirectShow) the playback was quite stuttery.
However, using the same script but converting to 50fps produced a lovely smooth video that was identical to the original 30fps video.
So looks like i can get on converting these files now. A daunting task awaits me as there are 141 clips totalling 2h50mins. After a few tests i've worked out that it should take about 12.63mins per minute of video. Depending on the amount of motion in the video i presume?
Ah well. It'll be worth it in the end. I would rather spend time doing a good job then taking short cuts and ending up with something i'm not happy with.

So here's my final script for converting 30fps 4kp to 50fps 1080i....

DirectShowSource("J:\Scotland 2014\4K Files\20140801_193207.mp4", fps=30, convertfps=true)
Lanczos4Resize(1920,1080)
#ConvertToYUY2(matrix="PC.601") #For RGB sources
super = MSuper(pel=1)
#pel=1 can also be pel=2 or pel=4
backward_vec = MAnalyse(super, isb = true)
forward_vec = MAnalyse(super, isb = false)
MFlowFps(super, backward_vec, forward_vec, num=50,den=1, ml=100)
separatefields()
selectevery(4,0,3)
weave()

I'm encoding to Cineform through VirtualDub for editing in Vegas Movie Studio.
Is there anything i should know in regards to Cineform or VirtualDub settings?

I've been doing a few tests and so far they look good. The Cineform settings i've using in my tests are.....
Both boxes ticked for "Use ITU.Bt.709 [HD] colorspace [default on]".
Encoding Quality - Medium to High (Can't really tell the difference but i guess it depends on the clip. Dark or well lit.)
Video Format and Pixel aspect Ratio - Automatic

I've read that when encoding to Cineform, higher bitrates are more CPU intensive while editing. Some guy on a forum said that Medium is good enough for 1080 Bluray.

I know this is now off topic but i'd appreciate if you could hold my hand just a little longer. Thanks.

Thanks to everyone for helping me out with this. I'm overwhelmed by the response i've had from all of you.

[poisondeathray]

Yes, 25fps is going to be more "stuttery" than 50fps.

You need to put AssumeTFF() before the re-interacing portion (before separatefields() ) . HD is always TFF by convention. Avisynth assumes BFF by default

Medium should be "good enough" for most people, but there are differences when you zoom in. By the final encoding after editing, you'd be hard pressed to see the differences but I guess it depends how picky you are

[RAB78]

@Skiller

That's my intention behind it all: make everything you throw at your NLE match the project properties so that you get predictable and high quality output.

This is the main thing i've learnt from this thread. To prepare all the files fully for the NLE so it's handling of the video during render is kept to a minimum.
To be honest, i never knew Vegas had to do so much to my files to conform them to the project properties. I just kinda put them all in the timeline and trusted Vegas would take care of them properly. I knew that the framerate had to match the project but all this about "interlacing the files before" and "not trusting the NLE to interlace" was unkown to me.
One of the downfalls of prepare such a massive amount of files other than the time it takes to do it, is the HDD space it takes up.
I worked out that i have roughly 8 hours of footage and that it would probably take up about 500GB of storage if i encode at Cineform High HD.
I should be ok though. It'll just be so time consuming though.

This thread sure has been a learning experience that's for sure and i've got you guys to thank for that.
My blurays are gonna look so much better!

[RAB78]

@poisondeathray

You need to put AssumeTFF() before the re-interacing portion (before separatefields() )

Thanks. I'll add that and run a test.

Medium should be "good enough" for most people

Glad to hear you say that.

but there are differences when you zoom in

Yeah, there will be some instances where i'll be punching into the 4k footage. That'll require a recode to High i guess.

it depends how picky you are

Not quite a pixel peeper but if i start seeing blocking artifacts in the shadows, i'll consider recoding to High.

[jagabo]

By the way, this type of motion interpolation works pretty well with panning shots but it can lead to severe distortions when motions get too large or complex. Try it with the attached video.

[newpball]

Not quite a pixel peeper but if i start seeing blocking artifacts in the shadows, i'll consider recoding to High.

Wavelet based compression does not use blocks.

[RAB78]

@jagabo
Yeah, i know what you mean. I did a test on a video shot in a car with the windscreen wipers going and it was doing all sorts of weird things to them.
That kinda thing doesn't bother me as it's not really that noticeable when you watch the video full speed.
Fast panning shots may cause problems however. We'll see how it goes.

@newpball
Oh right. I meant the effect a low bitrate can have on dark areas. Don't know the proper term.

[newpball]

By the way, for those who use Cineform with PP 2014.2 be careful there are a lot of people who complain about crashes (including me).