Resize plugin for Vegas or NLE with the Lancsoz/Sinc and x264 support?

[Digika]

My biggest issue with NLEs is that not many of them support resizing algorithm better than some lightweight version of Bicubic and the ability to encode with x264/x264vfw (with custom settings) support. So the process of simple video editing always goes in 2 really frustrating steps: exporting edited stuff into something lossless (Lagarith), which results in in sanely big files, up to 80GB+ for 1hr of 720p, and then encoding into preferred format with whatever tool you prefer (virtualdub/avidemux or manual muxing and encoding).

My dream is basically be able to:

a) edit the video in non-linear format (nothing fancy, simple text, transitions, overlay text, speed change, levels adjustments etc)
b) export the result into AVC/AAC (mp4) with x264 encoder, applying crop and decent enough resize.

Sony Vegas, for example, supports x264vfw, but its best resizing algorithm is "Bilinear" with "Best" project settings preset. There is simply no choice. I know it has a plugin set called Boris Continuum Complete, and it contains plugin SuperRez but I never found any info what algorithm it uses + I never was able to even test trial version, InstallShield simply won't let me install it and support can't help heh.

Adobe Premiere does has support for Lancsoz 2, but only with GPU rendering. Also, Premiere is kinda a monster I don't want to even deal with, it is like using a mortar against bees.

Anyone has any suggestions with NLEs that feature good resizing algorithms and support x264 or may be know some rare Vegas plugins that add resize/upscale algorithms to latter?

P.S. Yes, I'm aware you can make avisynth script for Vegas, but it is essentially the same thing with re-encoding and upscaling lossess [lossy] material and then importing into Vegas again. I want to eliminate that step completely.

[altarvic]

You should use DebugMode Frameserver for Vegas Pro

[racer-x]

Aviutl.

[Digika]

You should use DebugMode Frameserver for Vegas Pro

This is currently what I use but it has some small disadvantages.

[newpball]

Premiere Pro uses Lanczos provided you have CUDA enabled.

DebugMode frameserver has problems with Premiere Pro.

The real question is what is your problem with the built-in H.264 encoder? If you think that the results will be a day and a night difference you are completely mistaken, as long as you use a decent bitrate the encoding should be just fine.

And if you are not using a decent bitrate and declare victory if you can save a couple of odd bytes then then you are simply not quality and production driven.

[_Al_]

You should use DebugMode Frameserver for Vegas Pro

This is currently what I use but it has some small disadvantages.

Can you specify what exactly?

[poisondeathray]

P.S. Yes, I'm aware you can make avisynth script for Vegas, but it is essentially the same thing with re-encoding and upscaling lossess [lossy] material and then importing into Vegas again. I want to eliminate that step completely.

So all this is for UPscaling? Don't you want to use better upscaling algorithm than lanczos ?

The difference in scaling algorithm will be more noticable for things like simple animation/cartoons, titles, text, clean lines ; less noticable on live action content

So if you're adding titles etc.. in vegas, you probably want to upscale before adding them, not after. Otherwise they will be upscaled (unless that is the idea to degrade them to match the content?) . So this implies importing an upscaled video in the first place, not upscaling after

[Digika]

You should use DebugMode Frameserver for Vegas Pro

This is currently what I use but it has some small disadvantages.

Can you specify what exactly?

As I said, it is a small annoyance, like, WAV stream being damaged in some places and DMFS audio stream not recognized at all. No big deal but super annoying.

[Digika]

P.S. Yes, I'm aware you can make avisynth script for Vegas, but it is essentially the same thing with re-encoding and upscaling lossess [lossy] material and then importing into Vegas again. I want to eliminate that step completely.

So all this is for UPscaling? Don't you want to use better upscaling algorithm than lanczos ?

The difference in scaling algorithm will be more noticable for things like simple animation/cartoons, titles, text, clean lines ; less noticable on live action content

So if you're adding titles etc.. in vegas, you probably want to upscale before adding them, not after. Otherwise they will be upscaled (unless that is the idea to degrade them to match the content?) . So this implies importing an upscaled video in the first place, not upscaling after

Like what? SplineXX is no better than Lancsoz3/4 for upscaling, while SINC is [questionably] better [and] it requires a bit more complex match behind it, not in every case it could be better and I don't feel like retesting and re-encoding stuff just to find proper setup for it. Besides, the upscaling is just for around +40-30% of base height so no big deal.

That being said you do have a point regarding the applying effects later. So, what would be your suggestion? Feed the upscaled content through Vdub's frameserver, which not many editor understand.

[poisondeathray]

Like what? SplineXX is no better than Lancsoz3/4 for upscaling, while SINC is [questionably] better [and] it requires a bit more complex match behind it, not in every case it could be better and I don't feel like retesting and re-encoding stuff just to find proper setup for it. Besides, the upscaling is just for around +40-30% of base height so no big deal.

I meant dedicated upscaling algorithms, including retail ones. All them have one thing in common - something to combat aliasing (jaggy, "stair stepping" edges), +/- some sharpening. The free one commonly used would be nnedi3_rpow2 in avisynth . Retail ones also use superresolution techniques like video enhancer. Some other common ones are instant hd, topaz enhance, genuine fractals etc... You can read more on upscaling there is a lot of discussion on this topic on various forums, comparisons posted etc...

It won't be a huge difference on small upscaling %, and only makes a noticable difference on some types of content (basically anything with clean straight lines, like cartoons, titles etc...), so it might not be worth the effort for you. But if you're looking at other (UP)scaling algorithms, it's difficult to carry a conversation when you omit the ones typically used for upscaling. However they are slower / more CPU intensive than just "plain" scaling algorithms like lanczos, spline, etc...

That being said you do have a point regarding the applying effects later. So, what would be your suggestion? Feed the upscaled content through Vdub's frameserver, which not many editor understand.

Most common would be lossless intermediate, but you said you wanted to avoid that

People have used avs2avi before as well with vegas, but it doesn't always work. One that works with every program 100% is avisynth virtual file system . Of course the "negatives" of frame serving are extra overhead, poor NLE timeline performance / scrubbing (especially if you're using multiple filters). The "positives" are less HDD space compared to a lossless intermediate

[poisondeathray]

And if you just need simple edits, titles etc... racer-x's suggestion of aviutl is excellent

It's a bit "clunky" for a NLE compared to vegas or other retail NLE's, but it's really a gem for freeware. It has all the usual resize algorithms available like lanczos, spline, bicubic, a few others, etc.., can import basically anything MKV, etc.. (l-smash support) , has direct x264 export options, ffmpeg export options. Because of the l-smash and ffmpeg support, in some ways it's actually more versatile than "pro" NLE's; but realistically it cannot replace a "pro" NLE for big projects, speed of edits and other features

[Digika]

nnedi3 would be overkill indeed, also, in case with nnedi_rpow2, since the base factor is 2 or in specified range ("Image enlargement factor. Must be a power of 2 in the range [2,1024]."), I would need to do downsampling, so it is double pointless. Retail ones as well.

People have used avs2avi before as well with vegas, but it doesn't always work. One that works with every program 100% is avisynth virtual file system . Of course the "negatives" of frame serving are extra overhead, poor NLE timeline performance / scrubbing (especially if you're using multiple filters). The "positives" are less HDD space compared to a lossless intermediate

I know that AVSvf is kinda slow, just like VFAPI. Did not read much about avs2avi.

That being said:

Of course the "negatives" of frame serving are extra overhead, poor NLE timeline performance / scrubbing (especially if you're using multiple filters)

My main concern and the reason I just do editing first and then small upscaling.

And if you just need simple edits, titles etc... racer-x's suggestion of aviutl is excellent

It's a bit "clunky" for a NLE compared to vegas or other retail NLE's, but it's really a gem for freeware. It has all the usual resize algorithms available like lanczos, spline, bicubic, a few others, etc.., can import basically anything MKV, etc.. (l-smash support) , has direct x264 export options, ffmpeg export options. Because of the l-smash and ffmpeg support, in some ways it's actually more versatile than "pro" NLE's; but realistically it cannot replace a "pro" NLE for big projects, speed of edits and other features

I have AviUtl_ExtraPack1-10[Win7], but it is rather unstable and crashes every time I try to do anything with Layers. Error message on JP isn't helpful either :P

[poisondeathray]

nnedi3 would be overkill indeed, also, in case with nnedi_rpow2, since the base factor is 2 or in specified range ("Image enlargement factor. Must be a power of 2 in the range [2,1024]."), I would need to do downsampling, so it is double pointless. Retail ones as well.

Well if you want the best quality , I would argue it's not pointless. It might be overkill if your content doesn't benefit much from it. It's easy demonstrate - especially on certain types of content - it definitely makes a difference over plain lanczos3/4 or spline36 etc....Even on a 1.4x upscale lanczos will leave more aliasing artifacts

My main concern and the reason I just do editing first and then small upscaling.

Got it.

If it's just simple edits, aviutl.

I don't know of anything that integrates or a plugin that allows vegas to use alternative algorithm

DMFS would be what is commonly used, but you said you had problems with it

Or not ideal , but another option is you could use bicubic and add some sharpen in vegas

[_Al_]

WAV stream being damaged in some places and DMFS audio stream not recognized at all. No big deal but super annoying.

Yes, some versions of Vegas could behave differently concerning audio and dmfs.
Sometimes is good idea to pre-render audio as WAV , checking that box while exporting dmfs. It might start to work properly. Or running dmfs only once and then to re-start it might fix audio problems.

Or batch files are good to fix glitches like that. Dmfs AVI to MP4.zip,8.6 MB, encodes dmfs through avisynth script.

You can export just audio from Vegas, unchecking video stream option. As a name use exactly same name as later avi name using frame server. Save it to the folder where that _AVI_to_MP4.BAT is.
Then exporting dmfs as usually. Using that _AVI_to_MP4.BAT. Just drop that AVI and one INI file onto AVI_toMP4.BAT. It will encode video and it also would use that pre-rendered, external audio to mux mp4. If that audio is not found batch would proceed to encode that audio from avisynth script like normally does. Command line could be also used : _AVI_to_MP4.BAT signpost.avi some.INI

You can use INI file that looks like that:

path_destination      :C:\Destination
path_temp             :C:\Temp
x264_variables        :--crf 18 --ref 3 --vbv-bufsize 40000 --vbv-maxrate 40000
nero_variables        :-q 0.5
delete_temp_folder    :yes
mux_external_audio    :yes


write avisynth script between lines, no loading clip line in there!
--------------------------------------------------------------
ConvertToYV12(matrix="PC.709")
#LoadPlugin("C:\... your path, if this dll is not in Avisynth plugin folder......\nnedi3.dll")
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=1920, fheight=1080)
--------------------------------------------------------------

couple of INI files are already in that zip, they can be tweaked to fit and given any name.

[Digika]

Or batch files are good to fix glitches like that. Dmfs AVI to MP4.zip,8.6 MB, encodes dmfs through avisynth script.

This one is going to be handy, thanks!

[MaverickTse]

nnedi3 would be overkill indeed, also, in case with nnedi_rpow2, since the base factor is 2 or in specified range ("Image enlargement factor. Must be a power of 2 in the range [2,1024]."), I would need to do downsampling, so it is double pointless. Retail ones as well.


I have AviUtl_ExtraPack1-10[Win7], but it is rather unstable and crashes every time I try to do anything with Layers. Error message on JP isn't helpful either :P

Please report the Error with screen shot

[B/D]

I know it has a plugin set called Boris Continuum Complete, and it contains plugin SuperRez but I never found any info what algorithm it uses + I never was able to even test trial version, InstallShield simply won't let me install it and support can't help heh.

I imagine this is way too late for you to care, but you can probably just

• install the plugin in a different volume (e.g. "D:\" instead of "C:\") which won't be protected by InstallShield,
• then (all of the following as an administrator) copy the name of the folder the new plugin is in,
• create a new folder with that name at the same location in the original Vegas install directory (presumably C:\Program Files [(x86)]\Sony Vegas\[PluginsOrWhatever]\)
• open the new plugin's original folder and make sure everything is unhidden and owned by you or Administrator[s],
• then 'Edit > Select All'
• and 'Edit > Move to folder...' - select your newly created plugin folder in the original Vegas install directory. Done.

Takes about 16x as long to type it as to do it. You don't have to follow these directions to the letter - just install it somewhere else and move it, basically. I have to do this with HandBrake since its certs wont validate. Of course, with a plugin rather than an application, you might have to install a dummy copy of Vegas in the non-protected location first to install the plugin into. Or you could disable InstallShield through the Microsoft Management Console, but this way is simpler (and probably safer).