How to retain good deinterlacing in Vegas 12?

[Srivas]

Hi!
I'm editing a video where's lots of movement, camera panning 360 degrees, etc. The problem is that after conversion the overall quality is what I expected, but the exact places where's fast movement, look too jerky, ghostly.

I use Vegas 12 as a NLE, input files are full HD mts files, upper field first, from a consumer HD camcorder. I render out as Mainconcept AVC/AAC mp4, with slightly modified preset "Internet HD 720p" (frame rate set to 25 as the source file, variable bit rate, where max. is 8000). When I change deinterlacing in project settings into "interpolate fields", the result is better, but how then overall frames are handled, is also not nice. There's only two deinterlacing options. Is there anything to do? Is that all Vegas can do? What's the best I can get out of Vegas?

Thanks for any help guys.

Oh, I will also upload a sample if it helps. That is my rendered file (deinterlacing: blend fields). BTW. The original looks really good on Windows Media Player (no ghosting at all), but not on Potplayer that I usually use (looks the same as if after rendering). The difference is huge, WMP does a really good job of deinterlacing, although the functionality of the program is horrible. How is that? No good functional media player with good deinterlacing?

[poisondeathray]

You need to double rate deinterlace (bob) to retain the smoothness similar to WMP . So export 50p instead of 25p in vegas

You can do than in any player on the original file similar to how WMP does it (ie. just for playback of the original file), but some players require some extra setup, others do it automatically

But the actual quality of how WMP and vegas performs the deinterlace isn't that great

[TreeTops]

I would increase the bitrate.

Edited post to add "Poisondeathray beat me by 1 minute"

[Srivas]

Thanks Poison, it worked! It's quite better than 25p, but not as good as I would like it to be. And rendering time is almost twice as time. Now that I will always sacrifice for quality, but can I get it even better? I guess you gave the best solution if I want to stay with Vegas, right? Or is there something more you're not telling me)? Can I tweak something else out of it?

[poisondeathray]

Understand that when you playback the original file in WMP , it's bob deinterlacing 1920x1080 50 fields per second (25 frames per second) to become 1920x1080 50 full frames per second. It's not resizing down to 1280x720.

You can get slightly better quality if you bump up the bitrate much higher and use 1920x1080p50, just like WMP is doing it. Yes it takes longer, because you have 2x the number of frames. And if you decide to use the higher resolution it will take even longer.

Alternatively, you can export it interlaced , like the original, and just let the player do the deinterlacing . It depends on what the goals are, how you are viewing it etc... . If the destination is web, you'll probably want to deinterlace it (and some sites like youtube support 50p/60p now)

There are alternative deinterlacing methods, but more complicated requiring scripts, avisynth etc.., but that's the best you can do within vegas . Most people find "interpolate" better than "blend" . Blend makes "ghosted" blurry images

[Srivas]

Thank you, that's quite clear now.

Actually yes, interlaced does give even a better result (of course). I do it for my friends (who watch it on their computers, smartphones) and also upload to Youtube. But Youtube is a separate issue, then I use higher bitrate etc. So as for distribution, I try to keep the file size as small as possible with decent quality.

One more question. If I leave it interlaced, is that just as ok? I mean, does most of the devices/players, as above mentioned, play it back just fine, without any extra settings? Then I guess I could just keep it interlaced.

[poisondeathray]

Actually yes, interlaced does give even a better result (of course). I do it for my friends (who watch it on their computers, smartphones) and also upload to Youtube. But Youtube is a separate issue, then I use higher bitrate etc. So as for distribution, I try to keep the file size as small as possible with decent quality.

One more question. If I leave it interlaced, is that just as ok? I mean, does most of the devices/players, as above mentioned, play it back just fine, without any extra settings? Then I guess I could just keep it interlaced.

Leaving it interlaced is ok, only if the intended playback device/chain can handle it. Really that is only some players on computers (some software players need to be configured, and won't auto-deinterlace), some BD players . Most other portable devices (e.g. ipad / iphone / androids) etc.. won't display it properly or will have problems without some workarounds and 3rd party software

RE: Filesize as small as possible with decent quality - You'd want to use a different encoder than vegas AVC or mainconcept AVC. They look ok if you give enough bitrate. x264 will give you higher quality/compression . There are ways to use it in vegas, the most popular method would be debugmode frameserver and some x264 GUI like megui. You can also use x264vfw directly in vegas but it saves AVI only - then remux it later into another container like MP4 . And if you were doing those extra things, you might even look at the alternate deinterlacers in avisynth

[Srivas]

Ok, thanks. I'll check them out!

[sheppaul]

This thread will give you the answer for the deinterlacing quality of potplayer vs WMP.
http://goo.gl/v7FUbW

[Srivas]

yep, I already found it.)) But thanks anyway!

[Srivas]

Hi!

RE: Filesize as small as possible with decent quality - You'd want to use a different encoder than vegas AVC or mainconcept AVC. They look ok if you give enough bitrate. x264 will give you higher quality/compression . There are ways to use it in vegas, the most popular method would be debugmode frameserver and some x264 GUI like megui. You can also use x264vfw directly in vegas but it saves AVI only - then remux it later into another container like MP4 . And if you were doing those extra things, you might even look at the alternate deinterlacers in avisynth

I'm back here on the thread since I started to go further and got into frameserving to MeGui, added Nero AAC for audio (by using a guide from here) and want to deinterlace using Yadif (or anything else) but an error occurs and I cannot solve it. When setting the deinterlacing on in MeGui - Avisynth script creator and hitting "preview AVS Script" it gives the "Video must me YUV" error. So I did some research and added the line "ConvertToYV12(interlaced=true)" to the script, but it still gives the same error. How can I get rid of this?

Otherwise the process according to the above guide is working nice. Now even when I render without deinterlacing, it still comes out as progressive (I guess that's what the x264 encoder assumes?), and overall quality is better that the Mainconcept avc filter (I test with 4000 kbps), but the colors are slightly dimmed and also less sharpness than the direct Vegas mainconcept render. How can I effectively improve that?

And since I'm here already, should I try some other/better deinterlace filter which will be fast but better than Yadif?

That brings me to the thought that I can just frameserve the original file (after cutting/splitting, doing all the editing which is convenient in the NLE) and apply the filters in avisynth, if they do a better job in terms of quality and rendering speeds.

[poisondeathray]

Yadif isn't much better than what vegas uses for deinterace interpolate. The artifacts are slightly different (different type of "marching ants") . I would use QTGMC in progressive mode - that along with x264 will make the biggest difference . You don't have to use the default "slower" mode. Use a faster preset (it's still going to be a few times slower than yadif, unless you decide to use avisynth MT - but that has it's own issues)

Now even when I render without deinterlacing, it still comes out as progressive (I guess that's what the x264 encoder assumes?)

You can encode interlaced (MBAFF) if you want, by using the correct switches. In x264, it's --tff or --bff for Top or Bottom field first, respectively. If you don't specify the switch, x264 encodes progressive



Post your full avs script .


When frameserving out of vegas, from a 8 bit project, with native camera files such as AVCHD - you will be using studio RGB in vegas. Make sure you set debugmode frameserver to frameserve out RGB, not YUY2. That also means you should be using a PC matrix with convert back to YV12 (this basically "undos" what vegas did - that's why levels are different - what you call "dimmer") , and the interlaced switch if your project was the same as your camera files

ConvertToYV12(matrix="PC.709", interlaced=true)

That brings me to the thought that I can just frameserve the original file (after cutting/splitting, doing all the editing which is convenient in the NLE) and apply the filters in avisynth, if they do a better job in terms of quality and rendering speeds.

If you are doing multipass encodes, then it might make sense to export from the NLE as a lossless intermediate then run multiple passes

But if you were using 1 pass CRF or ABR encoding (assuming you're doing editing in vegas), then probably frameserving out of vegas directly is faster

[Srivas]

Ptd, thanks for a quick reply!

Yadif isn't much better than what vegas uses for deinterace interpolate. The artifacts are slightly different (different type of "marching ants") . I would use QTGMC in progressive mode - that along with x264 will make the biggest difference . You don't have to use the default "slower" mode. Use a faster preset (it's still going to be a few times slower than yadif, unless you decide to use avisynth MT - but that has it's own issues)

Ok. I got the QTGMC installed. What exact line do you use in avisynth? I'm sorry, just that makes things a little faster for me, but I will go through the manual also.

[Srivas]

You can encode interlaced (MBAFF) if you want, by using the correct switches. In x264, it's --tff or --bff for Top or Bottom field first, respectively. If you don't specify the switch, x264 encodes progressive

Yes, I saw that, but since I need progressive files, that won't be required. Just to kick in good deinterlacing.

[poisondeathray]

For general use , I typically use the "faster" preset in QTGMC, with sharpening turned down (I find it oversharpens by default) . You'll eventually find settings that you "like" personally. There is documentation with almost all avisynth filters. QTGMC has a lot of settings and switches you can read about them later - but start with the preset settings like "fast" , "slower" etc...

Start with and learn about non-MT avisynth (non multithreaded) first , then you can explore MT later . It can be problematic with serious stability and quality issues (some people can't get it working, or have mixed up frames, or other errors like green frames), but it does offer a substantial speedup if you can get it working, easily 1.5-3x for something like QTGMC alone . The more filters you stack (espeically temporal filters), the higher chance of problems

Your script from a vegas=> debugmode RGB frameserve might look something like this

AVISource("Signpost.avi")
ConvertToYV12(matrix="PC.709", interlaced=true)
AssumeTFF()
QTGMC(preset="faster", sharpness=0.6)

[Srivas]

Post your full avs script.

Basic script:
<input>
<deinterlace>
<crop>
<resize>
ConvertToYV12(interlaced=true)

I guess the last line is the only real script, isn't it?

When frameserving out of vegas, from a 8 bit project, with native camera files such as AVCHD - you will be using studio RGB in vegas. Make sure you set debugmode frameserver to frameserve out RGB, not YUY2. That also means you should be using a PC matrix with convert back to YV12 (this basically "undos" what vegas did - that's why levels are different - what you call "dimmer") , and the interlaced switch if your project was the same as your camera files

ConvertToYV12(matrix="PC.709", interlaced=true)

Yes, frameserving is on RGB32. The line you suggested fixed the colors! Thank you!

If you are doing multipass encodes, then it might make sense to export from the NLE as a lossless intermediate then run multiple passes
But if you were using 1 pass CRF or ABR encoding (assuming you're doing editing in vegas), then probably frameserving out of vegas directly is faster

I'm doing ABR encoding. Should I do like that? I tested 2 pass before and didn't see any advantage in quality (although many around the internet say there is), but it's twice as slow. I render long files, around 2 hour classes, so saving rendering time is quite important for me.

And Pippas from this forum suggested to use Grass Valley intermediate file, but I didn't test it out yet. And that means more encoding again (but that's probably quite fast).

[Srivas]

Ok, I applied the script and it gives an error.
[Attachment 33136 - Click to enlarge]

I have the latest avisynth installed.

[poisondeathray]

Gathering the filter dependencies can be a pain. Some filters need a specific version of a .dll (there might be a dozen different .dll's with the same name)

There is a package that has everything you need in a single archive (see the "plugins package" link)

http://forum.doom9.org/showthread.php?t=156028

[Srivas]

If I set the deinterlacing in MeGui script creator, I guess I don't need the line "AssumeTFF()"?
And here's the real script:
AVISource("C:\Users\Admin\Desktop\test\test.avi", audio=false).AssumeFPS(25,1)
#deinterlace
#crop
LanczosResize(1280,720) # Lanczos (Sharp)
ConvertToYV12(matrix="PC.709", interlaced=true)
AssumeTFF()
QTGMC(preset="faster", sharpness=0.6)

Should I just go with regular 25 fps or do Bob deinterlacing?

[poisondeathray]

If I set the deinterlacing in MeGui script creator, I guess I don't need the line "AssumeTFF()"?
And here's the real script:
AVISource("C:\Users\Admin\Desktop\test\test.avi", audio=false).AssumeFPS(25,1)
#deinterlace
#crop
LanczosResize(1280,720) # Lanczos (Sharp)
ConvertToYV12(matrix="PC.709", interlaced=true)
AssumeTFF()
QTGMC(preset="faster", sharpness=0.6)

Should I just go with regular 25 fps or do Bob deinterlacing?

You should do bob deitnerlacing if you want the smoothness - you saw the difference earlier

You can't resize interlaced material vertically with a straight resize, before deinterlacing. ie. the LanczosResize has to come last if you are resizing down to 1280x720. (But you can speed it up slightly by resizing horizontally first)

[Srivas]

It works! And quality is impressive! I'm very happy with the results. That's what I was looking for. Thank you Pdt!

About the last suggestion:

You can't resize interlaced material vertically with a straight resize, before deinterlacing. ie. the LanczosResize has to come last if you are resizing down to 1280x720. (But you can speed it up slightly by resizing horizontally first)

Somehow this makes the rendering quite slower (maybe 30%), and moreover, in my test clip there's a fast camera movement which appears wavy after putting Lanczos at the end. I cannot see the difference in quality with the previous settings, but fast movement looks ok. I have to use a longer test clip to notice real difference.

[poisondeathray]

You probably mixed up some of the tests - "wavy" artifacts are from resizing while it's still interlaced. Unless you're referring to something else. The resize definitely is supposed to go at the end. Yes, it's slower but it's the proper way to do it. The bottleneck filter is QTGMC . If you resize to smaller dimensions before, of course it's faster (QTGMC has fewer pixels to process) - but that will give you artifacts

[Srivas]

You probably mixed up some of the tests - "wavy" artifacts are from resizing while it's still interlaced. Unless you're referring to something else.

Nope. Strange as it is, but the artifacts (shimmering) appears when lanzcos is applied after deinterlacing (although overall sharpness and details are a little higher).

Here's the sample of correct resizing. Look for the shimmering when camera pans left at the middle of the video.

test-muxed.mp4

This is the script (did get the MT working and it works good):

SetMemoryMax(512)
SetMTMode(3)
AVISource("C:\Users\Admin\Desktop\test\test.avi", audio=false).AssumeFPS(25,1)
SetMTMode(2)
#deinterlace
#crop
ConvertToYV12(matrix="PC.709", interlaced=true)
AssumeTFF()
QTGMC(preset="faster", sharpness=0.6, EdiThreads=1)
LanczosResize(1280,720) # Lanczos (Sharp)

[_Al_]

If there is nothing wrong with original HD interlace footage and Vegas project is set with the same properties as original, for HD source, Yadif should be enough, no striking difference at all for casual viewing, nobody notices anything. For SD footage, to deinterlace, QTGMC is a must though. But you have HD footage ... I think there is something else going on here.

Why don't you provide a sample of original footage, things after that usually progress much faster ...

[Srivas]

Ok, here it is.
This is rendered deinterlacing set to "none" in Vegas as for frameserving.

[poisondeathray]

test.m2ts ? Was that the original , original footage ? or that was vegas export ?

If that was the original fotoage, there are serious problems with test.m2ts . Are you sure that was straight from the camera ? It has been resized already, improperly (non interlace aware resize), resulting in the "wavy" lines

[jagabo]

test.m2ts is messed up. A vertical resize was performed somewhere along the line and the two fields are no longer separate. You need to cut out a small sample of the original source without reencoding. If that is your original source you're screwed.

[Srivas]

I'm sorry, didn't realize that I already cropped the video in Vegas. Here's the clip as it is. There are some tools to cut the avchd without re-encoding, but no in Vegas. As I cut the original clip from a long video and didn't mark it, I cannot find it so easily. So I did a Sony AVCHD render without changing any paramaters of the original and doing no deinterlacing. That should be ok.

But I also realize that I would like to do some cropping in Vegas and it's keyframed according to the scene, so how can I do that and at the same time go to avisynth and do deinterlacing and maybe some other effects? Or should I do deinterlacing and resizing first in Avs and render into an intermediate file, which to edit then? Seems like some things are easier to do in a NLE, at least where I'm at now.

Here's the file:

[_Al_]

Cropping scene by scene differently means that Vegas would deinterlace internally so using Avisynth later on would make little sense. Damage would be already done. You'd need to batch transfer all clips to 50p intermediate (UTvideo or any lossless) prior to editing and then to load those into Vegas , edit those clips, crop them, and then to export them, but in this scenario, you could use even Vegas export templates because resizing of progressive footage in Vegas is ok and you'd basically already used resizing algorithm within Vegas already (by cropping footage) . You still could use dmfs and then Megui (and x264 encoder) to get H.264 and using Avisynth because using x264 encoder has its its advantages as oppose internal Vegas H.264 encoder, but that would be up to you.

Making of that intermediate would be another challenge for you. Not mentioning if you have lots of those original clips.
--One easy scenario is to load your clip into Vegas exporting UTvideo YUV 420 BT709 (Video for Windows.avi / custom/ video format), you'd need to have utvideo installed prior to that. Do not forget to click configure, assume interlaced. Or use some other lossless. Then bob deinterlace that clip (Avisynth /VirtualDub into lossless again) and load that into Vegas.
--If you manage to load that m2ts into Avisynth then skip that first Vegas lossless exporting step and just load Avisynth script into VirtualDub to get bob deinterlace lossless.

Nothing straight forward. Maybe Cineform can batch transcode to cineform (visually lossless format) using solid bob deinterlace with one nice graphic user interface, not sure.

[jagabo]

Yes, the best way to handle this is to use a quality deinterlacer in AviSynth before editing in Vegas. But be prepared for slow processing and very large lossless or near-lossless compressed intermediate files. As a visual reference, your sample after QTGMC() in AviSynth (h.264 encoded):