How to Improve Commercial DVD with Avisynth that has no problems?

[VideoFanatic]

Demuxed DVD Sample

Improving a Commercial DVD

I've bought a pay per view wrestling DVD. It doesn't have have any problems apart from a little grain and tiny bit of blocking. I can fix that easily and I can use LSFMod to sharpen the picture. I was just wondering if there are any other improvements I could make that you could suggest?

Converting Commercial DVD to Progressive

It's a PAL MPEG2 at 25 fps. If I try to use QTGMC to convert to progressive and keep the same frame rate (to keep it DVD compliant), it speeds up the video. In my video encoder I'm encoding the video as progressive. I don't want to lose half the temporal resolution and I thought that since this is a commercial DVD then it should already be progressive or I should be able to encode it as progressive with it's original framerate without losing half the temporal resolution? Having said that MediaInfo shows that the MPEG2 is Interlaced.

Full script I'm using:

setmtmode(5,9) 
Mpeg2Source("H:\2 = New\video.d2v", CPU=6) 
setmtmode(2,0)  

AssumeTFF() # I've tried with and without this line but it made no difference 
QTGMC(Preset="fast") 
SelectEven() # Documentation says to add this line to keep the original frame rate. I also tried with SelectOdd instead but it made no difference

[themaster1]

Use mdegrain1 and perhaps sharpen horizontally only assuming you want to keep it interlaced. And obviously a quality mpeg2 encoder , mainconcept to not name it

# 1) Denoising:
fields=AssumeTFF().SeparateFields() # or AssumeBFF super = MSuper(fields) backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=2) forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=2) MDegrain1(fields, super, backward_vec2,forward_vec2,thSAD=400)
# 2) Sharpening:
# try edgemode=1 (if you want to sharpen edges only)
LimitedSharpenFaster(ss_x=1.6,ss_y=1.0,Smode=3,str ength=100,soft=0,edgemode=2,wide=false)
weave()

[VideoFanatic]

Thanks. Also I was just wondering why when I demux the DVD to an MPEG2 then look at the info, it says it's "progressive or interlaced". How can it be either, I thought it can only be one or the other?

It's a PAL MPEG2 at 25 fps. If I try to use QTGMC to convert to progressive and keep the same frame rate (to keep it DVD compliant), it speeds up the video.

[themaster1]

you have probably missed the part to put it back to 25fps
QTGMC( Preset="fast")
SelectOdd() <<< this part

[VideoFanatic]

I tried that, it didn't work. I've just created a thread just for this issue: https://forum.videohelp.com/threads/351016-How-to-convert-Commercial-DVD-to-Progressive...91#post2201091

[sanlyn]

AFAIK, most Standard retail PAL DVD is 25fps interlaced. Open in Avisynth using SeparateFields() and view it one frame at a time. If you see each separate frame depicting a separate movement, the video is interlaced. If you see two consecutive fields depicting the same movement, it's progressive.

If its already progressive, you don't have to deinterlace to run your filters. However, for the simple denoise that you want to do, you don't need all those plugins and sharpeners unless you have a rather crappy video. So to keep it progressive and do some edge cleaning, degrarining, and smooth motion a bit -- but retain progressive 25fps PAL without deinterlacing -- use this:
QTGMC(preset="medium",InputType=1)

Source is interlaced and you want to retain 25fps interlaced for output:
QTGMC(whatever presets)
....
....
SeparateFields().SelectEvery(4,0,3).Weave()

Source is interlaced and you want 25fps progressive output after deinterlacing:
QTGMC(whatever presets)
....
....
SelectEven() #<- or SelectOdd() if you think Odd looks better.

Source is interlaced and you want to retain 25fps interlaced for output, without deinterlacing or discarding frames:
AssumeTFF().SeparateFields() # <- or Assume BFF if required
....
....
Weave()

[jagabo]

QTGMC() isn't your problem with changes in running time. Look elsewhere. A quick filter in AviSynth (quick for me to write the script, the processing was slow) :

Mpeg2Source("Commercial DVD Interlaced and Progressive.d2v", CPU=6, Info=3) # full deblocking and de-ringing
QTGMC()
McTemporalDenoise(settings="medium")

Original video running time 55.960 seconds. MKV video running time exactly the same.

[intracube]

You'd need to post some unscaled stills or ideally, a short sample video from the DVD. The choice of filters will be very dependent on the source.

Other than noise and sharpness, common problems are; wrong video levels, poor colour balance, chroma shift.

EDIT: Just spotted your other thread with sample video.

The video levels, colour balance look OK and there's no obvious chroma shift.

[sanlyn]

The only problems I saw in the clip were some excessive interlace combing effrects and some mosquito noise -- likely due to a bitrate somewhat lower than required for very fast action. I fixed it with this:

MPEG2Source("E:\forum\hg_interlaced\Commercial DVD Interlaced and Progressive.d2v")
AssumeTFF().QTGMC(preset="medium")
LSFMod(strength=50)
SeparateFields().SelectEvery(4,0,3).Weave()

If you see grain or block noise, you might suspext a player and/or tv that don't handle motion or deinterlace very well.

[VideoFanatic]

I've updated my 1st post with the information from the closed thread and a sample MPEG2.

Improving Commercial DVD

Is there anything else I could use to get a better picture quality? At the moment I'm deblocking and using LSFMod and QTGMC (keeping it interlaced).

Converting to Progressive

It's a PAL MPEG2 at 25 fps. If I try to use QTGMC to convert to progressive and keep the same frame rate (to keep it DVD compliant), it speeds up the video. In my video encoder I'm encoding the video as progressive. I don't want to lose half the temporal resolution and I thought that since this is a commercial DVD then it should already be progressive or I should be able to encode it as progressive with it's original framerate without losing half the temporal resolution?

Using the likes of QTGMC or yadif to deinterlace, you don't lose "half" of anything. The results of the deinterlace are interpolated, full-data, frames, not "half-fields". So if there is any loss, it's too neglible too worry about and is probably mostly noise. Why are you deinterlacing?

AFAIK retail standard PAL is usually 25fps interlaced.

I'll second jagabo's motion: how about a sample?

Sanlyn, jagobo mentioned half the temporal resolution here: https://forum.videohelp.com/threads/346916-Remove-Interlacing-Artefacts-without-convert...=1#post2168471

You can can see my script in my first post. When I convert to progressive, the video looks speeded up. Is there no way to keep the original frame rate and convert to progressive without causing that problem?

Manono - I never said the audio was speeded up, I just said the video was speeded up. I'm only working on the video file.

[sanlyn]

I'm working the video 4 ways: original, progressive, re-interlaced, and SeparateFields().Weave(). There's no difference in playback speed or time between any of the 4 versions, neither numerically nor visually. I even played it at the same time in 4 windows. It's a very fast-action video to begin with.

I don't see any loss of detail between any of the versions, just some noise cleaned up with QTGMC. The original clip looks interlaced from beginning to end. If you're worried about losing temporal resilution, use SeparateFields and Reweave(). That won't clean the excessive interlace combing, though, unless you use some different filters for that if you're not using QTGMC.

[manono]

Manono - I never said the audio was speeded up, I just said the video was speeded up. I'm only working on the video file.

Oh yeah, you're right. I apologize. And I even quoted from you where you said the video was speeded up before answering about the audio. The same point applies, though. There's nothing in the script you showed us that speeds up the video. sanlyn, just above, confirms it. You did it elsewhere, perhaps in the encoder. Or perhaps accidently in a slightly different script from what you showed us.

[VideoFanatic]

I notice that if I use RemoveSpots() then it blurs the video quite a lot (I use it before QTGMC). I realize I don't need RemoveSpots in this video but I'm just using it as an example because you can clearly see the blurring in this video (I only use RemoveSpots with VHS to DVD conversions of stuff recorded off TV).

I used the same bitrate for with QTGMC and with QTGMC & RemoveSpots and the RemoveSpots version was more blurred and the bitrate the encoder desided to use was lower so the file size was lower. I tried increasing the bitrate so the both file sizes matched but the RemoveSpots version was still blurred. Is there any way to get RemoveSpots to work without blurring the video?

[jagabo]

RemoveSpots() is specifically for removing spots not general noise reduction. Notice the greenish spot on the cloud above the man's head:



This type of spot comes about because of junk on the film, or a piece of the film has flaked off. They last for a single frame and can be replaced with a bit from the frame before or after.

[VideoFanatic]

I know, but on my VHS-to-DVD conversions, I have noise spots that McTemporalDenoise won't remove but RemoveSpots does remove. Is there any way to get RemoveSpots to work without blurring the video?

[jagabo]

RemoveSpots() doesn't work well on high motion video like wrestling. Try RemoveSpotsMC() instead.

[sanlyn]

RemoveSpots is indispensible, unless you want to work frame by frame and require psychotherapy afterward . But sometimes I've done it that way (which probably explains a lot, I guess??). The filter will also calm some noisy backgrounds as well, and I've often used it for that.

I use it all the time, but don't think it's the only filter that softens. Plenty of common smoothers do that (TTEmpSmooth, FFT3D, variations of RemoveDirt, and even MCTemporalDenoise and many deblockers, to name a few). There's no way to remove serious grime without side effects. Follow it with a smart sharpener like LSFMod or LimitedSharpen. If you sharpen before using smoothers or similar cleaners -- which a lot of people do! -- you're just causing problems. Sharpening should be one of the last steps in a filter chain, or you're sharpening noise and artifacts and making them more difficult to remove. Using QTGMC afterward does some resharpening anyway, and despotting before QTGMC makes a cleaner deinterlace IMO.

I've worked with videos that were fairly soft to begin with but had no way to remove 45 frames of spots without RemoveSpots, so I used two techniques. One is to fiddle with sharpeners later and learn to adjust their parameters. It's not difficult once you get the hang of particular sharpeners. The other method is to use ReplaceFramesSimple and replace the frames that had the worst spots. Of course that takes time; you have to specify the frames you want replaced. If you have a clip with 400 spots (and I've had them), this won't do. If you just have 3 frames with spots, it's not so much trouble.

Fast-action videos are a problem, I agree, even when they're fairly prisitine. You can get clever sometimes with functions like ReplaceFramesMC and interpolate a new frame between two clean frames. But with fast action, this likely won't work; you'll end up with really weird artifacts.

It comes down to this: rescuing crappy video is a chore. Some of it's easy as pie, some of it drives you bananas. That's just the way it is.

[manono]

I used the same bitrate for with QTGMC and with QTGMC & RemoveSpots and the RemoveSpots version was more blurred and the bitrate the encoder desided to use was lower so the file size was lower.

You're contradicting yourself. Two different encodes of the same length with different filters (one with and one without RemoveSpots) using the same bitrate will wind up with roughly the same size. Now, if you had done quality or quant based encodes to compare, then they could conceivably wind up with significantly different sizes. But you specifically said 'I used the same bitrate...'. The final file size has nothing at all to do with the filters used or not used, but only on the length and bitrate.

I use RemoveSpots quite a lot myself and haven't noticed any significant blurring of the resulting video. Oh yeah, I just noticed jagabo's comment about it not working well with fast-moving sports. Mine aren't like that so maybe what he said applies and can account for the blurring you see.

[VideoFanatic]

I used the same bitrate for with QTGMC and with QTGMC & RemoveSpots and the RemoveSpots version was more blurred and the bitrate the encoder desided to use was lower so the file size was lower.

You're contradicting yourself. Two different encodes of the same length with different filters (one with and one without RemoveSpots) using the same bitrate will wind up with roughly the same size. Now, if you had done quality or quant based encodes to compare, then they could conceivably wind up with significantly different sizes. But you specifically said 'I used the same bitrate...'. The final file size has nothing at all to do with the filters used or not used, but only on the length and bitrate.

I use RemoveSpots quite a lot myself and haven't noticed any significant blurring of the resulting video. Oh yeah, I just noticed jagabo's comment about it not working well with fast-moving sports. Mine aren't like that so maybe what he said applies and can account for the blurring you see.

The filters DO make a difference to the bitrate and file size! The more noise you remove the lower the file size!

Try it yourself if you don't believe me with the sample from my first post. Try the following script with and without RemoveSpots using Simple x264 Launcher encoding at CRF 19. Use Preset VeryFast and the following custom parameters: --tff --sar 16:15. Simple x264 Launcher is portable so it doesn't install anything.

setmtmode(5,9)  Mpeg2Source("H:\2 = New\video.d2v", CPU=6)  setmtmode(2,0)   
RemoveSpots()
AssumeTFF()  QTGMC(Preset="Medium") 
SeparateFields()
SelectEvery(4,0,3)
Weave()

You'll see that the file size & bitrate with RemoveSpots is lower and the picture quality looks more blurry.

[manono]

ReplaceFramesMC(1790,1)

That's an awful lot of typing. Maybe you have a bunch of them in a script template all ready to go, so you just remove the '#' and type in the frame numbers. But I have a version that requires only RX(1790,1). Just ask if interested. And I use Gavino's FixBadFrames for single frame interpolation which requires only typing in the single frame number after typing in the function's name the first time:

FixBadFrames("1790 1895 2000 2100")

And I do go through the thing frame-by-frame (holding down the 'forward' key in VDub) to find the stuff, and an over-two-hour old film might need hundreds of instances of frame interpolation. Even after using RemoveSpots.

[manono]

Try the following script with and without RemoveSpots using Simple x264 Launcher encoding at CRF 19.

You said you used the same bitrate. That's not true as you did quality-based encodes (using CRF 19), and I also mentioned differing sizes might result from quality-based encodes. Yes, you also said you later increased the bitrate to get the file-sizes to match, but had you said you used CRF encoding in the first place there would have been nothing for me to contradict. Your post was very misleading.

[VideoFanatic]

OK sorry. I didn't think most people would know what I was talking about if I said I used CRF 19 with Simple x264 Launcher so I just said I used the same bitrate in both encodes - it's really a variable bitrate. Anyway the fact remains that even if I use a better quality CRF setting with RemoveSpots, the quality still looks worse (more blurred) than a lower quality CRF setting without RemoveSpots.

[poisondeathray]

So don't use RemoveSpots if you don't like it. Or use different settings. All spot/dirt removal filters will damage the picture to an extent, just like all denoising filters will soften the video to an extent

You shouldn't use spot/dirt removal filters on interlaced video, while it's interlaced (you need to separate odd/even fields before applying, then re-weave, or apply it to progressive video since you are using QTGMC)

[VideoFanatic]

So don't use RemoveSpots if you don't like it. Or use different settings. All spot/dirt removal filters will damage the picture to an extent, just like all denoising filters will soften the video to an extent

You shouldn't use spot/dirt removal filters on interlaced video, while it's interlaced (you need to separate odd/even fields before applying, then re-weave, or apply it to progressive video since you are using QTGMC)

If I want an interlaced video should I use it like this?

AssumeTFF() # or AssumeBFF() depending on the source
QTGMC(Preset="Super Fast") # Removes interlacing artefacts. Strangely this is faster than Ultra Fast
RemoveSpots()
SeparateFields()
SelectEvery(4,0,3)
Weave()

[sanlyn]

ReplaceFramesMC(1790,1)

That's an awful lot of typing. Maybe you have a bunch of them in a script template all ready to go, so you just remove the '#' and type in the frame numbers. But I have a version that requires only RX(1790,1). Just ask if interested. And I use Gavino's FixBadFrames for single frame interpolation which requires only typing in the single frame number after typing in the function's name the first time:

FixBadFrames("1790 1895 2000 2100")

And I do go through the thing frame-by-frame (holding down the 'forward' key in VDub) to find the stuff, and an over-two-hour old film might need hundreds of instances of frame interpolation. Even after using RemoveSpots.

Since I'm importing the ReplaceFramesMC script anyway, I just copy it from the import line and paste, or paste from a different line. That video sample clip has about 2900 frames; I used the "RX" 3 times.

Yeah, I've gone frame by frame. ANd in a dar5kiened room, or you miss half the spots. That 2900-frame clip has over 200 bad spots. Bad tape? You betcha, and most of it seems to come from the mastering source. Digital broadcasts of the movie have the same spots !

Anyway, I'm mostly cut and paste. When you clean up 200 frames at a shot, "RX" is still too much typing and too many typos when you get tired .

[sanlyn]

If I want an interlaced video should I use it like this?

AssumeTFF() # or AssumeBFF() depending on the source
QTGMC(Preset="Super Fast") # Removes interlacing artefacts. Strangely this is faster than Ultra Fast
RemoveSpots()
SeparateFields()
SelectEvery(4,0,3)
Weave()

I guess you missed post #6 four days ago: https://forum.videohelp.com/threads/351015-How-to-Improve-Commercial-DVD-with-Avisynth-...=1#post2201102

[VideoFanatic]

So don't use RemoveSpots if you don't like it. Or use different settings. All spot/dirt removal filters will damage the picture to an extent, just like all denoising filters will soften the video to an extent

You shouldn't use spot/dirt removal filters on interlaced video, while it's interlaced (you need to separate odd/even fields before applying, then re-weave, or apply it to progressive video since you are using QTGMC)

Thanks. I put it after QTGMC and I no longer get the blurring and it looks the same quality as without RemoveSpots and it still lowers the file size a bit which is a good thing.

[jagabo]

RemoveSpots() after QTGMC() still causes some blurring and artifacting. It's most noticeable on the referee's shirt:

Before RemoveSpots (4x point resize):


after RemoveSpots (4x point resize):


If you interleave the before and after images and zoom in you'll see lots of bluring and loss of detail in the crowd in the background.

RemoveSpotsMC() was much closer to the original (and still removes spots in spotty videos).

The truth is though, If you're going to make it back into a DVD, I think you're wasting your time "fixing" this video. The blocks and DCT ringing are going to come back when you encode as MPEG 2. The other problems are minor and you'll lose detail in any attempt to fix them.

[VideoFanatic]

I have just got round to trying it now. Could you please tell me where I can get the RemoveSpotsMC() script?

[VideoFanatic]

In the mean time, Here is a few seconds of a spotty video. I noticed an interesting phenomenon. The AddGrainC function seems to cover up the remaining spots that RemoveSpots didn't remove. Please try this script yourself with and without AddGrainC. I can see a clear difference on my 37" TV - AddGrainC covers up more spots! Can anyone explain this? Also if I have RemoveSpots and AddgrainC before McTemporalDenoise then even more spots are removed, however I can't use RemoveSpots while the footage is interlaced because RemoveSpots would then cause blurring.

setmtmode(5,9)
Mpeg2Source("H:\video.d2v", CPU=6)
setmtmode(2,0)

McTemporalDenoise(settings="medium", interlaced=true) 

AssumeTFF() # or AssumeBFF() depending on the source
QTGMC(Preset="Super Fast") 
RemoveSpots()
AddGrainC(var=1.0,uvar=1.0) 
Vinverse()

SeparateFields()
SelectEvery(4,0,3)
Weave()

AddBorders(8, 0, 8, 0)