Need help removing some artefacts

[kotyczka]

Hi, I am preparing a movie for DVD publishing. Quality is OK,
except at the very beginning, where I have one scene (company
logo) with heavy artefacts. Unfortunately there is no better
footage availible.

Therefore I had the idea to reduce the artefacts with Avisynth.
But since I am not so familiar with Avisynth I have no idea,
which filters might help me.

I tried msu_deblock and msu_smartdeblock as well as antialiasing.
But do not know which parameters might be useful and so I do not
see any progress in the results.

It would be very nice, if someone with more knowledge of Avisynth
could take a look at a short clip showing the problem and suggest
me what to do to reduce the artefacts.

The file is about 7 MB: https://rapidshare.com/files/2704180024/Logo.zip

[sanlyn]

Not enough info to address your problem. What kind of artifacts? What is your video source, format, etc.?

[themaster1]

If mid/heavy blocking if your problem and if you work with avisynth perhaps you should try this script and there is also this one and this one for noise

[sanlyn]

The file at your link is an MPG. AViSynth won't work directly on MPG without using certain import procedures. In any case, all MSU filters work on AVI files in an RGB32 colorspace. They are designed to work in VirtrualDub (which uses RGB32) and their filters offer an easy-to-see GUI interface in VirtualDub. The MSU script for AviSynth shown on the MSU_Deblock web page will not work with your video.

The original video has more than just block noise. It has bad flutter and many other problems.

[mgh]

you can use smart skating rink filter in virtualdub
download from link at
http://www.infognition.com/VirtualDubFilters/
there are five sliders, move all to the right, there are five check boxes, check all of them.
if you are not satisfied, you can add multiple instances of the filter
finally you can frameserve from virtualdub to your mpg2 encoder

[sanlyn]

The video has bad flicker and severe temporal noise along the top and top-right. A spatial filter smooths the blocks, won't help damage like this. Anti-flicker won't help, because most of the flicker is in the upper right, the rest of the clip looks stable.

[kotyczka]

The video has bad flicker and severe temporal noise along the top and top-right. A spatial filter smooths the blocks, won't help damage like this. Anti-flicker won't help, because most of the flicker is in the upper right, the rest of the clip looks stable.

Thanks for your attention.
If I understand you right it wil be quite impossible to improve this kind of flicker.
I asked the DVD publisher, I am working for, for an alternative source of this short
scene. They sent me three files, only one of which is usefull. The other two are even
worse.
In this alternative source the sky (esp. at the right) looks much better. But the
triangular logo with the japanese letters looks smashed, there are no clear contours
any more. Is there any chance to improve that kind of bad footage?
If you wish, I can upload the second file tonight.

[kotyczka]

RapidShare is imposing a 5-minute wait at the download link.

Sorry for the delay. I wasn't aware that it's 5 minutes. First I had tried at megaupload, but that upload failed.

[kotyczka]

The file at your link is an MPG (PAL DVD 16:9). AViSynth won't work directly on MPG without using certain import procedures. In any case, all MSU filters work on AVI files in an RGB32 colorspace. They are designed to work in VirtrualDub (which uses RGB32) and their filters offer an easy-to-see GUI interface in VirtualDub. The MSU script for AviSynth shown on the MSU_Deblock web page will not work with your video.

The original video has more than just block noise. It has bad flutter and many other problems.

ACK about the problems.

I first generated a .d2v file for the .mpg with DGIndex. Then in my .avs script I opened the .d2v
with mpeg2source, converted it to RGB32 and then applied the MSU_smart_deblock filter.
I think that I see a difference to the MPG, but it is not really an improvement.
Is the script complete wrong?

I am not sure about deinterlacing. However mpeg2source with info=1 tells me that the
scene is already progressive. Immediately after the scene it changes to interlaced.

[sanlyn]

I am not sure about deinterlacing. However mpeg2source with info=1 tells me that the
scene is already progressive. Immediately after the scene it changes to interlaced.

My GSpot program tells me the clip is interlaced. But you were correct to use DGIndex. The script to convert to RGB32 should include these statements:

[jagabo]

You can use DgMpegDec's built in deblocker.

Mpeg2Source("filename.d2v", CPU=6)

[sanlyn]

As you watch the clip frame-by-frame, whether loading with AviSYnth or directly with VirtualDubMPEG, you'll see the horizon change with every frame. It gets bright, dark, darker. It's easy to see in the upper right corner. This light-to-dark happens every 3 or 4 frames. At the horizon's darkest point, the entire frame is darker by about 10 RGB values. Also, the horizon isn't just filled with block noise. Everything in the sky changes shape with every frame.

If you view the clip using SeparateFrames, you'll see that each frame is a few pixels shorter than the previous frame. Every two frames, the image "hops" up, then down, then up, down, on and on. Look at the rocks frame by frame and you'll see that they are moving up and down.

The clip plays like damaged film or damaged tape, or defective playback if tape. Where did this clip come from?.

[sanlyn]

Thanks for your attention.
If I understand you right it wil be quite impossible to improve this kind of flicker.
I asked the DVD publisher, I am working for, for an alternative source of this short
scene. They sent me three files, only one of which is usefull. The other two are even
worse.
In this alternative source the sky (esp. at the right) looks much better. But the
triangular logo with the japanese letters looks smashed, there are no clear contours
any more. Is there any chance to improve that kind of bad footage?
If you wish, I can upload the second file tonight.

No problem about the slow download, it happens all the time.

There is flicker in the video (every 3rd or 4th frame darkens), but it's more than that. The sky has no constant or repeatable detail anywhere, so no processor would really know what the "correct" image is supposed to look like in that area. There are complex ways of doing a cleanup, frame by frame, but it would take forever (almost literally). Many filters would smooth the sky, but it didn't stop the rapid changing of shapes and brightness. What we're talking about isn't "artifacts", it's damage.

Don't know what you mean by "mashed". Can you post a single frame capture of what it looks like?

[poisondeathray]

You can use various deflicker filters, like deflicker, msu deflicker, timelapseDF ,didee's made a bunch of custom ones as well (look for them at doom9), or Donald Graft's deflicker in vdub

But they seem to generate various artifacts in the sky, like posterization, but they do handle the flicker in the lower half of the frame fairly well

So one approach is to composite a "clean sky" over a deflickered base. It doesn't take that long in After Effects, and you don't have to do it frame by frame (there is keyframe interpolation). You just do a rough feathered garbage mask over the crashing waves. This took ~30 min total

I used DFTTest() to generate the "clean sky" for 1 frame, but also cleaned up the white junk in the upper right hand by cloning it out. I used Donald Graft's deflicker in vdub for the "base" clip

You could also fix the rough "halo" edge around the logo (that's in the original source), but that's a bit harder to do

[kotyczka]

There is flicker in the video (every 3rd or 4th frame darkens), but it's more than that. The sky has no constant or repeatable detail anywhere, so no processor would really know what the "correct" image is supposed to look like in that area. There are complex ways of doing a cleanup, frame by frame, but it would take forever (almost literally). Many filters would smooth the sky, but it didn't stop the rapid changing of shapes and brightness. What we're talking about isn't "artifacts", it's damage.
Don't know what you mean by "smashed". Can you post a single frame capture of what it looks like?

Well, in the japanese letters the interior is not dark (as it should be), but also
almost completely white. As if you increase the contrast too much and the light colors all become white.
See image 1 vs. 2.
The original clip you will find at http://www.mediafire.com/?vo74tjlex6olci2 (12MB)
and the other clip at http://www.mediafire.com/?vsqcdlshdum99d8 (20 MB).

[jagabo]

Something like:

Mpeg2Source("Logo.d2v", CPU=6)
Crop(0,72,-0,-70)
Stab()
Crop(2,2,-2,-2)
TTempSmooth(maxr=7, lthresh=32, strength=5)
AddBorders(2,74,2,72)

Still flickers in the upper right corner. I'd probably add a blur() to soften those over sharpened edges a bit.

[sanlyn]

Great work, poison! I see fixes like this and it tempts me to latch onto Adobe. But I'd have to go Vista to use anything new from the big "A". It would take a week to do this with free software, if not longer.

[kotyczka]

You can use various deflicker filters, like deflicker, msu deflicker, timelapseDF ,didee's made a bunch of custom ones as well (look for them at doom9), or Donald Graft's deflicker in vdub

But they seem to generate various artifacts in the sky, like posterization, but they do handle the flicker in the lower half of the frame fairly well

So one approach is to composite a "clean sky" over a deflickered base. It doesn't take that long in After Effects, and you don't have to do it frame by frame (there is keyframe interpolation). You just do a rough feathered garbage mask over the crashing waves. This took ~30 min total

I used DFTTest() to generate the "clean sky" for 1 frame, but also cleaned up the white junk in the upper right hand by cloning it out. I used Donald Graft's deflicker in vdub for the "base" clip

You could also fix the rough "halo" edge around the logo (that's in the original source), but that's a bit harder to do

Thank you, poisondeathray, that looks great!

May I ask you to do it again with the whole scene. I only sent a part of it, just to illustrate the problem.
The whole clip is about double the size of the one I already have sent.
If you are so kind, may I also send you the parameters, how I wish to encode the clip in TMPGenc Xpress?
These are:
MPEG2-Video
Profile & Level: MP@ML
Size: 720 x 576
Aspect ratio: Display 16 : 9
Frame rate: 25 fps
Rate control mode: VBR (Average bitrate), Average bitrate 8500 kbps, Maximum bitrate 9000 kbps, Minimum bitrate 2000 kbps, padding allowed
VBV buffer size: 224 kB
Video System: PAL
DC precition: 9 bit
Display mode: progressive
Motion search precision: highest
GOP I/B/P: 1/5/2, max GOP length: 15, closed GOPS, sequence header for each GOP: yes, Detect scene change: yes
Quantization matrix: Default
Other: 4 check buttons checked (default settings)

The target file shall contain exactly 337 frames. The upper and lower black bars should be very black
(RGB(0,0,0)). Furthermore it would be great to adapt brightness/contrast so, that the black parts in the very center at the beginning and at the end of the clip are also true black.

The whole source file (13 MB) is now at http://www.mediafire.com/?vo74tjlex6olci2, from that file the frames 0-336 should be used.
The alternative source (20 MB) is at http://www.mediafire.com/?vsqcdlshdum99d8, here it's frames 0-334 (2 frames are missing). But using this clip there should be a resize, such that the black bars left and right
are comletely removed and that the black bars at the top and bottom are exactly 70 pixels each.
But I am not sure if this clip comes into accout at all.

If you would really would make the efford again, thank you in advance. My first option would be MPEG2 result with the parameters specified above. Or something lossless. Which would be your preferred way to send me some result?

[poisondeathray]

The fade in /out make this more difficult to do

If you understood what I did - I replaced the sky with a cleaned up still . It's no longer a single masked out sky if you have fades - you have to adjust the luminance per frame, this will take quite a bit longer match it up

Sorry , it's more work than I want to do - it's not a simple "fun" project for me anymore

[kotyczka]

The fade in /out make this more difficult to do

If you understood what I did - I replaced the sky with a cleaned up still . It's no longer a single masked out sky if you have fades - you have to adjust the luminance per frame, this will take quite a bit longer match it up

Sorry , it's more work than I want to do - it's not a simple "fun" project for me anymore

OK, I see that it takes a lot of work. (BTW, I work for an official DVD release, but I am not paid for that. It's really a movie I am interested in. First it should be released just with japanese audio and without any subtitles. Therefore I took the task to produce the subtitles in three languages, which is about to get finished after 9 months.)

No, I did not yet understand all of your steps. If I got it right, you used vdub instead of avisynth. (Could it be done the same way in avisynth?)

Could you send me some kind of avisynth script you used or something similar for vdub? Maybe taking a look at it I could get an idea what you did exactly. Unfortunately I am not very used to either avisynth or vdub.

[poisondeathray]

No, I did not yet understand all of your steps. If I got it right, you used vdub instead of avisynth. (Could it be done the same way in avisynth?)

Only part of it.

You can use vdub or avisynth for eg. the cleaning up, denoising, or deflickering part. But to make it look better you have to composite bits & pieces together.

This is tough to do in avisynth or vdub *alone* . I used compositing tools (after effects - it's similar to photoshop , except for video). Think of compositing as putting together a puzzle. You've seen video's where you replace someone face with someone else's face - it's the same idea. In this video I used the cleaned up sky from 1 still and pasted the bottom half of the deflickered video on top of it using masks

Another reason you want to use masks, is to apply filters selectively to certain areas. If you use a strong denoiser, you degrade the entire image. For example if you use a strong denoiser and deblocker to treat the sky, the waves and rocks also become blurry. So if you want to apply just to a small section or the sky, you can apply through masks. You can do simple masks or even use stackvertical/horizontal + crops in avisynth too, but it's much much much more difficult , especially when the masks animate (moving masks). For example, you need to animate the mask to follow the waves. This would take days to do in avisynth accurately frame by frame, but takes minutes in AE.

I used vdub (donald graft's deflicker) for only part of the workflow - to generate a deflickered base clip. Try it out for yourself, it works ok on the bottom part, but there are artifacts remaining in the sky, so i cut & pasted parts of the video together using masks.

But all deflickering plugins/methods (I listed some others above you can try), they all have problems with fade in/outs. This makes it even more difficult - you often have to redo or retime the fade, or process it in segments (with/without filter)

[jagabo]

Try this:

mask=ImageSource("mask.png")
Mpeg2Source("Logo.d2v", CPU=6)
Crop(0,72,-0,-70)
Stab()
Crop(2,2,-2,-2)
v1=TTempSmooth(maxr=7, lthresh=32, cthresh=32, strength=5)
v2=TTempSmooth(maxr=7, lthresh=128, cthresh=128, strength=5)
Overlay(v1, v2, 0, 0, mask)
AddBorders(2,74,2,72)

With this mask:



Fine tune the lthresh and cthresh values to your liking.

[poisondeathray]

I tried your TMPGEnc encoder settings, but they result in "blockier" macroblock gradients in the sky (it's one of the weaknesses of TMPGEnc's MPEG2 encoder) . Mainconcept Reference or CCE tend to do better MPEG2 encodes along gradients with minimal preprocessing or dithering, or without playing with matrices

Note there's also a few frame jumps in your source (like missing frames)

[kotyczka]

I tried your TMPGEnc encoder settings, but they result in "blockier" macroblock gradients in the sky (it's one of the weaknesses of TMPGEnc's MPEG2 encoder) . Mainconcept Reference or CCE tend to do better MPEG2 encodes along gradients with minimal preprocessing or dithering, or without playing with matrices
Note there's also a few frame jumps in your source (like missing frames)

poisondeathray, I am really deep impressed, thank you so much for your help.
Unfortunaely I have only TMPGEnc availible. I've read that it tends to produce
blockier results. May I ask which encode you have used for hte attached intro.m2v?
I have no knowledge about the history of this clip, I only got is "as is".
So I cannot tell anything about the frame jumps you mentioned. Maybe it was
converted from NTSC to PAL?
I do not know if I may ask this (after so much help): The rest of the movie
is encoded with closed GOPs. I would like to make the final movie (which I
will give to the DVD publisher for authoring) look like from one source.
Your intro.m2v has open GOPs. Could you render it (one very last time) again
with closed GOPs?

[manono]

Unfortunaely I have only TMPGEnc availible.

HCEnc is free and better than TMPGEnc. Faster, too.

You can mix open and closed GOPs. I doubt anyone would ever notice. Or care. Open GOP is better anyway, in my opinion, unless you have multiple angles or something. It produces slightly better quality for the same file size.

I don't think much of your encoder settings anyway. No one should use anything other than 10-bit DC Precision unless maybe the quality is borderline bad to begin with. I don't believe TMPGenc allows for the use of different quantisation matrices, which is one good reason not to use it.

[jagabo]

So I cannot tell anything about the frame jumps you mentioned. Maybe it was
converted from NTSC to PAL?

This clip http://www.mediafire.com/?vo74tjlex6olci2 has both missing and duplicate frames. This clip http://www.mediafire.com/?vsqcdlshdum99d8 doesn't. The first clip also has blown out brights and darks, is much blurrier, and has different colors. I hope the rest of the movie isn't like the first clip.

[poisondeathray]

As manono said, open GOP is better in every way (except closed required for multiangle) . It's more efficient (better quality at a given bitrate), and completely compatible with MPEG2 DVD specs. You can append it to another section with closed. Are you sure you want it closed ?

I used Mainconcept Reference in this example.

Manono suggested HCEnc , but one of HCEnc's weaknesses IMO is gradients as well.

In HCEnc, you can play with AQ settings or matrices , but they have little effect on the blocking that occurs in the "cleaned" sky (or any gradient). It's AQ is ported from x264 (AVC encoder), but definitely does not work as well. Mainconcept oddly enough is the only MPEG2 encoder that does well with minimal dithering or preprocessing . CCE (which IMO is overall the best MPEG2 encoder) even doesn't do as well. I say "oddly enough" because Mainconcept's AVC encoder does the exact opposite - massive banding and posterization along gradients.

The observations are repeatable and consistent and accross different sources. I can post examples if you want . Note there are other important criteria to look at when evaulating quality, but I just mentioned gradients here because it pertained to the cleaned sky, and TMPGEnc doesn't do very well.

This clip http://www.mediafire.com/?vo74tjlex6olci2 has both missing and duplicate frames. This clip http://www.mediafire.com/?vsqcdlshdum99d8 doesn't. The first clip also has blown out brights and darks, is much blurrier, and has different colors. I hope the rest of the movie isn't like the first clip.

Jagabo is correct. There is a missing frame is beween 49,50 and there are duplicates at 36,37 ; 52,53 ; 60,61 ; 84,85 ; 109,110, 133,134 ; 158,159 ; 207,208 ; 255,256 ; 280,281.

I don't know if this piece has audio (maybe there is waves crashing ?) , but I can delete the dupes. This will look better (less choppy) but will change the timing. If the audio is unrelated, then you can probably remove the dupes without problem, or even insert longer black fade in/out.

I can even generate a missing frame for you to fill in the gap

In retrospect, it might have been better to start with the 1st part of the 2nd clip, but I didn't even look at it. The levels are better, but it has other problems with a shifting logo, pixellated logo fade out, more macroblocking in the waves (macroblocking not really worse in this case, since the other clip has the waves partially blown out and there is less detail there to begin with)


EDIT: here is the clip with dupes removed, missing frame filled in (interpolated), open GOP. If you really want closed GOP, I can do that

Cheers

[kotyczka]

As manono said, open GOP is better in every way (except closed required for multiangle) . It's more efficient (better quality at a given bitrate), and completely compatible with MPEG2 DVD specs. You can append it to another section with closed. Are you sure you want it closed?

Yes, I am.

...

This clip http://www.mediafire.com/?vo74tjlex6olci2 has both missing and duplicate frames. This clip http://www.mediafire.com/?vsqcdlshdum99d8 doesn't. The first clip also has blown out brights and darks, is much blurrier, and has different colors. I hope the rest of the movie isn't like the first clip.

No, the rest of the movie is of MUCH better quality.

Jagabo is correct. There is a missing frame is beween 49,50 and there are duplicates at 36,37 ; 52,53 ; 60,61 ; 84,85 ; 109,110, 133,134 ; 158,159 ; 207,208 ; 255,256 ; 280,281.
I don't know if this piece has audio (maybe there is waves crashing ?) , but I can delete the dupes. This will look better (less choppy) but will change the timing. If the audio is unrelated, then you can probably remove the dupes without problem, or even insert longer black fade in/out.
I can even generate a missing frame for you to fill in the gap

That would make it perfect (removing double frames, filling missing frames).
However there is audio. This video had DD2.0 audio, but there was another source with weaker video but better audio (even DD5.1). I got it managed to get this video and the other audio synchronized. The audio files are already finished, I would not like to touch them any more. Therefore the intro clip should be excatly 337 frames. If some duplicate frames will be removed (about 9 or 10) I would just add as much black frames at the end of the intro. The triangular logo will fade earlier then, but that will be OK.
And yes, please with closed GOPs. The rest of the movie is encoded with a average video bitrate of 8500 kbps (the whole movie is only 52 minutes). I hope with such high bitrates the disadvanteges of closed GOPs will not be too bad.

In retrospect, it might have been better to start with the 1st part of the 2nd clip, but I didn't even look at it. The levels are better, but it has other problems with a shifting logo, pixellated logo fade out, more macroblocking in the waves (macroblocking not really worse in this case, since the other clip has the waves partially blown out and there is less detail there to begin with)
EDIT: here is the clip with dupes removed, missing frame filled in (interpolated), open GOP. If you really want closed GOP, I can do that

Would be really great. As above, closed GOPS, overall 337 frames (filled with black).

[poisondeathray]

Closed GOP , removed dupes, filled missing frame, extended fade out, 337 frames

Also evened out the color flicker (there was more than just luminance flicker, some frames are more red than others)

Check if the audio matches ok , it depends how the audio matches with the logo, because some of the removed dupes are before the logo appears, some after . I'm not sure if some parts of the audio are timed with the logo or other things like waves crashing

If not, another option might be to extend the fade in instead, or a combination of beginning/end

Cheers