How to generally remove the large amount of artifacts from this old video?

[DMFOREVER98]

I have this old video taken from YouTube with a series of huge artifacts + low resolution, I do not expect miracles because the state of the video is bad, however, is it possible to clean rainbow artifacts?, blocking, etc., etc. as far as possible?


[Attachment 81039 - Click to enlarge]

[Attachment 81040 - Click to enlarge]

[Attachment 81041 - Click to enlarge]

General
General
Complete name : test video.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/avc1/mp41)
File size : 8,99 MiB
Duration : 6 min 51 s
Overall bit rate : 183 kb/s
Writing application : Lavf58.76.100

Vídeo
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L1.3
Format settings : CABAC / 3 Ref Frames
Format settings, CABAC : Sí
Format settings, Reference frames : 3 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 6 min 51 s
Source duration : 6 min 51 s
Bit rate : 181 kb/s
Width : 320 píxeles
Height : 240 píxeles
Display aspect ratio : 4:3
Frame rate mode : Constant
Frame rate : 30,000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.078
Stream size : 8,87 MiB (99%)
Source stream size : 8,87 MiB (99%)
Writing library : x264 core 155 r2901 7d0ff22
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
mdhd_Duration : 411533
Codec configuration box : avcC

[Selur]

I doubt you can do much about such a source





The 'best' would probably be, to:
a. smooth the hell out of it
b. throw away the color and use some DeOldify, DeepEx, DDD combination to recolor it based on some reference frames if they exist.
(aside from looking for another source or applying some especially trained machine learning model, I doubt there is one atm., on it)
320x240 avc encode with a x264 version from 2019, is that your most untouched source ?

Cu Selur

[jagabo]

I concentrated mostly on reducing the chroma stripes:


[Attachment 81067 - Click to enlarge]

First 1100 frames, before/after:

[DMFOREVER98]

I doubt you can do much about such a source
The 'best' would probably be, to:
a. smooth the hell out of it
b. throw away the color and use some DeOldify, DeepEx, DDD combination to recolor it based on some reference frames if they exist.
(aside from looking for another source or applying some especially trained machine learning model, I doubt there is one atm., on it)
320x240 avc encode with a x264 version from 2019, is that your most untouched source ?

Cu Selur

Hello Selur, thanks for your reply, the video is really horrible, with respect to having the original source, I would find it difficult since I would have to contact the person who uploaded the video (who uploaded it 16 years ago) although he was active in terms of videos 2 years ago, I cannot guarantee his response.

[DMFOREVER98]

I concentrated mostly on reducing the chroma stripes:


[Attachment 81067 - Click to enlarge]

First 1100 frames, before/after:

Overall it's a fantastic rainbow removal result, what did you use jagabo?

[jagabo]

I used AviSynth:

###################################################
#
# build a mask of areas where there are 
# alternating horizontal lines
#
##################################################

function CombMask(clip v, int "threshold")
{
    threshold = default(threshold, 5)

    Subtract(v, v.blur(0.0, 1.0).Sharpen(0.0, 0.6))
    GeneralConvolution(0, "
        0  8  8  8  0
       -0 -8 -8 -8 -0 
        0  8  8  8  0
       -0 -8 -8 -8 -0
        0  8  8  8  0", chroma=false, alpha=false)
    mt_lut("x 126 - abs")
    mt_binarize(threshold) # threshold
    mt_inpand()
    mt_expand()
    mt_expand(chroma="-128")
}

##################################################


LWLibavVideoSource("test video.mp4", cache=false, prefer_hw=2) 
src = last  # remember the source

cmasku = Spline36Resize(width, 80).Tweak(sat=3).UtoY().CombMask(8).mt_expand().Blur(1.0).BilinearResize(width,height) # mask of stripes in U channel
cmaskv = Spline36Resize(width, 80).Tweak(sat=3).VtoY().CombMask(8).mt_expand().Blur(1.0).BilinearResize(width,height) # mask of stripes in V channel
cmask = Overlay(cmasku, cmaskv, mode="add").GreyScale() # add the two masks together
bc = MergeChroma(last, Spline36Resize(width, 24).Spline36Resize(width,height))  # a vertically blurred version of the chroma (reduced color stripes)
Overlay(src, bc, mask=cmask) # overlay the blurred color only where the color stripes were

SMDegrain(tr=2, thsad=1000, PreFilter=6) # temporal noise reduction
aWarpSharp2(depth=5) # a little warp sharpening
Sharpen(0.3) # a little overall sharpening

[DMFOREVER98]

I used AviSynth:

###################################################
#
# build a mask of areas where there are 
# alternating horizontal lines
#
##################################################

function CombMask(clip v, int "threshold")
{
    threshold = default(threshold, 5)

    Subtract(v, v.blur(0.0, 1.0).Sharpen(0.0, 0.6))
    GeneralConvolution(0, "
        0  8  8  8  0
       -0 -8 -8 -8 -0 
        0  8  8  8  0
       -0 -8 -8 -8 -0
        0  8  8  8  0", chroma=false, alpha=false)
    mt_lut("x 126 - abs")
    mt_binarize(threshold) # threshold
    mt_inpand()
    mt_expand()
    mt_expand(chroma="-128")
}

##################################################


LWLibavVideoSource("test video.mp4", cache=false, prefer_hw=2) 
src = last  # remember the source

cmasku = Spline36Resize(width, 80).Tweak(sat=3).UtoY().CombMask(8).mt_expand().Blur(1.0).BilinearResize(width,height) # mask of stripes in U channel
cmaskv = Spline36Resize(width, 80).Tweak(sat=3).VtoY().CombMask(8).mt_expand().Blur(1.0).BilinearResize(width,height) # mask of stripes in V channel
cmask = Overlay(cmasku, cmaskv, mode="add").GreyScale() # add the two masks together
bc = MergeChroma(last, Spline36Resize(width, 24).Spline36Resize(width,height))  # a vertically blurred version of the chroma (reduced color stripes)
Overlay(src, bc, mask=cmask) # overlay the blurred color only where the color stripes were

SMDegrain(tr=2, thsad=1000, PreFilter=6) # temporal noise reduction
aWarpSharp2(depth=5) # a little warp sharpening
Sharpen(0.3) # a little overall sharpening

Thanks jacobo, it would be too much to ask if you could process the entire video, I'm using StaxRip and I have no idea how to use Avisynth for this type of things, greetings.

[Selur]

if it's just the rainbows,...
try MiniDeen with high thresholds and radius for chroma,..
somehing like:

MiniDeen(clip=clip, radius=[1,7,7], threshold=[10,100,100])

in Vapoursynth should translate to:

MiniDeen(radiusY=1, radiusUV=7, thrY=1, thrUV=100, y=3, u=3, v=3)

in Avisynth.

Adjusting the thresholds and may be boosting the saturation after MiniDeen should work fine.
(image was taken without saturation boost)
According to https://staxrip.readthedocs.io/generated/tools.html, StaxRip should support MiniDeen.

[jagabo]

Here's the full video.

[DMFOREVER98]

Thanks, Selur-jagabo, what filters could I use to maybe "polish" the image, if you will? Something to improve the blocking artifacts perhaps, but without ruining the image more than it is, and if possible add some sharpening.

[jagabo]

Keep in mind that h.264 decoding already applies deblocking filters. So to get more deblocking you will have to use very strong settings that will blur away more detail and leave an even fuzzier picture. AviSynth has Deblock() and Deblock_QED(). The strong temporal filter I applied earlier, SMDegrain(), reduces some of the blocking and noise. Antialiasing filters can smooth out some rough edges, Santiag(), AAA(), etc. Then there are sharpening filters like aWarpSharp2(), Sharpen(), LimitedSharpenFaster(), etc.