Improving the quality of old family videos.

[echocoder]

I am attaching the details of the video I am working on, it's interlaced and distorted which the info does reveal i guess. It's from the 1980ish, i wasn't born so not sure. Please do advice. I want to upscale to watch on TV, maybe upscale to HD from the current resolution.... The info is generated from MediaInfo
General
Complete name :
Format : CDXA/MPEG-PS
File size : 782 MiB
Duration : 1 h 18 min
Overall bit rate mode : Variable
Overall bit rate : 1 411 kb/s

Video
ID : 224 (0xE0)
Format : MPEG Video
Format version : Version 1
Format settings : BVOP
Format settings, BVOP : Yes
Format settings, Matrix : Default
Format settings, GOP : Variable
Duration : 1 h 18 min
Bit rate mode : Variable
Bit rate : 1 150 kb/s
Width : 352 pixels
Height : 288 pixels
Display aspect ratio : 1.056
Frame rate : 25.000 FPS
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.454
Time code of first frame : 00:00:00:00
Time code source : Group of pictures header
GOP, Open/Closed : Open
GOP, Open/Closed of first frame : Closed
Stream size : 642 MiB (82%)

Audio
ID : 192 (0xC0)
Format : MPEG Audio
Format version : Version 1
Format profile : Layer 2
Duration : 1 h 17 min
Bit rate mode : Constant
Bit rate : 224 kb/s
Channel(s) : 2 channels
Sampling rate : 44.1 kHz
Frame rate : 38.281 FPS (1152 SPF)
Compression mode : Lossy
Delay relative to video : -85 ms
Stream size : 124 MiB (16%)

[johnmeyer]

Your TV will upscale. For VHS video or SD DVD, I don't think you'll notice much difference by doing the up-scaling in software prior to viewing.

[echocoder]

I would like to make a note of some of the issues other than upscaling that I would like to get rid of if possible. They are :
1. Multiple random horizontal while lines on the video
2. Visible multi colored distortion at the bottom of screen
3. Lack of contrast and sharpness
4. Horizontal fluctuations of intensity resulting in dark horizontal lines sometimes
I am open to reading the discs again with better software ( I have used ImgBurn or copied the .dat files initially) Please so advice further on this restoration process.
Thank You for the reply.

[davexnet]

Post some video sample(s).. Are you talking about conversions from VHS?

[lollo]

Your TV will upscale. For VHS video or SD DVD, I don't think you'll notice much difference by doing the up-scaling in software prior to viewing.

I am not a big fan of upscale, but, once more, like for deinterlacing, it dependes on who does the better job between the software and the TV.
Recently a user posted a comparison for a PC visualization: https://forum.videohelp.com/threads/410359-Help-with-VHS-deinterlace?p=2697515&viewful...=1#post2697515

Please so advice further on this restoration process.

And on what we should base our advice, on a MediaInfo report?

Post a sample of your capture, together with a description of your workflow

[TubeBar]

IMO upscaling is never good for archival purposes or future proof. As hardware on-the-fly upscaling improves you get better upscale. I suppose you can keep the original source and upscale a copy to watch, chances are that upscaled copy will stick around forever and archive source long forgotten about. Technology changes fast.

[echocoder]

The TV definitely did some upscaling and retouching by itself while I was playing using VLC through HDMI. But I still want to improve the video quality and remove the obvious like the horizontal white lines and the obnoxious distortion line at the bottom of all the frames. A part of the video was also lost because it greyed out into some weird black and white patterns

Sample of the video : https://files.videohelp.com/u/306690/sample.mkv

The obnoxious state of the video where it has been erased over time, maybe unrecoverable but the next part of the video is alright.

[Attachment 72540 - Click to enlarge]

[lollo]

IMO upscaling is never good for archival purposes or future proof.

Yes. Always archive the raw 4:2:2 YUV lossless capture without any processing.

[lollo]

But I still want to improve the video quality

Do just a standard AviSynth denoise/sharpening

and remove the obvious like the horizontal white lines and the obnoxious distortion line at the bottom of all the frames.

Impossible, you can just mask or interpolate few good lines; here some hints from master jagabo:

https://forum.videohelp.com/threads/358000-How-to-remove-horizontal-white-lines-in-Vir...=1#post2259432

https://forum.videohelp.com/threads/397742-This-is-working-for-me-PAL-VHS-capture-unco...e2#post2587587

[johnmeyer]

Having now seen a video clip, upscaling is totally the wrong thing to be thinking about. The quality of the video and the enjoyment of watching it will not be improved even 1% by clever upscaling, but it WILL be improved if you denoise, as Lollo has already stated. Done properly, denoising will make a massive improvement with this clip.

[echocoder]

Thanks for the advice, I will try them and let you know the result. As for the part which has been blacked-out over time and has become like the image I have uploaded, I think there's nothing more to be done there citing the phrase 'garbage in garbage out'.
The media info report said this was a lossy clip, but this is the .dat file I found on the VCD, is there a way to get the lossless file from the VCD ? I have a standard laptop CD/DVD reader that I have used so long.
Also is there any improvements that can be suggested for the color correction or improvement in the color to make them more vibrant in the video?
I am also uploading some images of the random white lines I am talking about just for further clarification.

[Attachment 72563 - Click to enlarge]
[Attachment 72564 - Click to enlarge]
P.S. Also in these images you will see some distortions of color and the video as a whole, if possible please do comment on them too.
Thank You

[Alwyn]

I understand the general philosophy here is for free software, but I put up a Neat Video example to show, for a few bucks, what can be achieved with a few clicks in a total WYSIWYG environment. IMO it really is killer software for noise that is easy to use.

The first is using the MSU denoiser in VDub.

The second is using Neat Video for denoising.

I "cropped all the cr@p" and dropped the saturation a tad.

[davexnet]

You can clean it, and lighten it a little; probably something like VidCoder would work, it has crop/resize and some noise removal filters.
For this version I used a simple Avisynth script, improved the levels and use some temporal and chroma noise reduction

Those horizontal noise lines were caused by damage to the tape from which this was captured

[Alwyn]

Pardon me for being frank, Dave, but those borders have got to go. The asymmetry is very off-putting, not to mention the fuzzy left edge.

[davexnet]

You're right, that's why I mentioned vidcoder and crop/resize.
The MSU version looks pretty good, I had forgotten about this set of filters until very recently

[Alwyn]

Cheers Dave.

[echocoder]

You guys were working at the clip and made it awesome while I was pleasing myself with results from inbuilt filters of Avidemux cause when I tried using Avisynth, the steep learning curve made me cry within half an hour. Over free software options the learn curve seems too steep. But I must say that you guys have shown your experience and left me in awe of your skills through these sample videos above. Maybe I need a software with a learning curve not as steep as Avisynth and better documented and it being free will be a cherry on the top.
Cheers Guys, already learning and knowing a lot on the forum.

[Alwyn]

@echocoder, your wish is my command:

http://aaproductions.net/virtualdub2.htm

Everything I did above was in VDub2 32bit. I had to "install" the MSU Denoiser and Colormill, but I've put in instructions on how to do that. Pipe up if you have any questions.

[lollo]

IMO it really is killer software for noise that is easy to use.

Be careful with your usage of Neat Video, because details may be erased.

A comparison between original and Neat Video:

https://imgsli.com/MTkzMDU1

https://imgsli.com/MTkzMDYx

Same frames processed with 1-line not optimized AviSynth code:

https://imgsli.com/MTkzMDU4

https://imgsli.com/MTkzMDYz

I also have the impression that you are introducing a distortion of the proportions: https://imgsli.com/MTkzMDYw but I did not deeply checked.

Not a criticism, just sharing impressions

[Alwyn]

Be careful with your usage of Neat Video, because details may be erased.

Thank you for that.

Lollo, what did you expect? Of course you are going to lose data with such hard denoising. You spend too much time analysing single frames. People don't look at single frames when they are watching a video. A couple of fingers out of place a 25fps won't make a hill of beans difference to the overall experience.

Here's a superimposition:


[Attachment 72573 - Click to enlarge]

I'd suggest you do a more in-depth "I did not deeply checked".

BTW, I have assumed it's 4:3. The heads look too stretched to be 1.056 and 352x288 just happens to be half 704x576... More than a coincidence, methinks.

And when you have published your comprehensive AVISynth 101, I'll proofread it for you. AKA read post #17.

[lollo]

Lollo, what did you expect?

A better denoise.

NeatVideo in the right hand is powerful, but not with a few clicks. And there is nothing that Neat Video can do that cannot be beaten by AviSynth / VapourSynth.

Master poisondeathray gave demonstrations of how NeatVideo should be used for optimal results, and how the simplest AviSynth denoise can otherwise outperform it: https://forum.videohelp.com/threads/403073-Why-is-Neat-video-the-best-video-denoiser

Instead of planning a proofreading of a guide that I never published, better check what already exist in the forum

[Selur]

Just for the fun of it, here is what scunet + spotless do,...

[lollo]

Just for the fun of it, here is what scunet + spotless do,...

Nice, but Spotless, although powerful, tends to erase objects. To avoid mutilating people like Alwin does a mask is required, but I am sure you already know the story:
https://forum.doom9.org/showpost.php?p=1944708&postcount=101

Is there any reason for doubling the framerate in your file?

[Alwyn]

Nice

You need your glasses checked. There's nothing nice about that at all. In fact it is downright awful. Dave's and my attempts are so much better it's not funny.

To avoid mutilating people like Alwin does

Factual example please.

[poisondeathray]

The media info report said this was a lossy clip, but this is the .dat file I found on the VCD, is there a way to get the lossless file from the VCD ? I have a standard laptop CD/DVD reader that I have used so long.

VCD is already digital. The MPEG1 file is already the best starting point you're going to get from a VCD source

I am also uploading some images of the random white lines I am talking about just for further clarification.

How "random?" 1 frame ? 17 frames ? Do they "move" or static? ie. What is the distribution within a frame, and over time ? Are there any good frames in that scene ?

The characteristics are important as there can be different approaches to different types and distributions of defects. Some of them might not be "fixable", some of them you might need manual fixes (photoshop), some of them might be able to be aided by scripts - it really depends on the problem characteristics

A Video sample is much better than most people attempting to describe something....

P.S. Also in these images you will see some distortions of color and the video as a whole, if possible please do comment on them too.

Same thing - does it change per scene ? Is it consistent a certain tone or color cast ? One looks light blue... But are there other colors? Does it randomly change etc...

A Video sample is much better than most people attempting to describe something....

[Alwyn]

PDR, I don't have one to experiment with.

Is it possible to smart-copy a portion of a CDXA video such as those that echocoder has in AVIDemux? I know it does for other file types.

[Selur]

Just wanted to see what scunet does on the video and mainly added spotless to lesses the artifacts at the bottom a bit.

[johnmeyer]

If you look at the doom9.org thread on Spotless that Lollo linked to, you'll see that StainlessS developed it primarily to remove large objects, like dirt on film. You can see this from the examples he used in that thread: they are movie film examples. You'll see that I posted in that thread, because I do a lot of film restoration and Spotless is actually somewhat better for some (not all) movie projects. When I say "better," it removes certain types of dirt that RemoveDirtMC misses. However, both filters most definitely remove objects -- some of them quite large -- that you don't want removed. I do a lot of restoration on old American football films, and I have to turn off the dirt removal while the ball is passed or kicked because it often removes the ball in flight.

The attempt to correct for this using masks was a noble effort, and I was cheering them on, but despite what is shown in the thread, the masks never worked very well.

As for using Spotless for standard video denoising, it is the wrong tool for that job. MDegrain or KNLMeans would be better tools to try. I have never been a fan of Neat, but it sounds like they've improved it. I haven't used it for at least five years, but back when I last tried it, no matter what settings I used, it removed too much detail. I still believe that people like Neat for one of two reasons: it is easier to set up and use than AVISynth and its plugins (as others have already pointed out); and it is quite capable of removing ALL noise. The neophyte turns up the noise reduction until the noise disappears and is so delighted with that result that they often don't notice all the details that have been removed.

AVISynth plugins always seem to provide a better balance between noise reduction and detail retention.

[lollo]

Nice

You need your glasses checked. There's nothing nice about that at all. In fact it is downright awful. Dave's and my attempts are so much better it's not funny.

It was a nice attempt using an alternative approach, not a nice result, from somebody who does not need to prove anything, especially to a minimalist beginner.

[lollo]

John, Selur was using Spotless trying to reduce the white bands defect, not to denoise. Impossible task, anyhow.

I used RemoveDirtMC, and all its variants, intensively to soften scratches in the video masters I capture, and, as you, faced many times the side effect of removing real small object. Spotless, in combination with DeltaRestore is a big step forward. Thanks to the doom9’s community (you included) for this great piece of software!

Concerning denoising, NeatVideo is not really a few clicks tool, and requires the same effort than AviSynth/VapourSynth to produce same or inferior results. When not used in the proper way produces mediocre results. Even because in that conditions looks good to newbies because somehow remove all the grain with the noise, and we all know that some grain and some noise is essential to keep a “natural” look of the analog captured videos.