so where's all the Topaz Video Enhance AI discussion?

[Selur]

A (maybe) interesting resource about BasicVSR++

that's how most folks here are using BasicVSR++

[rrats]

DVD (720p) sources

dvd is 480/576 only

[poisondeathray]

Do you think is possible to train algos with well-known movies released at 1080p ? (example: Criterion BDs)

Of course it's possible - the training code is published for most of the open source projects including BasicVSR++

But it takes a long time to train ( many GPU's help) and there are other many potential issues.

The degradation step for the LR has to be done well to include noise, MPEG2 blocking, artifacts to simulate what you see on a real DVD. A good way to do it would be to encode a real DVD for the LR.

If open source community got more involved, a larger repository could possibly be shared.

[rrats]

-> I'll send you a link to my current Hybrid dev version and my current torch addon

that seems interesting

[JoelHruska]

That's a bogus before/after.
The color changes have an affect on perceived sharpness. So it's worthless.

Ask and ye shall receive. There are no deliberate color changes in the new comparisons below. I use derainbowing filters and those do change color slightly compared to the DVD. All comparisons are different from previously shown output:

https://imgsli.com/Nzc0ODA = Credit sequence.

https://imgsli.com/Nzc0ODE = End credits sequence. Some shift in wormhole color. Not deliberate.

https://imgsli.com/Nzc0ODM

https://imgsli.com/Nzc0ODQ

https://imgsli.com/Nzc0ODU

https://imgsli.com/Nzc0ODY

https://imgsli.com/Nzc0ODc

https://imgsli.com/Nzc0ODg Seasons 1 - 3 of DS9 are absolutely riddled with rainbowing. It took me weeks to figure out a filter chain that could clean it without annihilating too much detail. My filters + the upscale leaves a bit of visible damage on the pinkish-red patch on the desk. I feel this is a very small price to pay for the improvement in absolute terms.

https://imgsli.com/Nzc0ODk Topaz struggles a lot with these longer shots where background characters have minimal pixels devoted to them.

https://imgsli.com/Nzc0OTA This is a frustrating example of a shot where TVEAI improves on the entire frame -- and the frame, nonetheless, is left less-than-great. Visible sharpness errors on cuffs and such. There's no way to improve the show to maximum potential without incurring sharpening errors in some spots, though fortunately most of it isn't too bothersome. This is older output, I've improved the issue a bit.

https://imgsli.com/Nzc0OTE Defiant firing phasers. Included to show the heavy rainbowing on the original DVD and how Topaz handles a very different kind of shot.

All of these comparisons are against output with no deliberate color changes applied.

Selur,

Clip is available here. This is my first serious experiment with color grading, so I make no promise as to the quality of the effort. The clip benefits from couch-distance viewing (not worth it here, it's only 75 seconds long, but in general). The low quality of the source make a little distance very helpful.

https://drive.google.com/file/d/11zBINTXGzqfmGTTenuZPV1GOcyGQoHKG/view?usp=sharing

[lordsmurf]

https://imgsli.com/Nzc0OTE Defiant firing phasers. Included to show the heavy rainbowing on the original DVD and how Topaz handles a very different kind of shot.

I saw Defiant, so clicked it first. Not looked at others yet.

Right away, all I see here is halo/ringing from oversharpening. Nothing else.

The rainbow removal is simple chroma NR. But if you look elsewhere, you'll see that aggressive chroma NR has wiped out fine colors. That's the tradeoff with harder chroma NR. No magic there.

I could do that 10+ years ago (better, actually). Topaz not needed.

I find the star field too unreal now, every star is now a "North Star" of brightness.

[JoelHruska]

Right away, all I see here is halo/ringing from oversharpening. Nothing else.

My tutorial will include information and examples of how people can tune these settings for themselves. If you'd like a smoother Defiant, there are other options:

https://imgsli.com/Nzc1NDE (New AviSynth encode #1 + upscale, against OG DVD)

Perhaps that's a little smooth for you. It is for me. I prefer the below:

https://imgsli.com/Nzc1NDI (New AviSynth Encode #2 + upscale against OG DVD)

Here's a comparison shot of the previous Defiant shot I gave you in the earlier post against this new one:

https://imgsli.com/Nzc1Mzg

The difference in these images is coming from AviSynth, not Topaz. I mostly control sharpening there. You want smoother? No problem. Fewer stars? Dimmer stars? No problem. It's all tunable and I plan to tell people what to tune in order to make whatever adjustments they want to make. I'm still testing various things myself.

If what people want is just an AviSynth-based improvement, that's fine too. My AviSynth scripts are tuned for later upscaling not immediate viewing, but the show can be measurably improved without upscaling at all. The red color shift can be prevented or tuned back. It's caused by a color conversion from 8-bit to 10-bit that doubles as a denoising and debanding filter (improving the phaser blasts in the process). I color correct in Resolve, but if I was planning to only run the episode through AviSynth and wanted to keep original color I would handle denoising in a different way.


https://imgsli.com/Nzc1NTc

The original DVD versus after-AviSynth. The slight detail loss on the rear-left side of the ship (visible as a bit of greebling that becomes blurred) is a result of filter settings. I have an alternate filter chain that avoids this problem, but I'm a bit limited in terms of which output I have generated at the moment (and which is suitable for viewing, since what works well for Topaz and what works well for eyeballs are often different).

Topaz responds differently depending on how a video is processed in AviSynth. Tuning my AviSynth encodes allows me to tune my Topaz output. Then I composite Topaz outputs together in Resolve. The end result is tunable at every step but AviSynth is the first tool in the workflow and has the largest impact on final output.

[lollo]

Excellent job JoelHruska!

Combine/select the 2 worlds (Topaz alone is not that good in many cases, as we all have tried out) is the right approach.

Looking forward to your next results...

[Selur]

Clip is available here.

I was interessted in a short source sample with the scene.
That clip looks to smoothed for me as it is atm.

[JoelHruska]

That clip looks to smoothed for me as it is atm.

WIP. I'm very new to blending and compositing in DaVinci, and continue to experiment with which Topaz encodes should be blended for best results. I upscaled the first 2000 frames of multiple AviSynth encodes approximately 140 times in total and continue sorting through the output. Some outputs are noisier than others and the final noise is a tunable parameter.

I can get you a source clip, yes. I'll have to do it later, but I'll upload one.

[Selur]

Thanks, no hurry.

btw. https://forum.doom9.org/showthread.php?p=1953008#post1953008 shows what you can to with BasicVSR++s restoration models on anime content that you otherwise probably had a hard time filtering.

[JoelHruska]

https://drive.google.com/file/d/1OzoW-iYop3LFJszX9EJf07O1OlDoQYjF/view?usp=sharing

First few minutes. Some scenes upscale more easily than others, even in this quick clip.

That BasicVSR++ clip is really impressive. Great restoration.

[JoelHruska]

Had a bit of a breakthrough tonight.

Improved shots here, compared against my own previous work on the left-hand side:

https://imgsli.com/Nzc4MDc

https://imgsli.com/Nzc4MDk

https://imgsli.com/Nzc4MTI

https://imgsli.com/Nzc4MTY

Finally, compared against DVD:

https://imgsli.com/Nzc4MzM

While I give up a very small amount of detail, the improvement to eyes and uniforms is far better than the trade-off. Haloing dramatically reduced in this output compared to previous.

https://imgsli.com/Nzc4MDM

No more slight moustache on Kira's upper lip and better line recovery on her uniform than previously. Lines on her hand have also been restored.

EDIT:

https://imgsli.com/Nzc5NDU

This is from Season 6, Sacrifice of Angels, against OG DVD. Still got a few details to hammer out, but this works and works well.

[sum_guy]

I've read all 13 pages of posts, I've come away from this discussion with my own conclusions. No one tool (software) will produce the desired results. Beauty is in the eye of the beholder, what looks good to 3 people, looks like crap to 3 people. Workflow is a process and that process could include almost any software to produce the beauty that is in the eye of the beholder. Is Topaz actually a machine learning process, I don't think it qualifies as that. Is Avisynth the king o the hill, yes but, only in the hands of the right script or scripter. Some people work 60 hours 6 days a week and only want a point and click solution, some people are hobbyists or retired with mountains of free time and multiple machines to test scripts on. No one tool or workflow is correct. No one person will get the desired effect with a mouse click. Do I have Topaz Video AI, yes I do, do I have Mercalli, yes I do, do I have avisynth and a bunch of precanned scripts, yes I do. I have a whole host of tools. They are all just arrows in the quiver.

Did you get a halo effect? maybe that looks incredible to the observer. Did you get a halo effect? maybe it looks like crap to the observer. I'm trying to remain neutral in the discussion. I'm not going to suggest anyone have a be all end all solution in one tool, video and life isn't like that.

You all know what looks good to you. So, lets mention upscaling. Garbage in Garbage out, simple as that. If your source material is poor, no matter what you do to it, the output will be poor. Maybe the expectation is unreasonable, 480i to 4k is simply a dream. Does the resolution need to be that high for the observer? 4k at 2500 bitrate? a 8k file with a 50,000 bitrate and it sill looks bad. Lower your expectations. There are some things that just weren't meant to happen.

Thank you all for a lively discussion.

[Cornucopia]

I think that "sums" up what most of us have been trying to say all along.

Scott

[JoelHruska]

Sum_Guy,

I agree. Even in the situations where Topaz could be considered a turnkey solution, that's not how you get *good* output from it.

https://imgsli.com/NzgwOTM

This comparison is not against the original DVD, it's against my first publicly published work from April 2020. AviSynth and Topaz were used to create the output on the left. AviSynth and Topaz were used to create the output on the right. The difference between the left and right-hand images is not the software used, it's the skill of the operator and the capabilities of the software. I've learned a great deal on this project and Topaz has gotten better as well. The scripts I'm using today are different than the ones I was using then, and I've learned much more about how to tune AviSynth to create good Topaz output. It takes time, effort, and a lot of careful tuning, and not everyone has the time or the horsepower to pull it off. I could never have done this project with a very fast PC and a lot of SSD storage. Topaz has also improved, which helps a lot.

As you say, all of these tools are arrows in the quiver. They suit different purposes. I recently did a video restoration for someone in which I didn't touch Topaz because I could find no way to goose the program into yielding good results. Every video is different. Every situation requires a different touch.

[forart.it]

https://imgsli.com/NzgwOTM

This comparison is not against the original DVD, it's against my first publicly published work from April 2020.

Yes, technologies are still evolving...

...often faster than we think:


Source: https://github.com/cszn/KAIR/

From Image Restoration Using Swin Transformer study:

We conduct experiments on three representative tasks: image super-resolution (including classical, lightweight and real-world image super-resolution), image denoising (including grayscale and color image denoising) and JPEG compression artifact reduction. Experimental results demonstrate that SwinIR outperforms state-of-the-art methods on different tasks by up to 0.14∼0.45dB, while the total number of parameters can be reduced by up to 67%.

dvd is 480/576 only

You're right, sorry: I meant 576p.

[lordsmurf]

Did you get a halo effect? maybe that looks incredible to the observer. Did you get a halo effect? maybe it looks like crap to the observer. I'm trying to remain neutral in the discussion.

No. You can be too neutral, too forgiving.

Yes, there is some subjectivity. But not everything. There has to be a baseline. Sharpening the hell out of video, where everything glows with halos, is just lousy work. There's no reason for it. There are many excuses, but it's always in effort to defend self or product, not the quality.

The only exception is art. When you intend to make it ugly.

[JoelHruska]

"Sharpening the hell out of video, where everything glows with halos, is just lousy work. There's no reason for it. There are many excuses, but it's always in effort to defend self or product, not the quality."

True. It took me two months of work to figure out how to use TVEAI to create an ultra-clear output that wasn't sharpened and, like all approaches, it remains a work in progress. I only figured it out two days ago. But advances I made this week allowed me to nuke almost all of the haloing error with no loss of visual quality.

https://imgsli.com/NzgxODk

The communicator sharpening error and uniform sharpening errors are gone, while haloing around the shoulders is much reduced. It is not completely eliminated. It is, however, effectively invisible when watching from couch distance. I've checked it. The haloing above is not visible on a 55" LG CX OLED when sitting eight feet away. After over 300 test encodes in Avisynth and several hundred TVEAI tests I'm comfortable with the overall quality of the encode and the final quality of the video after compositing. I have developed what I set out to develop: One method for handling early DS9 and one method for handling later DS9, with different AviSynth scripts and processing chains for each.

Hopefully I won't have to subdivide by season. Hoping to only need one method for S1-S3 and one for S4 - S7.

[rrats]

TVEAI

qtgmc is better that that rubbish

[Selur]

okay, folks I'm leaving this thread, so don't expect any reactions from me here anymore.

Cu Selur

[sum_guy]

"qtgmc is better that that rubbish"

Oh no, you just hurt TVEAI's feelings.

[JoelHruska]

Yeah, I think I'm done as well. Thanks to everyone for the chat.

[lollo]

It's a shame that smart guys are leaving the thread because the useless comments from most of the users, without any real contribution in term of effort and achieved results.

And I refer to both sides of the audience.

Selur and Joel provided superb and concrete results, and people common reactions are "it's crap" or "why you care, I just want a push-button doing everithyng". The first should provide "their" results, the others imho should not denigrate the efforts done.

It reminds me to the topic on the Neat Video denoiser https://forum.videohelp.com/threads/403073-Why-is-Neat-video-the-best-video-denoiser where poisondeathray and Selur posted remarkable achievements, and while the thread was closed because honestly it was a trolling excercize and a self-satisfaction for the OP, the typical comments were similar to what we can find in this thread.

I am wondering why we care to discuss about improving capturing/restoring of analog video and do not simply use a simple dvd recorded as converter, or why we want to improve bad DVD/BR video releases insted of watching them as they are, in what is supposed to be a video help forum.

I do not mean to offend anyone, just my thought.

[lordsmurf]

The first should provide "their" results, the others imho should not denigrate the efforts done.

It's a great idea, but only if you have agreed-upon sources to work from. Either something we can all download --- and from a not-crap location, ie no Mega, but mainstream reputable like Google Drive, iCloud, Dropbox. Or something we can all easily acquire.

When a "comparison" thread uses many random scenes, from many random videos, it's somewhat pointless. Nothing is actually head-to-head compared. Nobody can learn from the results. It's merely a showcase (good or bad) of user's random output from random sources.

For example, a known-flawed source, like the not-great DS9 DVDs, simply is not going to help with comparisons.
Youtube butchery is even more ridiculous as a source.

This is the same issue we have with the cherry-picked "samples" from Topaz (or other companies). It's just not real-world useful, all the way to damned deliberate BS. At least with companies, you know what the ulterior motive is.

Make sense?

[lollo]

It's a great idea, but only if you have agreed-upon sources to work from. Either something we can all download --- and from a not-crap location, ie no Mega, but mainstream reputable like Google Drive, iCloud, Dropbox. Or something we can all easily acquire.

Often it is not the case; and if a source location is not suited, asking a change to the OP is possible.

When a "comparison" thread uses many random scenes, from many random videos, it's somewhat pointless. Nothing is actually head-to-head compared. Nobody can learn from the results. It's merely a showcase (good or bad) of user's random output from random sources.

I agree, but I do not think this apply to this thread.

For example, a known-flawed source, like the not-great DS9 DVDs, simply is not going to help with comparisons.

No. A "bad" source is a good starting point, justifying why we want to restore. And Joel aptitude was clever enough to include AviSynth processing after his own discoveries and our recommendations, once we showed him that Topaz alone was not entirely adequate.

This is the same issue we have with the cherry-picked "samples" from Topaz (or other companies). It's just not real-world useful, all the way to damned deliberate BS. At least with companies, you know what the ulterior motive is.

I think that in this forum we are able to differentiate between marketing BS and real cases.

Make sense?

Yes, if you stress on "having real and reliable test cases". No if somebody considers this whole thread (and similar discussions) useless without adding his own results

[rrats]

It's a shame that smart guys are leaving the thread because the useless comments from most of the users, without any real contribution in term of effort and achieved results.

TVEAI converts interlaced video to losless RGB images before deinterlacing them. It introduces chroma upsampling artefacts

[lollo]

Yes, we know. http://www.digitalfaq.com/forum/video-restore/12192-how-make-vhs.html

I think that nobody here considers TVEAI stand-alone as a reference. It may or may not provide good results (does not in my case) in a full avs workflow

[lordsmurf]

No. A "bad" source is a good starting point,

No. You're compounding issues.

Furthermore, the issues present in a bad DVD transfer are not natural errors found on most sources. Bad DVD releases are special.

And as I wrote many posts ago, "restoring" a retail release is pointless. Whatever you do at home will eventually be far surpassed by a subsequent retail release. (And let's forego the conspiracies of "the made it bad so they could resell it later".) With few exceptions, such as censorship on latter versions, or ADD-fueled "fixes" by George Lucas.

I think that in this forum we are able to differentiate between marketing BS and real cases.

You give too much credit to casual non-posting visitors, or newbie members. They don't know what they don't know. It has to be pointed out to them. There are lots of gullible people in the world, that take things at face value. (There are also lots of kooks, who think everything is lies. Education helps you determine lies from truths. It's never all one way, or all the other. You have both good and bad players. For example, Topaz samples are obvious cherry-picked BS if you have video knowledge.)

Yes, if you stress on "having real and reliable test cases".

That's it. Scientific method requires some constants, it can't be just variables.

[lollo]

No. You're compounding issues.
Furthermore, the issues present in a bad DVD transfer are not natural errors found on most sources. Bad DVD releases are special.

I need to practice to improve my scripts.
Without Selur effort to "deblock" a video of mine, I was not thinking to this operation as an option for my filtering, because it was an analog video. But its origin is from a low quality highly compressed dvb-s broadcast, so his approach was appropriate.
Just to say that any reasonable video source with problems may bring more expertize.

You give too much credit to casual non-posting visitors, or newbie members.

Maybe. But the best teaching is to show real examples. However, I also understand that not everybody is willing to spend time and effort for that, so I accept that they should trust opinion from knowledgeable members. Often it is not the case, but, as you said, that's live!