so where's all the Topaz Video Enhance AI discussion?

[brandon87]

I would find it hard to believe that I know about something the rest of the world doesn't know...

A month ago I read this about Topaz Video Enhance AI:
https://www.extremetech.com/extreme/314653-remastering-deep-space-nine

I found it on usenet and began testing on my little NUC Win10 box (I hate 10, I use XP for most things). Immediately I could see even at 6 sec/frame it was creating detail like magic.

A week later I spent $200 and bought the program, plus $200 for Windows 10 Pro Retail and another $800 in hardware to build a minimal gaming machine (I don't game), to give it the GPU it needed (about 10 times faster than CPU). I wanted to get the RTX 3080 but it seems mostly unavailable at this point so I settled for a GTX 1080 Ti.

After 3 weeks of testing, I created a better version of one of my most favorite movies ever, originally released at 480i letterboxed with a poor film to DVD transfer that appears to have had no digital cleaning.

I used all the tricks I know: IVTC, QTGMC and got it to 706x346p before I fed it to Topaz VEAI. I outputed it as .tiff images and fixed 949 frames with Photoshop. Most of the dirt and hair is now gone. The final version is an honest 1116x576p. I tried 720p, but it didn't look good. Still, the increase in detail is impressive. Like the DS9 article, I also saw the best results with the Gaia-CG model. If you search hard enough, you'll find the movie. I don't think I can post a link here (without getting banned).

I'm currently testing it on some live concert video, proshot and high bitrate 1280x720p, that I think I can get to 1920x1080p around 27Mbps with added visible detail without anyone knowing it was originally released as 720p. That's 150% more detail folks.

I just took at look at the lastest test- a noisy but detailed 1280x720p 50FPS video converted to 1920x1080p 50FPS with no noise and even more detail. I'm testing 3 minutes for each run. This one had settings:
run5 Theia-Detail: EU,P 100,100,30 150% 1920x1080

That's max sharpen, max restore detail, 30% reduce noise and looks awesome! I'm going to set it to do the full gig with these settings, about 90 minutes of video.

I had a theory that setting Topaz to upscale higher than the final resolution (later downscaled) might produce better results. After about 20 tests, I can say that is false. It only increases the upscaling time and size of your output.

For best results, set Topaz to scale to the desired resolution, but have it output to .tiff so you can tweak your x264 options. I wish they had a Lagarith output option, as it would allow the use of more encoders. With .tiffs, I only know of Virtualdub2 to handle those and the included x264 can be a bit tricky to use.

Topaz does work with Lagarith progressive input. Forget about using Huffyuv for input. It always thinks it's interlaced, even when it's progressive.

Now why am I one of the few using this software?

[johns0]

Nobody here cares about it cause its just another ai so called up sizer.

[manono]

With .tiffs, I only know of Virtualdub2 to handle those and the included x264 can be a bit tricky to use.

Open them in an AviSynth script with ImageSource and use any encoder that accepts AviSynth scripts as input (most of them):

Input Formats

All AviSynth color spaces are supported.

BMP and EBMP are supported natively; with DevIL 1.7.8, supported image formats are:
dds, jpg/jpe/jpeg, pcx, png, pbm/pgm/ppm, tga, tif/tiff,
gif, exr, jp2, psd, hdr, raw, sgi/bw/rgb/rgba

http://avisynth.nl/index.php/ImageSource

[lordsmurf]

Now why am I one of the few using this software?

It's overpriced garbage. The "AI" marketing suckers newbies. You've been had.

It's just an upsizer, with sharpening, and Avisynth does better.

Topaz is/was a low-end Photoshop plugin company. Their stuff works, but always worse (sometimes MUCH worse) than slightly more manual methods.

A lot of those DS9 Topaz examples really suck, lots of artifacts.

[brandon87]

lordsmurf,
Have your actually tried it? If yes, you really didn't see the improvement in detail?

Can you provide a link to a process for upsizing with Avisynth? If it can do better, I would like to use it instead.

manono,
Thank you for the info about Imagesource. With Avisynth, almost every day I learn something new.

I just read this regarding Topaz VEAI and the RTX 3080:
https://community.topazlabs.com/t/rtx-3080-is-not-work-on-video-enhance-ai/16611

If I wasn't so impressed with ML, I'd laugh. Nvidia's latest GPUs aren't supported. And Topaz doesn't even mention that on their site...

What other upsizers use AI?

[manono]

Can you provide a link to a process for upsizing with Avisynth?

I'm sure lordsmurf can, but I'll save him the trouble:

nnedi3_rpow2

[davexnet]

Can you provide a source clip and your upscaled result?

For Avisynth, usually something like this for upscaling 720p > 1080p

NNEDI3_rpow2(rfactor=2,cshift="spline36resize",fwidth=1920,fheight=1080)

This function upscales by two, then downsizes to 1080p

[lordsmurf]

Can you provide a link to a process for upsizing with Avisynth?

I'm sure lordsmurf can, but I'll save him the trouble:

nnedi3_rpow2

Just remember, that's only part of it. The Topaz app also sharpens (mostly oversharpens). So to get more apples-to-apples, you'd need to toss in some smart sharpening scripts. Unlike the rpow2, which sharpen gets used is somewhat content based.

[isapc]

The Topaz app uses A.I it isn't just another upsizer.To understand A.I you first have to understand Tensors.

https://www.youtube.com/watch?v=bpG3gqDM80w

https://www.youtube.com/watch?v=uaQeXi4E7gA

[lordsmurf]

The Topaz app uses A.I it isn't just another upsizer.

All of the "AI" from Topaz is marketing for gullible newbies. Most of their stuff is now "AI".

[El Heggunte]

The Topaz app uses A.I it isn't just another upsizer.To understand A.I you first have to understand Tensors.

https://www.youtube.com/watch?v=bpG3gqDM80w

https://www.youtube.com/watch?v=uaQeXi4E7gA

Well, I'm glad that I learned something about tensors by reading the books of Theodore von Kármán in the 80s,
not by watching second-class videos on YouTube...

[brandon87]

Thank you for the info about Tensors. I'll have to read up about them when I have some time. It would be nice to have a basic understanding of the magic I'm seeing.

I was first interested in Topaz to see if it could fix the low bitrate problem Youtube has. Around 2016, Youtube cut the bitrate to 1/3rd to 1/4th what it had in 2014 and 2015. Presumably to save money. No longer were videos of adequate bitrate. I wanted to see if Topaz could add detail to the overcompressed videos. The answer is no, it cannot. Bummer!

My second interest is to make better versions of DVD content that will not see a Bluray or web stream release. The answer is yes it can add 150 to 200 percent more detail than existed previously. This I find seriously impressive. Now I have several hundred projects to work on. ST Voyager is one of the series (yes I know others are doing this, we'll see whose is best).

My third interest is upscaling 1280x720p concert videos to 1920x1080p. So far the answer is a qualified yes it can be done. I have noticed that the 1280x720 video must be a relatively high bitrate for the 1920x1080p to look good. The ultimate test will be to seed them at 1080p, but not tell anyone Topaz was used, and see if anyone posts wondering where the release came from. If I get really brave, I'll seed on Dime (they have a bazillion rules including no reencoding no matter what (quite arbitrary don't you think?), which I would be violating...).

It should be noted that the final video needs to have a bitrate appropriate to the quality level. Or else the overcompression will make moot your efforts (beginners may not realize this).

I find it awe inspiring to watch Topaz processing a video, showing the input and output videos in one window, with the left pane looking so so.. and the right pane looking great. And this is before I do any final tweaks with the output .tiffs.

I'm not going to post any samples. You can get a trial edition and do some testing yourself. I'll probably return to just lurking now.

[johns0]

That's all nonsense about ai with tensor,doesn't prove anything about video ai,show us how matrix with axle rotation applying to video encoding.

[lordsmurf]

That's all nonsense about ai with tensor,doesn't prove anything about video ai,show us how matrix with axle rotation applying to video encoding.

To expand on this:

"AI" is a modern term for magic.

People only believe in magic when they don't understand what's actually happening.

The Topaz software isn't AI, and isn't magic. All that's happening is (1) upsize, (2) sharpening, (3) potentially some interpolation.

VideoHelp, digitalFAQ, Doom9, and some others, are not full of gullible newbies. We understand video. We're not bamboozled by nonsense, and understand the underlying processes that are being done to the video.

In the case of Topaz, it's done badly. There's artifacts -- unacceptable artifacts. Artifacts that you won't get with better Avisynth work. jagabo did some examples of this in another thread some months back.

Yes, video is math. But watching some guy explain math on Youtube still doesn't make this software AI, nor does it have any relevance to the quality (or lack thereof) from the Topaz software.

[Troc]

That's all nonsense about ai with tensor,doesn't prove anything about video ai,show us how matrix with axle rotation applying to video encoding.

To expand on this:

"AI" is a modern term for magic.

People only believe in magic when they don't understand what's actually happening.

The Topaz software isn't AI, and isn't magic. All that's happening is (1) upsize, (2) sharpening, (3) potentially some interpolation.

VideoHelp, digitalFAQ, Doom9, and some others, are not full of gullible newbies. We understand video. We're not bamboozled by nonsense, and understand the underlying processes that are being done to the video.

In the case of Topaz, it's done badly. There's artifacts -- unacceptable artifacts. Artifacts that you won't get with better Avisynth work. jagabo did some examples of this in another thread some months back.

Yes, video is math. But watching some guy explain math on Youtube still doesn't make this software AI, nor does it have any relevance to the quality (or lack thereof) from the Topaz software.

Hi, Lordsmurf. You don't know me, but I've read and benefitted from your guides. You are obviously very knowledgeable in regards to video enhancing and I don't mean to disparage your in any way.
However, I do feel that you're being needlessly harsh towards VEAI. I have used it and Gigapixel a lot for video and picture work and there are usecases where it works wonders. Any video enhancer will struggle with unfitting input types. There's also many different models, even more nowadays. The earlier editions of the software were much more limited than they are now.

My main gripe is that it simply isn't just sharpening. Check this out:
https://ibb.co/dkhQ18b
Simply sharpening would increase the jagged lines in my admittedly pretty garbage input from Youtube. Instead, the important lines between elements are transformed to be more of what they should look like, smoother and still sharp. I have to assume you were exaggerating when you said that it's simply sharpening. I have seen badly sharpened video and this is different. VEAI also does another very nice thing for videos, compare the gradient around the pagoda:
https://ibb.co/jMkcDP7
The program does some pretty nice deblocking which again, sharpening filters would make more, not less apparent.

The main problem and benefit of VEAI is the simplicity. (Second one being that 200 bucks for one year of updates is rather steep) You put in video, click and wait, afterwards the video will look better. It's very apporachable for newcomers and the media, who in turn influence more people and hardware makers in the long run.
If I had a lot of time and brain energy, I would of course would try out how different filters on Virtualdub handle the particular issues in my chosen video. That's how I'm actually handling Cutie Honey (1973), since it has a lot of unrelated issues that I need to fix before choosing a good upscaling method.
In the mountain, there is some annoying texturing that I wish wasn't there, sure. Everything else is better, so I consider this to be worth it. (Though if I knew how to get native 4K 60fps version, I wouldn't have spent all those GPU hours on it)

I do agree that AI is often used in bold letters for marketing. It's very annoying. In an AI group I follow, there's regularly advertisements for AI software that amounts to nothing and never pops up again after the initial bunch of posts. However, VEAI is different. It's an actively developed and maintained software that produces tangible results. I'm in the beta group and it's evolving nicely. Maybe it will be worth your while as well in the future?

[lordsmurf]

However, I do feel that you're being needlessly harsh towards VEAI.

I am completely against over-enthusiasm for a product that results in propagated myths. Whether it's Tesla (POS cars, 2x+ problems compared to other car maker), or 5G (1mi-per towers required), or this Topaz software. It gets a cult status, where problems are ignored, and weak strengths are praised with hyperbole.

Topaz made an upsizer that smart sharpens, with some interpolation, maybe some other finalizing like deblock. Great. But it's not very good compared to Avisynth. It has lots of artifacts on every output that I've seen to date. For $200, you're paying for a dummy-friendly GUI, the end.

Do you remember Topaz Moments? I do. Same BS, different day. "Now with AI!" is really the main difference now, 10+ years later.

Simply sharpening would increase the jagged lines

Bad source, bad deinterlace, create jaggies. Simple sharpening increases that. Smart sharpening does not.

I have to assume you were exaggerating when you said that it's simply sharpening.

It's simply smart sharpening.

who in turn influence more people and hardware makers in the long run.

Eh. That's a leap as stated. But I'd be interested in seeing a much longer hypothesis on this. You must remember that Topaz is a bit player, mostly known for lower-end Photoshop plugins. They're no Adobe.

[Troc]

I am completely against over-enthusiasm for a product that results in propagated myths. Whether it's Tesla (POS cars, 2x+ problems compared to other car maker), or 5G (1mi-per towers required), or this Topaz software. It gets a cult status, where problems are ignored, and weak strengths are praised with hyperbole.

I agree with the sentiment. When everyone is hyping something and ignoring all the downsides, I often am the bearer of bad news in my circle of friends. People should stay realistic, which also includes not being too negative. I wouldn't say that Topaz users have a cult mentality. The BETA group and the forum is full of people pointing out every shortcoming they find in firm terms. There is a lot of love for it too, and that is based on people genuinely enjoying the results.

Topaz made an upsizer that smart sharpens, with some interpolation, maybe some other finalizing like deblock. Great. But it's not very good compared to Avisynth. It has lots of artifacts on every output that I've seen to date. For $200, you're paying for a dummy-friendly GUI, the end.

Could you, in theory, make a script that does everything you mentioned and boosts the resolution? If it has faster speed, more general efficiency and better results, I'm very interested to try it out.

Avisynth has a certain difficulty to approach, unlike Topaz or even Virtualdub. Topaz makes many kinds of videos look significantly improved. Ease of use is important. Not that muchto me, since I'm ready to jump through crazy loops to get the optimal results, but for the normies. There has to be starting points for people. For me, VEAI was that first step. After finding better alternatives, I use it less frequently, but it's still a neat tool.

It's simply smart sharpening.

That is enough for many people. Is it a problem that they don't want to take a deeper dive?

who in turn influence more people and hardware makers in the long run.

Eh. That's a leap as stated. But I'd be interested in seeing a much longer hypothesis on this. You must remember that Topaz is a bit player, mostly known for lower-end Photoshop plugins. They're no Adobe.

I would also want to see how it pans out in real life. This might be one of the rare times I'm being too optimistic. I think that Topaz's success in selling AI upscaling will surely make the bigger players take note.

[jagabo]

Could you, in theory, make a script that does everything you mentioned and boosts the resolution?

A very basic script would be something like:

WhateverSource("filename.ext")
nnedi3_rpow2(4, cshift="Spline36Resize", fwidth=1440, fheight=1080)
aWarpSharp2(depth=10)
Sharpen(0.5)

But you may have to do some pre-processing to get a clean sharp image to start with. I often do the upscaling in steps with sharpening at each step:

# clean sharp 720x480, 4:3 source
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=960 fheight=720)
aWarpSharp2(depth=3)
Sharpen(0.3)
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=1440 fheight=1080)
aWarpSharp2(depth=5)
Sharpen(0.5)

[Selur]

just as a side note:
this some testing yesterday with Vapoursynth and to me using Waifu2x-ncnn + aWarpsharp at least for Cartoon/Anime content basically produced comparable output to Topaz on that content. Anim4KCPP might produce similar output, but I didn't test it.
jagabo should also work with Vapoursynth, but could use nnedi3cl to use gpu support and speed up the whole process.

@jagabo: do you use multi-step upscaling only on cartoon or also on normal real life content?

Cu Selur

[lordsmurf]

Avisynth has a certain difficulty to approach, unlike Topaz or even Virtualdub. Topaz makes many kinds of videos look significantly improved. Ease of use is important. Not that muchto me, since I'm ready to jump through crazy loops to get the optimal results, but for the normies.

I think this is the flaw in your "argument" (debate class argument, we're not really arguing here). The demographic willing to pay $200 to restore a video is the same audience willing to learn Avisynth. In fact, due to the free nature, Avisynth will capture some audience that the $200 app cannot.

There has to be starting points for people.

Selur's Hybrid is it. It's so advanced that even I'm adopting it for many tasks now. GUI is simple (admittedly slight learning curve, understanding of Avisynth helps), only area still lacking is the preview (very buggy). And I think $200 (donation) would be far more valuable to its continued development than Topaz. It would be hard to find a better, nicer, more responsive developer, than Selur.

I would also want to see how it pans out in real life. This might be one of the rare times I'm being too optimistic. I think that Topaz's success in selling AI upscaling will surely make the bigger players take note.

Again, from an academic stance, I'd be interested in hearing the argument "for" here. I just don't see it. In fact, I'll go further with Adobe mention. Because of the difficulty in a one-size-fits-all restoration approach (which is what this is), and the lackluster results from all attempts to date (going back 20 years, Topaz isn't doing anything new or amazing here), even a player like Adobe is afraid to approach this sort of task.

Perhaps if Adobe licensed some sort of Topaz SDK, or even acquired Topaz (2+ year lead time to make their products viable), then we'd have something to talk about. Barring major players, hardware makers won't care whatsoever. The emphasis these days is playback, raw encoding -- not anything else.

It's been hobby community, namely Avisynth, that retrofits filters to work with extant hardware accelerations. Sadly, sometimes that has backfired, as the hardware acceleration is dropped, making the filter ineffective. So small hobby and large players (Adobe) alike are gun-shy about over reliance on hardware accelerations. I think CUDA was a huge mess, and has affected everything since.

[Troc]

Could you, in theory, make a script that does everything you mentioned and boosts the resolution?

A very basic script would be something like:

WhateverSource("filename.ext")
nnedi3_rpow2(4, cshift="Spline36Resize", fwidth=1440, fheight=1080)
aWarpSharp2(depth=10)
Sharpen(0.5)

Thank you for the code, however, "There is no function named 'nnedi3_rpow2'." I thought I was supposed to copy these .DLLs into the Avisynth plugin directory, so I did it but it still doesn't work. https://github.com/dubhater/vapoursynth-nnedi3/releases
How do I proceed?

Edit: Also applies to aWarpSharp2, which I actually only managed to find aWarpSharpMT. All the filters seem to be tiny anyway, so it would be effective if I could simply download a version with every filter already in it.

[jagabo]

@jagabo: do you use multi-step upscaling only on cartoon or also on normal real life content?

Usually only on cartoons. If you sharpen real life video too much it starts looking very artificial. It becomes very obvious you are looking at upscaled video. Even though it has sharp edges free of aliasing artifacts there are no details between those edges. Actors all look like mannequins, etc.

[jagabo]

Thank you for the code, however, "There is no function named 'nnedi3_rpow2'." I thought I was supposed to copy these .DLLs into the Avisynth plugin directory, so I did it but it still doesn't work. https://github.com/dubhater/vapoursynth-nnedi3/releases

Vapoursynth != AviSynth

[lordsmurf]

Thank you for the code, however, "There is no function named 'nnedi3_rpow2'." I thought I was supposed to copy these .DLLs into the Avisynth plugin directory, so I did it but it still doesn't work. https://github.com/dubhater/vapoursynth-nnedi3/releases

Vapoursynth != AviSynth

http://avisynth.nl/index.php/Nnedi3/nnedi3_rpow2
http://avisynth.nl/index.php/AWarpSharp2

Use x64 AviSynth+
x64 AvsPmod to preview https://www.videohelp.com/software/AvsP

[Selur]

only area still lacking is the preview (very buggy).

Always working on fixes.

[lordsmurf]

only area still lacking is the preview (very buggy).

Always working on fixes.

I know.

Not to OT this thread, but the current preview GUI has preview rendering, in the style of (at least some versions back) Adobe Premiere. Everything freezes, becomes unresponsive, sometimes crashes. I wish the preview was more along the lines of AvsPmod, where it will simply advanced 1 frames at a time (and subsequent frames may render faster or slower, depending on CPU/GPU needed), rather than attempting x-length realtime renders (10s, 30s, etc). AvsPmod code changes are almost always instant, no rendering needed. I think that fundamental shift in preview type would go a long way to removing the buggy nature. Perhaps give an optional 2nd preview methods that relies on lengthy renders. But the best suggestion is to always just encode out a tiny snippet for the realtime (intraframe) preview. Rely on preview only for interframe (in-frame), not intraframe.

If Hybrid had a better GUI, something like the Topaz crapware would have no clear advantage.

The programming needed to do this is just beyond me, otherwise I'd try to help you.

This is why I still must rely on Avisynth directly, with AvsPmod, more than not. And why I don't use Vapoursynth more.

[b3nd0vah]

Okay, after following the documentation, this thread, and some help from Troc (above); I've managed to get avisynth, nnedi3, the sharpening syntax, and vdub cooperating. However, I don't actually know how to proceed from here and output a video that I can open with something like MPV (if that's even an option, I don't know).

Also, despite the hate all of Topaz's products seem to get here, for whatever reason, VEAI has given me good results when deblocking, denoising, and upscaling videos, not that I'd ever pay for the product. How would I go about replicating this function with AVISynth and what sort of quality comparison should I expect to see?

[davexnet]

Okay, after following the documentation, this thread, and some help from Troc (above); I've managed to get avisynth, nnedi3, the sharpening syntax, and vdub cooperating. However, I don't actually know how to proceed from here and output a video that I can open with something like MPV (if that's even an option, I don't know).

Also, despite the hate all of Topaz's products seem to get here, for whatever reason, VEAI has given me good results when deblocking, denoising, and upscaling videos, not that I'd ever pay for the product. How would I go about replicating this function with AVISynth and what sort of quality comparison should I expect to see?

Create your script, open in vdub2, set video/compression, file/save video

[lordsmurf]

despite the hate all of Topaz's products seem to get here, for whatever reason, VEAI has given me good results when deblocking, denoising, and upscaling videos,

When all you've ever eaten is dog food, you are unaware of what a steak or hamburger tastes like. Can you imagine somebody claiming that Alpo tastes great, and he/she doesn't understand why you hate it so much?

You don't know what you don't know.

We're trying to show you. Avisynth is steak. Topaz software is Alpo.

And can you image paying $200 for a can of dog food (because it has a pretty can, ie the Topaz GUI), when the steak is free?

[b3nd0vah]

despite the hate all of Topaz's products seem to get here, for whatever reason, VEAI has given me good results when deblocking, denoising, and upscaling videos,

When all you've ever eaten is dog food, you are unaware of what a steak or hamburger tastes like. Can you imagine somebody claiming that Alpo tastes great, and he/she doesn't understand why you hate it so much?

You don't know what you don't know.

We're trying to show you. Avisynth is steak. Topaz software is Alpo.

And can you image paying $200 for a can of dog food (because it has a pretty can, ie the Topaz GUI), when the steak is free?

And yet all I've seen you do is show prejudice rather than a comparison or method to produce a better result than what Topaz offers. Because from what's available on the internet, nothing else comes close. So if all you have to contribute is prejudice against something than I'd rather you didn't contribute at all.

I don't care about Topaz as a company or anything they produce, what I care about is the result; their stuff gives good results. I've run a video through nnedi_rpow2 with the script you posted above, it was noticeably worse than the result I got from VEAI. So please, if you're able to outdo the result I got, preferably in less time, then post it. All I care about is the result.