so where's all the Topaz Video Enhance AI discussion?

[forart.it]

Selur (interesting) tests for HINet: Half Instance Normalization Network for Image Restoration, then:

Has anyone tested https://github.com/HolyWu/vs-hinet ?

Here are a few screen shots: (not sure what to make of them and for what content this is really useful)

Mode: Deblur GoPro

Mode: Deblur REDS

Mode: denoise

Mode: derain

[forart.it]

Just discovered this interesting Youtube channel:

Fan made channel dedicated to uploading videos in 4K Ultra HD, using AI interpolation, with doubled fps and lossless flac remastered audio.

Currently uploading short videos (-10min) due to hardware limitation and problems with audiovisual copyrights. In a future (if I get some support), with a better graphics and more SSD storage space, longer videos will be remastered and published here (or through a download link).

This channel is not intended to make money, I don't have any rights over the audiovisual content, so I can't monetize the channel. Feel free to block ads here. If you want to help me to get a larger SSD, contact me using the email provided here (check the button in 'Details' section).

All rights from audiovisual contents belong to Pink Floyd.

https://www.youtube.com/c/PinkFloyd4K/videos

It don't specify which software is used, anyway how do you think is possible to obtain equal - or better - results with open source alternatives ?

https://www.youtube.com/watch?v=OFwJGVH2l-M

[forart.it]

Investigating Tradeoffs in Real-World Video Super-Resolution (aka RealBasicVSR) has just been released:

https://user-images.githubusercontent.com/7676947/143370499-9fe4069b-46cc-4f12-b6ff-5595e8e5e0b8.mp4

https://user-images.githubusercontent.com/7676947/143370350-91f751f3-0f33-4ee4-9b1a-b9279bf41c18.mp4

https://user-images.githubusercontent.com/7676947/143370556-9e7019d4-e718-46af-859f-54d5576cd370.mp4

https://user-images.githubusercontent.com/7676947/143370859-e0293b97-f962-476f-acf8-14fad27cea77.mp4

(videos have been compressed. Therefore, the results are inferior to that of the actual outputs)

[rrats]

https://www.youtube.com/c/PinkFloyd4K/videos

This is the worst most artificial looking shit I've ever seen

[khobar]

Hello,

I found this thread when searching for topaz ai video enhancer doesn't seem to do anything.

I shot some VHS and 8mm video in 1992 that I'd like to restore/improve. I brought them into the demo version of Topaz VEAI and, well, I honestly don't see any difference between the source and the 4K upscale. From many of the posts here, this seems to be expected. I have used Topaz AI Denoiser with excellent results, so I had high hopes for VEAI. I'm glad I held off on purchasing.

I did run one video through VirtualDub2 to deinterlace, and I can see it made a difference, but it is far from wow. I used this also with VEAI and remain unimpressed.

I am now wondering if these old .AVI files are worth revisiting or should I recapture the videos using my Diamond VC500 usb capture gizmo?
I have an old Sony TRV740 Digital8 for playback, and most of the footage was shot on 8mm video tape (not Hi8 or D8). The TRV740 has Firewire out, and I have a cable but, so far, no capture card. Would the Firewire provide better results than either composite or S-Video for my videos/would it be worthwhile to get a Firewire card?

Thanks.

[rrats]

I did run one video through VirtualDub2 to deinterlace, and I can see it made a difference, but it is far from wow.

Deinterlacing =/= upscaling

Also all the virtualdub deinterlacers suck

[khobar]

Hi and thanks! I'm not necessarily looking to upscale unless it improves my old footage, but I think first I need the best possible old footage I can get, and that might mean recapturing.

I looked through some old hard drives and found a some of my wedding videos from 1992. At the time, I would have connected my Sony 8mm camcorder (or consumer-grade VHS VCR for the VHS stuff) to a capture card (probably Hauppauge). I had hoped VEAI would do for my footage what it purports to do for youtube footage, but I didn't see anything like that at all.

[topazAIworkswell]

Topaz AI works really well. You need to pay attention to what your source video is though. If its a typical DVD, 720X380 or so or 848X364 then your settings in TOPAZ are prob going to be LOW for the source quality, then PROTEUS FINE TUNE for the algorithm. Scan thru the video first, find large pan scenes, then preview 5 seconds of it where you can compare two different algorithms. You have to do this step to know which one will perform the best on your footage. Also don't go from 364p all the way to 1080p, frames that have wide shots and fitting say 35 people on a street won't look great. The starting detail just isn't there. But PROTEUS and up to 1280X720 looks incredibly good. This program is not set it and forget it. You also have to pay attention to the frame rate. I found if you rip it to MKV and change the sound to STEREO then topaz does all of it, and you have a finished product that plays well on TV's. But..... most cheap ripping programs donit really detect what the source video is correctly. Most DVD's are 23.98 FPS not 29.97, yet most ripping programs will try to fit it into 29.97 which means the playback stutters. So when you rip your VOB files, make sure you actually know what the frame rate is of the source. I use 23.976 for the created DVD video clip that has the audio changed to STEREO, and the bit rate turned up real high, so a DVD is created as a 7GB MKV clip to ensure no loss, then let topaz move it up to 1280X720 and the result is awesome. I've actually tried other upscale programs, none are even in the same world as Topaz. You cannot tell it was originally a DVD when played thru plex onto your TV. The most common problem people have is they want to put "High" for source video when "High" to this program means you have say source 720X480 video filmed on a $3,000 Canon GL2. Most DVD's would have a quality this program would call LOW to MEDIUM at the highest. So.... tell the truth with the source video quality, know the frame rate and KEEP the original frame rate, and make sure your source put into TOPAZ uses STEREO only. I have an i5 CPU on a machine with a RTX 3060 card and I'm currently upscaling a DVD movie and it's cranking at 0.16seconds per frame. If its interlaced, don't use Topaz or upscale it. nothing will look good if its altered. I've tried dozens of tests.

[Cornucopia]

All DVDs are coded as either 29.97 or 25. That's it. They may be flagged with pulldown, wherein a smarter ripper+converter can use IVTC to get back to 23.976, but that would only be that way on the converted file - NOT the DVD.

And I think you are pulling "most are 23.976" out of thin air.

And, considering many DVDs are taken from hirez scans of digital or film masters, the idea that GL2 output would be considered better than that is ridiculous. I have a gl2, and it's great...for a DV cam. Not in the same league as a Arri or Viper, Dalsa, or Red Dragon though.

Plus, it has already been shown here on this thread, and elsewhere, that there are other apps and processes that can do better than Topaz, even when both are "optimized" for the content.


Better luck next time, mr. "I'm not shilling even though my handle is bragging the product".


Scott

[lordsmurf]

If its a typical DVD, 720X380 or so or 848X364

Huh? DVD is 720/704/352 x 240/288/480/576, period. Not whatever that is.
Almost all are 720x480/576

This program is not set it and forget it.

Definitely not.
I'd go one further, it's really "forget it and forget it", then move on to Avisynth.

Most DVD's are 23.98 FPS not 29.97

Probably not correct. DVD box sets vastly outnumber movies. Then again, how many are IVTC'd. Hmm... interesting. But do note Corn/Scott is correct here. Only 29.97 MPEG coded for NTSC, with pulldown flags.

move it up to 1280X720 and the result is awesome.

I've long advocated to stopping at 720p for upscaling SD work. When you get into 1080, or higher, things go sideways quickly. But sometimes you're forced, like documentaries.

I've actually tried other upscale programs, none are even in the same world as Topaz.

Other "programs" are mostly junky software. But that's not Avisynth. I'm guessing you've never used it.

filmed on

Technically, wrong word usage. Film is filmed, video isn't filmed. Video is shot. I know, seems silly, but sometimes words really matter in video. Too easy to confuse yourself and others, by choosing wrong words. "capture" (DV) and "rip" (VHS) are problems.

I'm currently upscaling a DVD movie and it's cranking at 0.16seconds per frame.

But what's the point? It won't change the ending of the movie, or anything like that. It's a small % boost, and sometimes arguably loss (especially with Topaz, weird artifacts). I see need to upscale for broadcast, mixed sources, etc. But why go to all this insane trouble to watch on TV at home? Such a waste of time and energy.

[Jarrod1937]

Lordsmurf... glad to see you still seem to be carrying your incorrect understanding from the beginning of this thread. I registered just to let you know, regardless of your general video expertise, you seem to be very mistaken in understanding AI. I say this as an electrical engineer who studied signal processing who is also a senior software developer, and a person who uses machine learning (classical techniques as well as neural networks), specifically towards image processing.

I have also used many free tools, including the ones you've discussed, in addition to Topaz Video. I can confidently say that Topaz does an excellent job at compression removal without hurting overall underlying texture. It also does a good job of smart sharpening, but goes beyond more basic algorithms. AI in general is not some word for "magic," although it can be misused in marketing for sure. However, a true deep neural network is able to encode so many parameters that it has the capability to outperform any manmade algorithm. Of course, proper training data, model structure, and hyperparameter tuning is needed to make that capability realizable. This isn't just about Topaz, I've seen neural networks easily outpace many manmade algorithms. Though, a lot of the time, pre or post processing the neural network input/output with traditional algorithms gives the best results as it gives the neural network an easier time. The biggest thing I've seen as a neural network advantage is that a larger enough model is able to effectively encode data about the problem as if it has a memory. Meaning, it in some sense 'knows' what should be. This can allow them to do things that are simply not possible with traditional handcrafted algorithm.

I sincerely suggest you open your mind instead of digging yourself into a deeper hole. You kind of come off as the old man yelling at the youngsters to get off their lawn. While I can respect your experience in other areas, you show yourself to be severely lacking in this area.

In regards to Topaz specifically, it has its issues, but I have found it can do a really good job at times. Those that complain of halo'ing/overshoot, or texture being wiped out/excessive low pass, may need to switch to a different model or alter their settings. I found Proteus fine tune to be the best all around model and I have upscaled multiple shows with it to fairly good results that certainly are better than the base video, and better than any traditional algorithm alone I've used up to that point. One particularly impressive moment was when I passed a japanese show through it, there was a poster on a wall and the text was unintelligible, almost approaching pure blobs. Taking this single frame and trying all kinds of processing on it wouldn't bring back the text, yet, Topaz actually restored the text to something semi-decipherable. Best I can tell is it is most likely the neural network doing a very good job with deblurring combined with multi-frame super resolution processing. Topaz has actually brought out real texture detail that my eye didn't know existed in the source material (or it is that good at inventing detail), and which never showed up using a multitude of other methods.
Don't get me wrong, I'm not the biggest Topaz fan. Their models are really good, but their software is very rough around the edges and could use a lot of work. I also found their automatic model suggestions and model parameters (for those that do have them) to be useless. Their recommendations almost always lead to overshooting and texture loss at the same time, so avoid them and do a more trial and error approach instead. It also needs to not be used on its own a lot of the time, as it has flaws that need pre or post processing.

"But what's the point? It won't change the ending of the movie, or anything like that. It's a small % boost, and sometimes arguably loss (especially with Topaz, weird artifacts). "

I find this statement particularly confusing. What is the point of bringing out details in a show or movie that you love that may never exist in a higher resolution format? That's like arguing we might as well all watch videos in 240p merely because the ending will not change. I can't tell if this is you just becoming illogical at this point, trying to argue to death against something which you dug in so deep on, or what. It's nonsensical for a community that revolves around processing videos. Though, at least in my case, it also isn't that much work or time. I have higher end GPUs with lots of VRAM so the processing is quite fast, for example, Topaz usually processes frames at a speed of 0.06 sec per frame.

Note, I likely won't respond if replied to. I just found this thread to contain some really bad information and kind of hurts the reputation of this forum as a whole. This is likely to drive away people that are interested in pushing video processing further, as it comes off as anti-intellectual at times.

[johns0]

Your response is wrong,Lordsmurf knows his stuff and topaz is crap no matter what you say.

[lordsmurf]

you seem to be very mistaken in understanding AI. I say this as an electrical engineer who studied signal processing who is also a senior software developer, and a person who uses machine learning (classical techniques as well as neural networks), specifically towards image processing.

If you think Topaz is AI, then you didn't pay attention in your studies, and I can only assume you also develop non-AI "AI" software.

I sincerely suggest you open your mind

That phrase is always uttered by snake oil salesmen. Nevermind facts, just "open your mind" (aka, believe the BS being presented).

You kind of come off as the old man yelling at the youngsters to get off their lawn.

You come across as gullible to marketing.

In regards to Topaz specifically, it has its issues, but I have found it can do a really good job at times. Those that complain of halo'ing/overshoot, or texture being wiped out/excessive low pass, may need to switch to a different model or alter their settings.

The problem with Topaz is altering settings just makes new problems.

better than the base video, and better than any traditional algorithm alone I've used

Words. Everything I've seen to date is worse than the source. Just sharper, with new odd artifacts.

Taking this single frame and trying all kinds of processing on it wouldn't bring back the text, yet, Topaz actually restored the text to something semi-decipherable.

Selur and others did this posts ago in this thread. Topaz was still illegible by comparison.

Don't get me wrong, I'm not the biggest Topaz fan. Their models are really good, but their software is very rough around the edges and could use a lot of work. I also found their automatic model suggestions and model parameters (for those that do have them) to be useless. Their recommendations almost always lead to overshooting and texture loss at the same time, so avoid them and do a more trial and error approach instead. It also needs to not be used on its own a lot of the time, as it has flaws that need pre or post processing.

That's exactly it. It's not a usable product as-is. You pay $200 for buggy software that works worse than freeware Avisynth. It's not AI, and it's very alpha grade at best. Maybe someday, but then you don't seem to know the history of Topaz. If and when Adobe claims to have something AI-like (they're too honest to BS you with claiming it's actual AI), then we'll probably have something to actually talk about. For now, no.

"But what's the point? It won't change the ending of the movie, or anything like that. It's a small % boost, and sometimes arguably loss (especially with Topaz, weird artifacts). "
I find this statement particularly confusing. What is the point of bringing out details in a show or movie that you love that may never exist in a higher resolution format?
That's like arguing we might as well all watch videos in 240p merely because the ending will not change.

The confusion is because you seem to think Topaz makes great quality differences. It's just not so. If you just want to pump the sharpen dial to 11, that could be done 20 years ago. If you want actual improvements, this isn't going to give it. You'll lose at least as much as you gain, due to artifacts and bad algorithm choices. It makes so-called "low resolution" footage extra sharp with artifacts, and no additional details are created or magically "brought back".

I have higher end GPUs with lots of VRAM so the processing is quite fast, for example, Topaz usually processes frames at a speed of 0.06 sec per frame.

I just don't believe that. I call BS. Either that, or details are missing.

Note, I likely won't respond if replied to. I just found this thread to contain some really bad information and kind of hurts the reputation of this forum as a whole. This is likely to drive away people that are interested in pushing video processing further, as it comes off as anti-intellectual at times.

Nonsense. What I hate to see is inferior software that overpromises, underdelivers, and for the low-low prices of gobs of money. "Cheap" (not cheap, just less costly than Adobe/etc) photo and video software suckers in lots of gullible newbies. They then form bad opinions on how video/photo can look, or should look. Sometimes products are obvious money grabs, and the poor execution is obvious to those in the field (either work/pro, or as serous hobby).

[Jarrod1937]

Lordsmurf

"If you think Topaz is AI, then you didn't pay attention in your studies, and I can only assume you also develop non-AI "AI" software."

I'm a full stack web developer, as well as application developer (Linux, Windows) in multiple languages (C/C++, Python, C#, Assembly, PHP, Java....etc), who has reverse engineering experience. I also have developed and used neural network and various machine learning models and have multiple in production right now including a SAAS product for my company. This is all aside from my electrical engineering experience, in particular, signal processing, where I have knowledge in analog and digital filter design (FIR/IIR).

This comment shows that you don't know what you're talking about. I have used Topaz and I can 100% guarantee you it is using neural networks, both in its usage of GPU tensor cores as well as by the fact that you can literally see the neural network models that it uses...

"That phrase is always uttered by snake oil salesmen"

No... it is literally to get you to not be as close minded. You basically have shown as much in this reply of yours since you very, very obviously haven't taken the time to verify any of the assertions you make. You honestly have such little knowledge in this specific area (I've been very clear about it only being about AI). The rest of your reply isn't worth replying to as it all stems from some illogical bias against AI and Topaz in particular.

I'm also kind of curious if you even understand various super resolution methods. Your comments seem to suggest that you're missing that information can be added to a video frame both temporally as well as via knowledge embedded in the processing system itself (it "knows" what is likely correct).

[Jarrod1937]

Johns0

"Your response is wrong,Lordsmurf knows his stuff and topaz is crap no matter what you say."

While I can respect his knowledge in other areas, he is 100% incorrect here. Don't fall for argument by authority anyways, this is actually a perfect example of why that is the case. He may be authoritative in one area, yet he is dead wrong here.

[Jarrod1937]

Lordsmurf, with that said, I am here if you have any questions about how neural networks function. I can go full in-depth on it. This would likely help broaden your view and allow you to see why neural networks have potential to beat out handmade algorithms. If you're interested, let me know. Otherwise, I'd suggest doing quite a bit of reading on your part so you can catch up on the last decade of image processing research.

[lordsmurf]

Some of this has already been covered here:

What is AI, NN, etc:
https://forum.videohelp.com/threads/399360-so-where-s-all-the-Topaz-Video-Enhance-AI-d...e3#post2602672

Topaz not needed to extract all available details:
https://forum.videohelp.com/threads/399360-so-where-s-all-the-Topaz-Video-Enhance-AI-d...e3#post2603010

If it was good software, I'd be using it, singing it's praises, etc. It's not.

FYI, my area if expertise is restoration of consumer analog formats. This software makes that source worse.

It may make HD into 4K, but even that still looks crappy. Why? Artifacts. The predictive algorithms suck. A magic 8-ball is better.

A few folks are (for whatever reason) intent in making Topaz work for them. At best, going scene by scene, and in combo with Avisynth, a few folks claim (without any samples) to have some success. But again, seems like a lot of work for the small gains.

You need to realize that I wish I was wrong. (I'm not.) I'm all for better hardware, better software. I'd like to have a simple GUI, with powerful ability to scale, sharpen, etc. But no such thing exists. Not now, maybe never. The best results, sometimes the only usable results, are far more manual.

[Jarrod1937]

Your first link just seriously shows you have no idea what neural networks, or AI in general are. It shows you have no idea of process of developing AI, or anything regarding it. Well, on that note, not worth my time to continue then, it's like talking to a rock. You're wrong, but you seem happy to be in your delusional world of being wrong. Respect to your analog experience, but you're fallen quite behind in the computerized portion of things. I offered to assist you in this area, but you appear to have rejected that offer.

[Jarrod1937]

To explain, as I don't wish to come off as insulting. Your linked "explanation" is incorrect in multiple ways, and it is also obvious it is from a high level perspective, where you don't know enough about the topic to put together a fully coherent explanation. No harm in not knowing, but there is harm in fooling yourself into thinking you do.
With that said, good day to you.

[lordsmurf]

To explain, as I don't wish to come off as insulting.

If you disagree, then you need to define AI. Right now, you're just parroting Topaz, and claiming you understand what AI is without demonstrating one iota of knowledge. While I'm not an expert in (actual) AI, I know how to detect marketing BS.

I offered to assist you

Where, how? I see no offers of any kind. Just "you're wrong" and "it isn't worth my time to explain further".

[Cornucopia]

The proof is in the pudding, as they say, and I for one have not been impressed by any of the non- tailored examples.


Scott

[lollo]

There is nothing that Topaz AI can do that cannot be beaten by AviSynt / Vapoursynth (quote lollo and Selur)

[Jarrod1937]

"Where, how? I see no offers of any kind. Just "you're wrong" and "it isn't worth my time to explain further"."

I quote:
"Lordsmurf, with that said, I am here if you have any questions about how neural networks function. I can go full in-depth on it. This would likely help broaden your view and allow you to see why neural networks have potential to beat out handmade algorithms. If you're interested, let me know."

... that is very clearly an offer to assist, is it not?

"If you disagree, then you need to define AI. Right now, you're just parroting Topaz, and claiming you understand what AI is without demonstrating one iota of knowledge. While I'm not an expert in (actual) AI, I know how to detect marketing BS."

This is kind of what I was afraid of, as I truly do have better things to do than prove myself to anyone. However, this can be used to teach I suppose, so I'll type up a quick high level overview as well as explain in your previous explanation where it is wrong.
----------------

First, let's get our terminology correct. Machine learning and AI is actually multiple methods, but most associate it with neural networks so I'll just cover neural networks here.

Here is a quote of your explanation:
"AI = artificial intelligence. There is no AI here. It's marketing.

Even nnedi3 claims to use "neural network architecture", but is really nothing more than predictive (aka opposite of reactive) algorithms.

"Neural networks" are a basis for AI, and the name "neural" comes from mimicking human (or any animal, really) brains. There is parallel processing, but there must also be an element of development. That would insist on the ability to save data for later analysis. And with Topaz, or nnedi3, or others, that isn't happening. There's no learning file being built.

So the "AI" here is dumber than an amoeba.

In reality, what happens here is that a pre-made "bank" of set data has to act as the basis of prediction, and it does not learn anything. And that's nothing new, having existed for decades. I think back to OCR scanning in the 90s, or speech-to-text in the 00s. Google Voice is probably a good example of actual (limited) AI, because the algorithm was allowed to grow, and has become far more accurate over time."

However, in reality, there are multiple things wrong with this. Neural networks are essentially universal function approximators. Function, in this case referring to the mathematical definition, input -> some process -> output. A lot of things in our day to day operation, as humans, are basically functions. The decision how how to steer given the particular state information presented to us, the way we hold and orient a spoon when trying to eat, the movement of our tongue when eating, the interpretation of visuals in general, the interpretation of audio, touch, or any of our senses, the general decision trees we go through...etc. A neural network is effectively being taught given a specific training set what a goal is (assuming proper goal alignment) and how to alter itself to achieve that goal. This is accomplished by how it encodes information and how it is trained.

The most common neural network implementation has single nodes that are basically individual neurons. These neurons have a weight (how important that neuron is in the path) and a bias. Neurons are given what is called an activation function, which is how it responds to different inputs in conjunction with its weight and bias. You can have linear and non-linear activation functions, and each can be useful depends on the network's function as well as the position of the neuron within the overall structure. A non-linear activation function is usually more useful since it allows a neural network to be more expressive in that you cannot express a global non-linear function given only linear components. The activation function is a mathematical operation that takes into account the weight, which gives overall importance to a neuron in the overall structure, plus a bias component, which is equal to providing a constant linear function. These neurons, added together, given different responses to different inputs, can create an approximation of an arbitrary non-linear function, resulting in a reactive and/or predictive algorithm, it depends on what you train it on.

Training is done by a training set (and usually a validation set to prevent overfitting and validation generalization). A training set can be anything, it just needs to be represented as an input/output pair for the overall training operation. You can do one-hot encoding for things that are categorical, or a continuous input/output pairing for things that are not (continuous in this case meaning a gradation, as all computer values are discrete at some level). The training output is known as the ground truth, and is effectively what sets the goal of the network. The individual weights and biases are initialized (sometimes to zero, usually just to random starting values), and the network has to find the association between the inputs and the described goal in the output. In this sense, it literally is learning according to the definition of 'learning.' The teaching process occurs through something called back propagation. Normally, you can use something called a gradient (calculus gradient) descent function to know where to move in the multi-dimensional solution space in order to move towards a global optimum, however, this is tricky with neural networks. So, neural networks approximate this by using a loss function at the output to know how far the output neurons are, each, from the given goal, and mathematically the error is back propagated from the output through the network to the input (but not including the input obviously). This effectively adjusts the weights and biases towards the overall goal. Though, in the earlier days of neural networks, they found deeper networks (more layers) had a tendency to have the gradient disappear before it reached the entire network. There are many solutions to the problem (although it's not entirely solved, just reduced), the main breakthrough happened when the Relu activation function was found and applied.

All of this is all well and good, however, you also need to have an appropriate structure for your neural network to accomplish the specific goal. This is known as your model. How many neurons for the input, output, how many layers, how many per layer, what type of layer...etc. You also have different classes of networks. Convolutional neural networks (normally used for image data), basic dense networks, Generative Adversarial Networks, recurrent networks, transformers, and so on. Each have their particular uses, and actually, you can mix things, like having a CNN layer as an input to a then basic dense network (perhaps for classification).

The important thing to remember with networks is that they can be an algorithm for a task, as in a series of steps to accomplish a goal, as well as actually introduce new information into the output that didn't exist in the input. This is because information from the training set is actually encoded into the network's structure. If you train a network to recover texture detail from a blurry photo, it will encode associations between variations of color, blurred pixel patterns, and so on, and the resulting texture detail it is meant to identify and replicate. The encoded information is quite evident in a few different network types. Auto-encoder networks basically are taught to take X as an input and output X, usually with a choke point in the middle. This requires the network to encode information about the training set in that choke point. This can be useful for super resolution applications, data compression, or anomaly detection since the output's loss function will increase when an input is presented that doesn't match the usual input examples it was trained on. GAN's are particularly good at introducing new data input the output, that hopefully matches the intention. A GAN is actually composed of two networks, one that differentiates, and one that generates. The generator is effectively trying to always trick the differentiator, and the differentiator is always trying to learn to find the generated results. This gives two useful networks at the end. The differentiator (also called the discriminator) can also be used for anomaly detection, since it learned to detect real from non-real. The generator has at that point learned to produce a believable output. GAN's effectively can invent new information at the output that didn't exist in the input, and the results are based on the training set. GAN's are used a lot these days, and my newest project is actually a GAN (it's a business idea, so I won't share the details at this time). The result of these is that a neural network absolutely can, despite your understanding, create new detail on the output that wasn't on the input. Further, this detail can be believable if the training set is an appropriate match for the input. Think of it as training a GAN to generate a second half of a cut off photo of cats. If trained well enough, a network can actually finish that photo of a cat that matches the first half. This actually exists, I don't recall the network, maybe GPT-3, where it was able to fill in (convincingly) an entire missing half of an arbitrary image.

Also worth mentioning, as I think your quote illustrated a common point of confusion. A neural network CAN learn in the wild, but it usually is not a good idea to do so. Almost all AI in production is taught elsewhere, then deployed to the real world, where it only does inference. The training alters the weights and biases, as discussed, and these are effectively an encoding of its learning. For deployment, you just need to know the model structure and these weights and bias values. It is usually a bad idea to have a neural network learn in the wild, as even during controlled training, your loss function minimization can be stochastic. Meaning, you can get spikes randomly during epochs of high inaccuracy before it settles back down. In the real world, this is dangerous.

"Google Voice is probably a good example of actual (limited) AI, because the algorithm was allowed to grow, and has become far more accurate over time."

Really, ALL neural networks are allowed to grow. It just depends if you get more training samples that provide more information for the network. In most deployed applications it can be a good idea for users to flag when an input doesn't produce an expected output, as this can be used as a training point. Neural networks can be sensitive to missing training examples, and if a particular category or type isn't in the training set, it may not generalize the problem well enough. Hence, iterative training, data augmentation, and other methods are commonly utilized.

In regards to Topaz. As I said, I'm not here to defend it specifically. I just saw a large amount of very incorrect information regarding AI. As for Topaz using AI... it 100% does, it is not marketing. I can see it using my tensor cores of my 3090's, something normally only used for tensor operations given it is of limited use for general compute, along with general compute CUDA cores. However, I also did some snooping in it's neural network models directory. They appear to be password protected zips, I'm sure I could find the password through some application reverse engineering, but I don't think it's worth my time (and legally questionable since I'm developing image processing AI's myself, though for completely different uses). I can also guarantee those are neural network models just based on the size of them. It is also obvious it is using neural network models since it has to fetch (or more so, can fetch) optimized models from their site during runtime These look to be specific models trained for a specific image size and parameter configuration. Most likely this is because they had a hard time getting the best results from a single model that had to generalize a lot more, as well as the fact that a less generalized model can usually be smaller in size (lower parameter count) and thus have a smaller memory footprint. Further, I know their processing does add texture that didn't exist in the input. It is most apparent when the settings are too high, or it is trying to extract and extrapolate texture from a video input that has too much compression artifacts. The results are unusual texture hallucinations. Regardless of your feelings, they are using AI, and I do believe they're benefiting from it in some areas. There are some examples where I've attempted multitude of manual methods and traditional algorithms (non-ML) and wasn't able to recreate it. Did I say it was the perfect solution? No... seriously, go read what I wrote. Ironically, I think one of the best places where it should be using AI but doesn't is in it's suggest model and suggested model parameters. Those appear to be manmade heuristics... and they are terrible, you effectively just have to ignore their suggestions because of this.

If you have specific questions, let me know and I can try to answer them, I didn't even cover basics like batch training and such. Though, I'm not particularly interested in defending Topaz in general, let alone against antifanatics that are irrational. If you aren't of this level, then you need to verify your assertions. You throw them out there quite easily, yet multiple were incorrect.
Hopefully I've at least explained how AI can do certain things a handmade human engineered algorithm cannot. Literally, a neural network is finding the global optimum (or a really good local) for a 175 billion dimension solution space (as is the case for GPT-3)... the complexity is well beyond human comprehension and ability. There's nothing magical about neural networks, except for the complexity they're able to represent. This leads to computational solutions to problems that human's didn't think possible prior. Because we don't like that much complexity, we rely a lot on heuristics instead, but these are limiting and equal to settling for a less than perfect local optimum in a given solution space.

[litmus]

If Topaz Video Enhance AI is such a joke then why did Janus Film use it for their new restoration of David Lynch's Inland Empire? https://i.redd.it/xu8ataqsiws81.jpg

When's the last time a commercial release was prepared using Avi Synth scripts?

[lordsmurf]

This is kind of what I was afraid of, as I truly do have better things to do

Ditto.

But I'll eventually read what you wrote. With a quick skim, it mostly looks like you went novel length, where I went for summary. And we don't disagree much on what AI/NN/etc is. But we do disagree on the effectiveness of Topaz, and implementation.

If Topaz Video Enhance AI is such a joke then why did Janus Film use it for their new restoration of David Lynch's Inland Empire? https://i.redd.it/xu8ataqsiws81.jpg
When's the last time a commercial release was prepared using Avi Synth scripts?

Commercial releases do dumb things all the time. Just think of all the awful colorizations we've had over the decades.

I also worked for studios. It's entirely within the realm of reason that a low-knowledge exec insisted some new magic method be used (Topaz, in this example), even at the protest of those actually doing the work. Why? It meant lower quality output. I remember instances where studios would try to appease (or coddle) certain "famous people", and cave to their (sometimes entirely unreasonable) demands. While I was generally left alone to do the good work that was needed, when a certain someone wanted a meeting, it was a thing of "what kind of BS do they want this time?"

Avisynth is used more often than you'd think. That actually where I honed my skills at it.

You also overlook paid sponsorships. There may have been an ulterior motive in the background (example, "use our software, and we'll pay for your restoration!"), something we're not privy to. Studios get solicitations all the time, and sometimes it's accepted. For some of these smaller companies/entities, a single mention (like a press release praising their software) is worth $$$$$ to them. They'll ride that as long as they can, use it for more marketing.

I'm not as gullible as John Q. Public, Joe Sixpack, or whomever. I suggest you get more savvy yourself.

[lollo]

When's the last time a commercial release was prepared using Avi Synth scripts?

You can ask Francesco Bucciantini (broadcast encoder for Sky Italy and Sky UK) or Derek Prestegard (video software engineer for Disney); FranceBB and Blue_MiSfit at Doom'9 forum respectively

[ConsumerDV]

@Jarrod1937, I appreciate the info you are sharing!

If Topaz Video Enhance AI is such a joke then why did Janus Film use it for their new restoration of David Lynch's Inland Empire? https://i.redd.it/xu8ataqsiws81.jpg

One would think that shooting a movie on a PD150 was passé in 2006, but apparently Lynch liked his camcorder so much he did not care to switch to the DVX100. Now he figured the artefacts from going from 29.97i to 24fps and then upscaling are insufferable, and wants to use AI to clean up the @#$%. If I understand correctly, the move will still be in 24fps, no one is going to re-edit the original DV and convert to 60p. The movie was shot "with a budget that’s impossible to calculate since Lynch basically just grabbed his camera, grabbed his leading woman and went for it," as Hollywood Reporter phrased it. Now he is going to spend - how much? - to AI his way out of the interlaced DV hole he got himself into sixteen years ago. This is the way of a true artist, I guess.

[dellsam34]

Lordsmurf, with that said, I am here if you have any questions about how neural networks function. I can go full in-depth on it. This would likely help broaden your view and allow you to see why neural networks have potential to beat out handmade algorithms. If you're interested, let me know. Otherwise, I'd suggest doing quite a bit of reading on your part so you can catch up on the last decade of image processing research.

Could you participate in processing some VHS captures using Topaz if you have some files or someone can provide you with samples? I'm all for progress, I saw some upscaled Britany Spears SD clips on youtube using this new technology and it was an astonishing achievement, I just think that it is still too enfant for noisy sources like consumer tape formats.

The same thing is going on in the video capturing arena, there is this new technology called RF capture where the VCR processing is all bypassed and there is a project going on here called VHS-decode, While I have high hopes for this technology it is still not perfect enough to beat an old school analog lossless capture method.

Please post more about this.

[rrats]

Topaz is Ai. A very crappy Ai but it is Ai. It could easily be beaten up by some avisynth pre-processing and VSGAN (vapoursynth-GAN) super-resolution

[rrats]

If Topaz Video Enhance AI is such a joke then why did Janus Film use it for their new restoration of David Lynch's Inland Empire?

Marketing. They had also done the same thing with Linus Tech Tips.

When's the last time a commercial release was prepared using Avi Synth scripts?

Bamboozled by The Criterion Collection. They even wrote a blog-post about it's restoration