so where's all the Topaz Video Enhance AI discussion?

[Xood]

Thank you @JoelHruska, this really helps me to gain some new knowledge.
So it seems AviSynth was been continued as AviSynth+ and forged to VapourSynth which uses Python instead of the original scripting language.
I will check out the other tools you mentioned.

But the key takeaway is, it is possible to inject them into Topaz Labs Video Enhance AI which then already receives the on-the-fly changes of the video.
Essentially allowing me to De-Interlace, Crop, Scale, Border, and directly AI update with VEAI.

[lollo]

Thank you @JoelHruska, this really helps me to gain some new knowledge.

If it can help, you can look to this example from scratch, based on 32bit classic AviSynth: http://www.digitalfaq.com/forum/video-capture/12805-first-video-capture.html#post85085

Essentially allowing me to De-Interlace, Crop, Scale, Border, and directly AI update with VEAI.

Yes, a sample of a possible flow here: https://www.youtube.com/watch?v=LW6XWp9e5ws&list=PLCvrZEXO1laGXZ_BZ7bDO3xfB8hRD9WRd&index=2

[JoelHruska]

You definitely can. I've created a virtual drive and done single-ingestion into TVEAI before. I don't typically do it because the method I tested required 700GB of storage for a DVD episode in order to store the uncompressed video. It's a big chunk of storage when I'm already using 1-2TB for storing intermediate files.

I've been using multi-model blending extensively, so I have a bit of a different workflow, but single-ingestion into TVEAI can absolutely work.

[lordsmurf]

When people write stuff like this, you just know they've never used it.

I may have muddled the difference between installing a front-end like StaxRip or Hybrid or installing AviSynth,

Ah, okay, Hybrid is quite good, and it packages many filters.

[forart.it]

Here's an interesting (Selur's) VapourSynth script from a 3ad @ Doom9 about Cleanup of heavy compression artifacts that performs "QTGMC + BasicVSR++ with Model 5":

# Imports
import os
import sys
import ctypes
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("I:/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Import scripts folder
scriptPath = 'I:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/EEDI3.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/temporalsoften.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
# Import scripts
import havsfunc
# source: 'C:\Users\Selur\Desktop\Zim_01_compression_artifacts.avi'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: top field first
# Loading source using FFMS2
clip = core.ffms2.Source(source="C:/Users/Selur/Desktop/Zim_01_compression_artifacts.avi",cachefile="E:/Temp/avi_5e80f5192093a24d3bd5bffe5fc965ee_853323747.ffindex",format=vs.YUV420P8,alpha=False)
# making sure input color matrix is set as 470bg
clip = core.resize.Bicubic(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 29.970
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# setting field order to what QTGMC should assume (top field first)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=2)
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=True) # new fps: 29.97
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
clip = clip[::2]
# adjusting color space from YUV420P8 to RGBS for vsBasicVSRPPFilter
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# Quality enhancement using BasicVSR++
from vsbasicvsrpp import BasicVSRPP
clip = BasicVSRPP(clip=clip, model=5)
# adjusting output color from: RGBS to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, matrix_s="470bg", range_s="limited")
# set output frame rate to 29.970fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Output
clip.set_output()

Hope that inspires !

EDIT: MMEditing v0.15.0 has been released !

[SaurusX]

Hope that inspires !

It's clean, but at the expense of a lot of background detail.

[Selur]

It's clean, but at the expense of a lot of background detail.

Problem is the surrounding frames in the original don't really help:




BasicVSR++ was the only thing is could find which produced anything usable.
Would be nice to find a better (automated) alternative than BasicVSR++.
Source is still linked in the thread and available, so anything better is welcome.

Cu Selur

[greymalkin]

What a lively discussion . I will admit I too was getting starry eyed over the Topaz Video Enhancer AI product...but my key takeaway from these 18 pages is to give Hybrid a try . I love avisynth and I've seen it do what I thought was impossible...anything to make it easier is just gravy...

[forart.it]

Topaz VEAI v3 is coming...

Changes from 2.6.4

• Completely new app from the ground up
• Full frame stabilization
• Models can be chained now
• Support for various output formats
• Command line support
• Parallel execution support
• Ability to keep previews
• Color depth up to 16bit now supported
• Video files are playable on crashing (recovery coming soon)
• Colorspace and color inconsistency issues reduced significantly
• Many many more things

Known Issues

• Certain VOB files do not open correctly
• Some videos can still have some seeking issues in Preview/Export
• Color space / HDR issues
• Keyboard navigation missing
• Intel codec not working on some machines
• Some quality mismatch between 2.6.4 and 3.0
• App doesn’t clear the temp folder
• App only shows resolution in DAR and not in SAR
• Multiple selection for input videos is missing
• Different view mode is still missing
• Customized Output bitrate will not reset to Preferences bitrate when importing a new file
• Some failures can happen with some image sequences
• Cropping while deinterlacing can cause failures
• Failure can happen when resizing to a big resolution like 4k or 8k without a warning if machine runs out of memory
• Speed can be slower than 2.6.4 with some models on some machines. We’ll be working on speed optimization in the next weeks

Problems & Known Solutions

• If the output doesn’t open in other applications, disable Allow Recovery before processing in Preferences > Output (note: Allow Recovery is required for previewing exported video in the app while processing)
• Some Mac users who participated in the Alpha or Beta may experience failures with all models. If this happens make sure the data directory is pointing at the correct version of Video AI
• H.264 encoders have a maximum resolution of 4096x4096, use a different encoder for 4k or 8k videos
• Changing data folder path can cause issues. To resolve: manually create a folder and name it models in the new location and copy the files from old to new location

Proteus Auto Caveats
The adjustments will not be visible and the sliders will show values of “0” when in auto mode. However, each frame is being analyzed for optimal settings while processing. If you keep Auto enabled and modify the sliders, the settings you choose will be value-adjustment percentages. For example, if Auto estimates the Sharpen slider with a value of 50, and you manually select -50, then the actual value applied will be 25.

Source: https://community.topazlabs.com/t/topaz-video-ai-early-access-v3-0-0-0/33949

[Selur]

Known Issues

jjust reading these I wonder in which cases that Topaz Video AI is expected to work without issues.
-> that seems like an 'alpha' release at best,...

Problems & Known Solutions
H.264 encoders have a maximum resolution of 4096x40

Bullshit. x264 encodes 8k fine here. H.264 Level 6.0, 6.1, 6.2 were added to the standard back in 2016 iirc.
Level 6.2 allows:

resolution@fps (max refs)
3,840×2,160@300.0 (16)
7,680×4,320@128.9 (5)
8,192×4,320@120.9 (5)

see: https://en.wikipedia.org/wiki/Advanced_Video_Coding#Levels

Cu Selur

[lordsmurf]

-> that seems like an 'alpha' release at best,...
Bullshit. x264 encodes 8k fine

That's really it. Fools and their money are easily parted. Topaz crapware is inferior to Avisynth, Vapoursynth, x264, etc. The retort is always that those tools are hard (wah!), and Topaz has a whizbang lazy GUI. But it still doesn't make Topaz tools NOT crap. Just cheapware/junkware with a GUI, that lazy people can use to molest video quality. Those seeking quality will not be using it, and have not thus far.

Supposedly there is some limited success using Topaz + other tools. But I've still not seen proof of this.

Nothing I've seen to date was impressive, and in fact massively disappointing. The artifacts hurt my eyes to watch. It's been impressive how the software makes video so bad that it causes me to blink. So I guess there's that? Kudos Topaz?

FYI, their Photoshop plugins are equally junk, though not as bad as the video offering.

...but my key takeaway from these 18 pages is to give Hybrid a try .

Yep. Try it, use it.

And donate a few bucks to selur, so he can afford to keep releasing updates, expanding features. Of all the freeware video tools available, his is by far one of those most worthy for a donation. It's impressive what he has accomplished. Fork over $25, $50, buy the man a virtual beer. (Topaz junk is overpriced at $200, just ridiculous for how half@ss it works.)

[rrats]

There are VSGAN filters out there to fix the awful artefacts produced by the bloody veai

[Xood]

h.264 encoders have a maximum resolution of 4096x40

bullshit. X264 encodes 8k fine here. H.264 level 6.0, 6.1, 6.2 were added to the standard back in 2016 iirc.
Level 6.2 allows:

Maybe they are refering to some internal restriction. Just because the codec specs might support higher resolutions, does not imply every software is able to use it.

I am happy to see progress on the new Topaz VAI version, lets see where it is leading.

[johns0]

Topaz will always be junkware,let others come here and swear it's the best,when i see any ai program that can improve the scenery in any film i will say it did it but not know.

[rrats]

I agree with you

[Corvius]

Topaz is not junkware. There is a copycat program "Video Enhancer AI" that is junkware.

I never use it alone. I only use it for 2x scale model, or 4x scale model, then do everything else in Hybrid and grading applications. I use png 8 bit output until final compression.

I used to do upscaling in Hybrid until Topaz.

https://www.youtube.com/watch?v=sF26DDfQiJA

https://www.youtube.com/watch?v=YmyQbr6D7PI

Of course it's not like real true HD but I believe it can produce some of the best results possible today, for an individual on a single computer. The technology hopefully will improve.

I'm not actually liking the direction that Topaz VEAI is headed, as for 3.0, they are adding all these different modules that I think are actually done better in Hybrid/Avisynth/VapourSynth (like deinterlacing and frame interpolation), and they completely redesigned their user interface for reasons I'm not clear on, and of course it broke tons of things that are taking them a lot of time to try and fix. I think they are leaving the scope of upscaling and trying to also make it a filtering application like Hybrid, which I think is a mistake and looks to me like the beginning of feature creep / bloat.

[Corvius]

There is actually some freeware that can upscale pretty good, called Cupscale. It uses user created models.

But Topaz VEAI, at least in my testing, gives the best results, which I would expect from a commercial application.

https://www.extremetech.com/extreme/338403-what-video-ai-upscalers-can-and-cant-do

BTW, none of anything released so far is "AI", that's just a marketing term. I believe the more correct way to refer to these applications are "weighted databases". So, upscaling using weighted databases. I still don't understand why people here seem to be just outright disregarding the tech as "junk" when it demonstrably can do what it claims.

[Xood]

I still don't understand why people here seem to be just outright disregarding the tech as "junk" when it demonstrably can do what it claims.

Yes, I noticed this quite early here on the forums.
Especially those users keep stating their dislike. This leads to nothing but constant backslash.

Maybe a thread would be great, listing all the possible tools for video enhancement.
Then list or brainstorm about the pro, cons, and possible use cases.

I personally saw room for improvement in Topaz VEAI, hence I am happy they started the internal re-design.
Their code was most likely too limited to be shaped and extended, lets wait and see where it all leads.

[rrats]

It low-key is junkware. Basically you can either upscale with very strong noise reduction and end up with oversmoothened video or use some of their more detailed models and get alright results with bunch of artefacts. Sorta like what you get around the endges when using virtualdubs smoothen filter combined with sharpening

[lollo]

Maybe a thread would be great, listing all the possible tools for video enhancement.
Then list or brainstorm about the pro, cons, and possible use cases.

We have 3 entire and dedicated forums to video restoration: doom9, videohelp and digitalfaq, with tons of examples showing AviSynth and VapourSynth results. As we often say there is nothing Topaz VEAI can do that cannot be matched or beaten by AviSynth/VapourSynth.

However, used just as upscaler, Topaz VEAI may have an interest and produce good results for some videos, if integrated in a full AviSynth/VapourSynth flow, as we experienced earlier in this thread.

[lordsmurf]

https://www.extremetech.com/extreme/338403-what-video-ai-upscalers-can-and-cant-do

Aside from that article reading like a literary blowjob to Topaz, the major issue is he always conflates sharpening with scaling. Much of what Topaz does is simple turning the sharpen knob to 11.

[SaurusX]

https://www.extremetech.com/extreme/338403-what-video-ai-upscalers-can-and-cant-do

Aside from that article reading like a literary blowjob to Topaz, the major issue is he always conflates sharpening with scaling. Much of what Topaz does is simple turning the sharpen knob to 11.

To me, the article seems to present the pluses and minuses of Topaz fairly. It does what it does and there are caveats, which are presented. Topaz is just another tool in the box, like AVISynth. They can complement each other.

[lordsmurf]

Topaz is just another tool in the box, like AVISynth. They can complement each other.

I agree with this.

But Topaz is also that weird tool you bought on the way out of the hardware store, and it never worked correctly as advertised. So it's mostly a regretted impulse buy, often useless for the task. Sometimes it kinda works when you jam a screwdriver into the side, so you keep it.

[Störmbreäker]

Personally, i have tried and would like to learn Hybrid, but i'm using Topaz VEAI because it's 100x easier to understand. For example, what would it take in Hybrid to upscale a slightly grainy video from 720p to 4K with similar or better results than Gaia HQ can provide in Topaz' software?

[defbiz]

I still don't understand why people here seem to be just outright disregarding the tech as "junk" when it demonstrably can do what it claims.

Yes, I noticed this quite early here on the forums.
Especially those users keep stating their dislike. This leads to nothing but constant backslash.

Maybe a thread would be great, listing all the possible tools for video enhancement.
Then list or brainstorm about the pro, cons, and possible use cases.

I personally saw room for improvement in Topaz VEAI, hence I am happy they started the internal re-design.
Their code was most likely too limited to be shaped and extended, lets wait and see where it all leads.

I don't necessarily think it's junkware, however, it's highly underwhelming for such price when their 'robust' de-interlacing model can't even do it properly and instead leaves in severe ghosting and blending artifacts, has color space issues and sucks overall for low resolution video (not severely compressed, as I tried a Pro-Res 240p and it looked better Bilinear-resized than AI upscaled). The program is severely flawed, still.

[johns0]

It is junkware,they resigned their whole software which means there old stuff was junk.

[Corvius]

What wouldn't you consider junkware? :P

[johns0]

Hybrid and avisynth.

[rrats]

https://forum.videohelp.com/threads/406860-Is-it-actually-possible-to-upscale-480p-to-...sembling-1080p

useful thread for replicating VEAI looks (and getting even better results in some cases)

[VoodooFX]

junkware

It has its purpose, but most of its users use it to produce junk. Tardware is more accurate definition.