so where's all the Topaz Video Enhance AI discussion?

[JoelHruska]

Furthermore, the issues present in a bad DVD transfer are not natural errors found on most sources. Bad DVD releases are special.

And as I wrote many posts ago, "restoring" a retail release is pointless. Whatever you do at home will eventually be far surpassed by a subsequent retail release. (And let's forego the conspiracies of "the made it bad so they could resell it later".) With few exceptions, such as censorship on latter versions, or ADD-fueled "fixes" by George Lucas.

I disagree. I waited 20 years for ViacomCBS to announce a remaster. They have not. I have made contact with multiple people who work on modern Star Trek and a few who worked on the original DS9 VFX team. To the best of their collective knowledge, there is no remaster in the works. I agree that it would make sense for ViacomCBS to do this, but the only word from VCBS is that they were not satisfied with sales of the Blu-ray TNG and will not be continuing with DS9 or VOY.

I didn't just decide to remaster DS9, though. I decided to write about it, partly in the hopes that other fans would find my work useful, and partly to raise the idea of a new DS9 remaster publicly. The stories I have written have collectively driven excellent traffic. They clearly interest people. I know of at least 15 people working on DS9 right now for their own collections. Considering how long I've spent polishing my technique, most of them have converted more of the show than I have. But many of those projects are based on the AviSynth scripts and ideas I published. Some of those ideas came from other folks. At this point, it's a community effort.

If ViacomCBS announces a new version of DS9 and VOY, I'll be thrilled. I'll consider it evidence that the collective efforts to raise interest in the topic from me and a lot of other people have worked. If they never announce a remaster, I'll have my improvements. And besides, I'll take what I've learned and use it to further punch up the remastered version. I've seen the TNG Blu-ray set. It could be improved, especially the scenes where *they* had to use upscaled footage. There are a few.

The techniques I have learned from this project transfer to other videos and I've used them in other restoration projects. In the process of learning how to restore DS9, I've also learned how to deinterlace, deblock, derainbow, composite video in DaVinci Resolve, extract edge masks for other uses, blend layers, and use complex filter chains. I had never edited a single video before I picked up this project. Now, I have an entire suite of tools and capabilities I never had before.

The secondary purpose of my work is to raise awareness of AI upscaling and its realistic capabilities. I've got stories planned for the near future on the exact capabilities of Topaz VEAI and how to use AviSynth and TVEAI together. This isn't strictly a hobby for me. I write about my projects and efforts as part of my job. I'm taking these ideas to Voyager next.

TLDR: If ViacomCBS announces a remaster of DS9 and VOY tomorrow, I get to keep everything I've learned about remastering video. I've already seen that the techniques work on other types of video, so the information transfers. No matter what ViacomCBS does, I win. I'll be first in line to buy the remaster, if one becomes available.

When I publish my final tutorials, I will have publicly created a version of Deep Space Nine that blows anything publicly available out of the water. I do not call it "HD" because it is not HD, but it *is* the best, most-polished DVD-source-based material I've ever seen. When I started this project, my goal was to create a tutorial people could follow. I decided it was worth the effort if one (1) person could use my work to improve their own viewing.

Based on reader feedback, I succeeded. My stories drove traffic. They drove interest. They sparked interest in AI and in older Star Trek. I've learned a tremendous amount and expanded my own capabilities. There is no downside here.

[rrats]

I've also learned how to deinterlace, deblock, derainbow, composite video in DaVinci Resolve,

avisynth is better than resolve for that kind of stuff. Just look at resolves deinterlacer, that thing is worse than yadif!

Also stop calling it a remaster, what you are doing is essentially a restoration. You are not re-editing it from original negatives/unedited prints.

[poisondeathray]

I've also learned how to deinterlace, deblock, derainbow, composite video in DaVinci Resolve,

avisynth is better than resolve for that kind of stuff

Not the compositing part (masks, roto, tracking, layers)

[JoelHruska]

I don't deinterlace in DaVinci. The workflow is AviSynth (deinterlacing, derainbowing, QTGMC), Topaz VEAI (multiple model outputs at multiple resolutions). The multiple AviSynth outputs and multiple TVEAI outputs are then combined in DaVinci. I'm still working out exactly how the composition should be done. Right now I use three AviSynth encodes to create six Topaz encodes. Then I composite a total of 10 layers created from six sources to generate final output. I'm still looking for ways to hit the same quality with a simpler workflow.

DaVinci for compositing and color-grading. AviSynth for deinterlacing, denoising, derainbowing, and other general improvements. TVEAI for upscaling.

"Restoration" is fine with me as a term.

[rrats]

Not the compositing part (masks, roto, tracking, layers)

Well yeah, obviously apart from colour grading and compositing

[lordsmurf]

I need to practice to improve my scripts.

We all do.

If ViacomCBS announces a new version of DS9 and VOY, I'll be thrilled.

Given ViacomCBS recent dedication to Trek, I do find it likely. The success of Picard has opened the door to more 90s Trek merch.

[JoelHruska]

I hope you're right.

[lordsmurf]

I hope you're right.

I would be shocked if 3 things didn't happen:
(1) We didn't get more TNG/DS9/VOY crossover character/story in Picard.
(2) We didn't get a better release of TNG/DS9/VOY, but likely 1st released to Paramount+ in HD/4K.
(2) We didn't get more TNG/DS9/VOY merch, mostly retro stuff, like pure re-release to BD/streaming, perhaps more licenses given out for toys/props/etc.

[rrats]

I wish they didn't throw out old film negatives. That has happened before with Monty Pythons flying circus. It's bluray is still from tapes

[Sharc]

I've also learned how to deinterlace, deblock, derainbow, composite video in DaVinci Resolve,

avisynth is better than resolve for that kind of stuff

Not the compositing part (masks, roto, tracking, layers)

It's going off topic, but is there a general way to import AviSynth scripts into NLEs? AviSynth and NLEs are complementary so it would be nice to have both in one. Now I do the preprocessing in AviSynth, export it to a low loss format and import it into an NLE for final editing (and re-encoding). Is there a better way to do it?

Edit:
Avisynth Virtual File System seems to work still under W10.
https://forum.doom9.org/showthread.php?t=133313

[Sandfly]

Yes, Topaz leaves artifacts in the form of patterns.

A lot of the stuff in this post regarding de-noise programs go over my head.
What would you recommend for de-noise with a good user interface. Win 10 64.
I find Neat Video is pretty good for reducing the larger compression blocking.

Do you have any suggestions for me to try.
Thanks.
SF

Now why am I one of the few using this software?

It's overpriced garbage. The "AI" marketing suckers newbies. You've been had.

It's just an upsizer, with sharpening, and Avisynth does better.

Topaz is/was a low-end Photoshop plugin company. Their stuff works, but always worse (sometimes MUCH worse) than slightly more manual methods.

A lot of those DS9 Topaz examples really suck, lots of artifacts.

[JoelHruska]

Yes, Topaz leaves artifacts in the form of patterns.

I composite them out using multiple encodes. Blending multiple outputs together eliminates this problem.

[JoelHruska]

Sharc,

I don't think so. I know that AviSynth can be run from within Topaz by creating a virtual file system. It allows you to mount an AviSynth script as a file. When Topaz reads the "file" (actually an AviSynth script), it runs the script against the file as part of its own processing. This allows for single-gulp ingestion as far as TVEAI is concerned... but I don't know a way to pull the same trick with DaVinci Resolve. I don't *think* it works.

[poisondeathray]

It's going off topic, but is there a general way to import AviSynth scripts into NLEs? AviSynth and NLEs are complementary so it would be nice to have both in one. Now I do the preprocessing in AviSynth, export it to a low loss format and import it into an NLE for final editing (and re-encoding). Is there a better way to do it?

Edit:
Avisynth Virtual File System seems to work still under W10.
https://forum.doom9.org/showthread.php?t=133313

The old avfs only works for x86 avs (but can work with x64 NLE). The new avfs is commandline driven and works for vapoursynth and avisynth x64 scipts. You can extract avfs.exe from the portable vapoursynth installer


It depends on which NLE -

NLE's typically have pixel format /fourcc compatibility issues. e.g. "YV12" 8-bit 4:2:0 is not a native pixel format in most NLE's, but "IYUV" is (also 8bit 4:2:0) .

Uncompresssed fourcc's that are not supported typically get converted to RGB , and are not lossless in editors that have YUV timelines (there can be issues, such as wrong matrix, interlaced vs. progressive)

Supported fourcc's get "pass though" and are lossless or get "preferred treatment". Typically, "UYVV" for 8bit422 and "IYUV" for 8bit420, and "v210" for 10bit422 . There is a UYVY , IYUV, v210 fourcc emulator for r55 or newer vapoursynth with the alt_output=x switch . "preferred treatment" - e.g. vegas olny works in RGB, but the imported file would get studio RGB treatment instead of computer RGB treatment

Resolve is different in that it was developed orginally for/on Macs. "AVI" container is not as well supported, even with "physical" files. Some older versions of resolve imported avfs AVI as v210 correctly (but you need vapoursynth ; v210 emulation is not supported by avs), but newer versions of resolve do not import any virtual AVI's of any format. A MOV container emulator would increase compatibility and has been requested numerous times, but nobody has taken up the task so far

[JoelHruska]

PoisonDeathRay,

I know the latest version of DaVinci implements MKV support, but that doesn't really get us anywhere, does it?

[Sharc]

Sharc,

I don't think so. I know that AviSynth can be run from within Topaz by creating a virtual file system. It allows you to mount an AviSynth script as a file. When Topaz reads the "file" (actually an AviSynth script), it runs the script against the file as part of its own processing. This allows for single-gulp ingestion as far as TVEAI is concerned... but I don't know a way to pull the same trick with DaVinci Resolve. I don't *think* it works.

Thanks. My primary application for now would be to provide a pre-processed input for an NLE like Shotcut. Shotcut can read .avi files and has its own filters, but these internal filters are not as powerful as those available in Avisynth. This seems to work so far.
DaVinci can't read .avi sorces directly, one has to convert the format to ProRes (or similar) AFAIK.

[poisondeathray]

PoisonDeathRay,

I know the latest version of DaVinci implements MKV support, but that doesn't really get us anywhere, does it?

Not really in terms of "virtual" files or emulation

[poisondeathray]

DaVinci can't read .avi sorces directly, one has to convert the format to ProRes (or similar) AFAIK.

High compatibility for MOV and image sequences. Low compatibility for AVI /VFW . Older versions worked for some types of AVI, and in those version the v210 emulation worked too

v210 is the standard currency for professional NLE's, even Resolve (but it has to be in MOV). Although Resolve works internally in RGB (float) v210 gets passthough treatment as YUV

[Sharc]

It's going off topic, but is there a general way to import AviSynth scripts into NLEs? AviSynth and NLEs are complementary so it would be nice to have both in one. Now I do the preprocessing in AviSynth, export it to a low loss format and import it into an NLE for final editing (and re-encoding). Is there a better way to do it?

Edit:
Avisynth Virtual File System seems to work still under W10.
https://forum.doom9.org/showthread.php?t=133313

The old avfs only works for x86 avs (but can work with x64 NLE). The new avfs is commandline driven and works for vapoursynth and avisynth x64 scipts. You can extract avfs.exe from the portable vapoursynth installer


It depends on which NLE -

NLE's typically have pixel format /fourcc compatibility issues. e.g. "YV12" 8-bit 4:2:0 is not a native pixel format in most NLE's, but "IYUV" is (also 8bit 4:2:0) .

Uncompresssed fourcc's that are not supported typically get converted to RGB , and are not lossless in editors that have YUV timelines (there can be issues, such as wrong matrix, interlaced vs. progressive)

Supported fourcc's get "pass though" and are lossless or get "preferred treatment". Typically, "UYVV" for 8bit422 and "IYUV" for 8bit420, and "v210" for 10bit422 . There is a UYVY , IYUV, v210 fourcc emulator for r55 or newer vapoursynth with the alt_output=x switch . "preferred treatment" - e.g. vegas olny works in RGB, but the imported file would get studio RGB treatment instead of computer RGB treatment

Resolve is different in that it was developed orginally for/on Macs. "AVI" container is not as well supported, even with "physical" files. Some older versions of resolve imported avfs AVI as v210 correctly (but you need vapoursynth ; v210 emulation is not supported by avs), but newer versions of resolve do not import any virtual AVI's of any format. A MOV container emulator would increase compatibility and has been requested numerous times, but nobody has taken up the task so far

Thank you for these explanations. I think it explains what I have occasionally experienced but I had no clue about the cause and context.

[JoelHruska]

Well, you can at least reduce the number of encoding steps if you use AVFS with Topaz. TVEAI can output to frames and DaVinci can import them, so the issue of re-encoding quality loss can be avoided. TVEAI will dump PNG and TIFF (8-bit or 16-bit for TIF). DaVinci reads both PNG and TIF. So I suppose you could mount a video using AVFS, encode + upscale to lossless image output, then composite the outputs together in DaVinci.

I don't think there'd be any data loss this way.

But it's not the one-stop ingestion into NLE that you wanted, either.

[Sandfly]

@lordsmurf

What would you recommend for de-noise with a good user interface. Win 10 64.
I find Neat Video is pretty good for reducing the larger compression blocking.

Do you have any suggestions for me to try.
Thanks.
SF

(Not interested in Topaz)

[JoelHruska]

@Sandfly,

LordSmurf has not answered, so I thought I'd respond to you. I understand you aren't interested in Topaz.

When you say you want a GUI, are you counting AviSynth-based tools like StaxRip / Hybrid / MeGUI, or are you looking for something baked into an NLE like Neat Video?

There are a number of good denoising filters in AviSynth, but all of them use GUIs like the ones above. It's still driven by text settings under the hood, though some AviSynth front-ends also support preview modes that let you look at denoising before you hit "Render."

I haven't purchased and used Neat Video but I've experimented with the demo. It seems very good at what it does. I know of some different tools to use in AviSynth, but nothing I'd say is stronger / more powerful than Neat Video, just different.

[sophisticles]

Also stop calling it a remaster, what you are doing is essentially a restoration. You are not re-editing it from original negatives/unedited prints.

I was actually going to say the same thing. A master is what's created when you take the original footage, special effects, audio, etc and create a final, finished product that will be used to create the deliverables, the copies meant for end users.

A remaster would be if you literally redid the master, either with new footage, new special effects, perhaps with higher resolution footage.

What these guys are doing is not remastering, they having remastered squat. At most they have enhanced the end user version, though even that is debatable in some instances.

[lordsmurf]

@lordsmurf
What would you recommend for de-noise with a good user interface. Win 10 64.
I find Neat Video is pretty good for reducing the larger compression blocking.
Do you have any suggestions for me to try.
Thanks.
SF
(Not interested in Topaz)

Learn Avisynth.
Start with Hybrid, use the GUI, learn from it.
Move to AvsPmod and writing scripts.

[poisondeathray]

but newer versions of resolve do not import any virtual AVI's of any format

Just an update; avfs does work on current resolve version 17.4 for v210 and RGB30 for vpy scripts . (I had configured an improperly configured firewall that was blocking resolve from the virtual directory)

For avs scripts, only "rgbp10" , because avisynth doesn't have v210 emulation (p210 is generally not accepted by any NLE type programs)

[JoelHruska]

I was actually going to say the same thing. A master is what's created when you take the original footage, special effects, audio, etc and create a final, finished product that will be used to create the deliverables, the copies meant for end users.

As I said: Totally fine with calling it a restoration.

[_Al_]

@lordsmurf
What would you recommend for de-noise with a good user interface. Win 10 64.
I find Neat Video is pretty good for reducing the larger compression blocking.
Do you have any suggestions for me to try.
Thanks.
SF
(Not interested in Topaz)

Learn Avisynth.
Start with Hybrid, use the GUI, learn from it.
Move to AvsPmod and writing scripts.

or, as long he is just starting:
Learn Vapoursynth (it uses python, which is more than handy anyway, Avisynth code on its own is no good for anything else)
Start with Hybrid, use GUI, learn from it.
Move to some Vapoursynth tool that create scripts or use any of many python editors.
Vapoursynth is using many of upscaling (so called AI workflows) because it is python and those methods for python are just rapidly popping up. Then they are ported to vaporsynth. Particularly user named HolyWu contributes enormously doing so.

[forart.it]

Particularly user named HolyWu contributes enormously doing so.

He has just ported SwinIR (requested by me) !

https://github.com/HolyWu/vs-swinir

Can someone make some "real world" test and post results ?

[forart.it]

Selur published some SwinIR tests here:

here are a few examples:

used:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DeinterlaceFilter/TIVTC/libtivtc.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/d2vSource/d2vsource.dll")
# source: 'E:\clips\VTS_02_1-Sample-Beginning.demuxed.m2v'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: telecine
# Loading E:\clips\VTS_02_1-Sample-Beginning.demuxed.m2v using D2VSource
clip = core.d2v.Source(input="E:/Temp/m2v_5d36292e1f7f53fd6e26be51d50bbf8c_853323747.d2v")
# making sure input color matrix is set as 470bg
clip = core.resize.Bicubic(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 29.97
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# Deinterlacing using TIVTC
clip = core.tivtc.TFM(clip=clip)
clip = core.tivtc.TDecimate(clip=clip)# new fps: 23.976
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
# DEBUG: vsTIVTC changed scanorder to: progressive
# cropping the video to 704x480
clip = core.std.CropRel(clip=clip, left=8, right=8, top=0, bottom=0)
from vsswinir import SwinIR
# adjusting color space from YUV420P8 to RGBS for VsSwinIR
clip = core.resize.Bicubic(clip=clip, format=vs.RGBS, matrix_in_s="470bg", range_s="limited")
# resizing using SwinIR
clip = SwinIR(clip=clip, task="real_sr_large", scale=4, tile_x=352, tile_y=240, tile_pad=16, device_type="cuda", device_index=0) # 2816x1920
# adjusting resizing
clip = core.fmtc.resample(clip=clip, w=1920, h=1474, kernel="lanczos", interlaced=False, interlacedd=False)
# adjusting output color from: RGB48 to YUV420P8 for x264Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="470bg", range_s="limited")
# set output frame rate to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# Output
clip.set_output()

some more using RealSR_large:




Cu Selur

[rrats]

Tbh I don't see a point in artificial upscaling real-life video. It makes sense for stuff like anime, but not for something that.. exists. In most cases it produces results that look like they came straight out of the uncanny valley. I don't think we were supposed to go beyond upscalers that remove anti aliasing