Strange issue DeInterlacing DV capture - QTGMC-Bob (Hybrid Vapoursynth)

[SupermanTV]

Hello.. First post, I'm new around here...

I'm having a strange issue when De-interlacing (QTGMC Bob) DV captured videos, using Hybrid.
[ProRes, very slow, order 2 for Lower field first]

Analyzing the output frame by frame, I noticed:
a - Every 'x' amount of frames, seems like it adds a 'prior' frame (so the video kind of rewinds momentarily before continuing)...
b - When I try it without Bob (29.97), instead of adding a 'prior' frame, it 'repeats' a frame, giving a momentary pause to the video...

Strange thing is that doing .mp4 comes out nice, but I need ProRes for further editing...

Hopefully someone around here can shed more light on this...
Thank you in advance.

** Selur (Hybrid) suggested using sRestore, but specially at 29.97...

- - - Links to test videos (41 sec), as well as the original:

(ProRes QTGMC-Bob 59.94) https://drive.google.com/open?id=1nif8-2mQlyviry_43eCRYtiBv8iBdH9h

(ProRes QTGMC 29.97) https://drive.google.com/open?id=1sTVhzfaIRJXSkrKGAkiQ0KYh9Oa3PAwK

(MP4 QTGMC-Bob 59.94) https://drive.google.com/open?id=1LhepUFweZS5WqDfHl3VqjGZVivLwIyUf

(Original DV avi) https://drive.google.com/open?id=1_N2nKAqiCF2oh6E0pMKRQIZ68ZC1gJDD

[poisondeathray]

Can you fix your links ? 3 are truncated (...) . The original is important one.

[SupermanTV]

Thanks for the heads-up, links fixed!!

[poisondeathray]

Did you use the same script for both the 59.94 mp4 , and the 59.94 prores mov ?

I see the prores problem. The source does have a few issues - but you should get the same thing for both 59.94 files

What were the command lines used ? or have a look for the log files that hybrid writes somewhere (I'm assuming it writes a log file)

If you discussed with Selur somewhere already , post the link to the thread so you don't have to retype any already mentioned background info

[SupermanTV]

Yes, I used the same QTGMC settings in Hybrid...

I'm attaching both Debug files, is that good enough?
Also, here's my discussion with Selur: https://forum.selur.net/showthread.php?tid=1096

[jagabo]

I used ffmpeg and had no problem with out of order frames:

AviSource("Crespo_Source.avi", pixel_type="YUY2") 
QTGMC()
Spline36Resize(640,480)

ffmpeg -i "input.avs" -vcodec prores_ks -profile:v proxy -quant_mat:v hq -pix_fmt yuv422p "output.mov"

[poisondeathray]

Yes, log file is fine. And log looks ok

When I do it locally (copy paste everything - script for vpy, vspipe command line to ffmpeg) , it works fine . That suggests an issue with that ffmpeg binary , since both used essentially the same script (both MP4 and MOV, except MOV had 1 extra step to scale to yuv422p10)

My prores file is a lot larger 367MB compared to your 101MB "Crespo_QTGMC-Bob.mov" prores file

looking at mediainfo, some strange entries in your file . I'll highlight the ones of interest

Writing application : FFmbc 0.7
Codec ID : apco

The log says ffmpeg is used, not ffmbc , are you sure you didn't mix up the files either the video or log file somewhere ?

It should be "apcn", because -vtag apcn was used with -profile:v 2 according to the log. (apcn is "normal" prores; apco is for prores proxy)

[poisondeathray]

Actually I didn't look farther down the log; he uses ffmbc to re-wrap the mov for some reason. I'm wondering if that caused the issue . Re-wrapping it using his same command works ok here, no frames out of order . But it's still apcn, the Writing application however matches FFmbc 0.7

[SupermanTV]

Thanks, yes, I see the good result you get!!

I did produce those debug files today in order to post them here, and I may have done proxy the first time...

I don't have the mp4 original debug file...
Here's the ProRes original debug:

[poisondeathray]

It works ok for apco here too . I think the vapoursynth part is ok; especially since your MP4 came out as expected. It might have something to do with that ffmpeg binary used by hybrid (you can try replacing it by dropping in a replacement) or some threading issue that you're having

The log says ffmpeg -threads 8 (and that's what I used too, but I'm on Intel) , but frames out of order like that suggests maybe some threading problem, maybe some AMD issue ? According to the log, the MP4 used x264 , not ffmpeg to encode

[SupermanTV]

Ok, I'll try replacing the binary... (I also tried UTVideo with same bad result...)
Thanks so much!!

[poisondeathray]

Ok, I'll try replacing the binary... (I also tried UTVideo with same bad result...)
Thanks so much!!

And I'm assuming hybrid encoding with UTVideo is performed through the same ffmpeg.exe ? (not VFW version of UT video ?)

Also , why not use x264 as the "editing" intermediate ? Why does it have to be prores ? What program are you using ?

[SupermanTV]

It's for Premiere Pro CC, where I actually like using Cineform, but not available through Hybrid...
Basically, I'm trying to bring the highest possible quality of this DV footage into Premiere; then a bit of color correction/editing into a 1280x720 final output...

[poisondeathray]

It's for Premiere Pro CC, where I actually like using Cineform, but not available through Hybrid...
Basically, I'm trying to bring the highest possible quality of this DV footage into Premiere; then a bit of color correction/editing into a 1280x720 final output...

x264 lossless is actually supported by PP CC , and is treated correctly as YUV and is actually....lossless (unlike many NLE's which do not handle it correctly)

But for other editors - even at very high bitrates , like low CRF values 1-4 maybe, will be significantly higher quality - than even prores XQ4444 or cineform filmscan 2

SD is usually not a problem for editing latency, but the timeline/scrubbing performance is "sluggish" compared to cineform , even with --tune fastdecode and --keyint 1 . Cineform is really good in that regard, even if the quality is "lower" (splitting hairs here at filmscan 2)

[SupermanTV]

Interesting, I'm going to look into it in Hybrid settings then...
Thanks for pointing me in that direction!!

[poisondeathray]

If you really wanted to use cineform, you could use the script generated by hybrid and use that as input into vdub2 . vdub2 supports both the cineform sdk (it has a filmscan 3), and the official go pro cineform implementation (filmscan 2 is the highest), as well as .vpy scripts . I still use cineform occasionally for UHD projects, when I'm not using proxy edits.

But the observations here suggest some issue with that ffmpeg binary since the x264 on your setup encoded as expected (vspipe was used for both the MP4 and MOV)

[SupermanTV]

Indeed, seems like the issue is somewhre within the ffmpeg binary implementation (or ffmbc), though I did replace them..

I did try x264 lossless and it looks pretty good...
Question about it: Since I have SD, do I stay with i420, or go i422/i444?

* I don't use vDub, but I might look into it as well for the Cineform option...

[poisondeathray]

Replacing the ffmpeg did not work ? Some work for selur to do then, because that should not happen since your x264/mp4 output did not have the issue (it can't be some global computer or hardware issue)

I would use i420, because for your source (if they are all similar) it already exceeds NTSC DV chroma . The rest is just upscaling the chroma, ballooning the filesize for nothing

The decoded lossless will be the exact output of the script processing - ie. no compression losses. If you used uncompressed video, it would look the same

But if you don't need truly lossless, you can choose any quality in between . There is a filesize tradeoff. It's difficult to see the difference even pixel peeping zoomed in on frames with low quantizers/high bitrates . Metrics, however, can see the difference

[SupermanTV]

Same result with the 'other' ffmpeg file...

I did try lossless i420/i422 and it produces comparable ProRes regular and HQ sizes, so that's fine I guess..
(i444 is almost double the size..)

Thanks a lot for all your input and help, I really appreciate it..
Now I have a few options to play with..

[nightowl3090]

In 2023 this situation still seems yet to be resolved. DV video -> Hybrid QTMGC bob -> ProRes 422 HQ = Frame order alternating 0, 2, 1, 3, etc.
DV flagged top field first by source. This is on an AMD multi-threaded CPU.
Tried with all 3 field order settings -1, 0, 1 and all gave the same result. Interestingly it doesn't look that bad when played in VLC, just looks 'jumpy'. But frame by frame in Vegas Pro shows improper frame ordering. Any fix on this other than just having to use x264 lossless for the export?

[jagabo]

DV is normally BFF.

[nightowl3090]

Sadly exporting as x264 did not solve my issue.

I'd really appreciate it if someone could take a stab at this. Here's a short clip from the video captured with WinDV. Lots of movement so the frame order problem is obvious.

https://drive.google.com/file/d/1wMSthKT5B0FhcU1m2g8wpykJm5NlOdr9/view?usp=sharing

# Imports
import vapoursynth as vs
import os
import ctypes
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
import sys
# getting Vapoursynth core
core = vs.core
# Import scripts folder
scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/FFT3DFilter/fft3dfilter.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/EEDI3m.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll")
core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import edi_rpow2
import havsfunc
# source: 'E:\VHS Recordings 2022 RAW\DV Tapes\Frame Order Test Source.avi'
# current color space: YUV411P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: top field first
# Loading E:\VHS Recordings 2022 RAW\DV Tapes\Frame Order Test Source.avi using LWLibavSource
clip = core.lsmas.LWLibavSource(source="E:/VHS Recordings 2022 RAW/DV Tapes/Frame Order Test Source.avi", format="YUV411P8", stream_index=0, cache=0, fpsnum=30000, fpsden=1001, prefer_hw=0)
# Setting detected color matrix (470bg).
clip = core.std.SetFrameProps(clip, _Matrix=5)
# Setting color transfer info (470bg), when it is not set
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5)
# Setting color primaries info (), when it is not set
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 29.97
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=2) # tff
# adjusting color space from YUV411P8 to YUV444P8 for VsQTGMC
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, range_s="limited")
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=True) # new fps: 59.94
# Making sure content is preceived as frame based
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive
# resizing using ZNEDI3
# current: 720x480 target: 2560x1440 -> pow: 4
clip = edi_rpow2.nnedi3_rpow2(clip=clip, rfactor=4) # 2880x1920
# adjusting resizing
clip = core.fmtc.resample(clip=clip, w=2560, h=1440, kernel="lanczos", interlaced=False, interlacedd=False)# before YUV444P8 after YUV444P16
# adjusting output color from: YUV444P16 to YUV422P10 for ProResModel
clip = core.resize.Bicubic(clip=clip, dither_type="error_diffusion", format=vs.YUV422P10, range_s="limited")
# set output frame rate to 59.94fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=60000, fpsden=1001)
# Output
clip.set_output()

The above code is with the problem present.

Edit: I just noticed the 'overwrite input scan type to' option on QTGMC and I set that to BFF. This seems to be solving the issue of frame order. Additionally setting input order to 0 (BFF) solves all issues with smoothness. Thank you jagabo!

Any idea why WinDV is marking my catpures as TFF? MediaInfo lists my files as Scan Type: Top Field Only.

Here it is with the 'fixed' override input to BFF and set field order to 0 (BFF)

clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=1) # bff
# adjusting color space from YUV411P8 to YUV444P8 for VsQTGMC
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, range_s="limited")
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=False) # new fps: 59.94

[poisondeathray]

clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=True) # new fps: 59.94

Works ok for me. It's BFF

Change the QTGMC setting to TFF=False

If that doesn't work for you - it some hybrid specific issue, because it works in vapoursynth and avisynth ok . Selur will need to fix it

[jagabo]

Yes, the clip is BFF. I don't know why MediaInfo says it's "top field only".