Hello.. First post, I'm new around here...
I'm having a strange issue when De-interlacing (QTGMC Bob) DV captured videos, using Hybrid.
[ProRes, very slow, order 2 for Lower field first]
Analyzing the output frame by frame, I noticed:
a - Every 'x' amount of frames, seems like it adds a 'prior' frame (so the video kind of rewinds momentarily before continuing)...
b - When I try it without Bob (29.97), instead of adding a 'prior' frame, it 'repeats' a frame, giving a momentary pause to the video...
Strange thing is that doing .mp4 comes out nice, but I need ProRes for further editing...
Hopefully someone around here can shed more light on this...
Thank you in advance.
** Selur (Hybrid) suggested using sRestore, but specially at 29.97...
- - - Links to test videos (41 sec), as well as the original:
(ProRes QTGMC-Bob 59.94) https://drive.google.com/open?id=1nif8-2mQlyviry_43eCRYtiBv8iBdH9h
(ProRes QTGMC 29.97) https://drive.google.com/open?id=1sTVhzfaIRJXSkrKGAkiQ0KYh9Oa3PAwK
(MP4 QTGMC-Bob 59.94) https://drive.google.com/open?id=1LhepUFweZS5WqDfHl3VqjGZVivLwIyUf
(Original DV avi) https://drive.google.com/open?id=1_N2nKAqiCF2oh6E0pMKRQIZ68ZC1gJDD
+ Reply to Thread
Results 1 to 24 of 24
Last edited by SupermanTV; 1st Aug 2019 at 20:50.
Can you fix your links ? 3 are truncated (...) . The original is important one.
Thanks for the heads-up, links fixed!!
Did you use the same script for both the 59.94 mp4 , and the 59.94 prores mov ?
I see the prores problem. The source does have a few issues - but you should get the same thing for both 59.94 files
What were the command lines used ? or have a look for the log files that hybrid writes somewhere (I'm assuming it writes a log file)
If you discussed with Selur somewhere already , post the link to the thread so you don't have to retype any already mentioned background info
Yes, log file is fine. And log looks ok
When I do it locally (copy paste everything - script for vpy, vspipe command line to ffmpeg) , it works fine . That suggests an issue with that ffmpeg binary , since both used essentially the same script (both MP4 and MOV, except MOV had 1 extra step to scale to yuv422p10)
My prores file is a lot larger 367MB compared to your 101MB "Crespo_QTGMC-Bob.mov" prores file
looking at mediainfo, some strange entries in your file . I'll highlight the ones of interest
It should be "apcn", because -vtag apcn was used with -profile:v 2 according to the log. (apcn is "normal" prores; apco is for prores proxy)
Actually I didn't look farther down the log; he uses ffmbc to re-wrap the mov for some reason. I'm wondering if that caused the issue . Re-wrapping it using his same command works ok here, no frames out of order . But it's still apcn, the Writing application however matches FFmbc 0.7
Thanks, yes, I see the good result you get!!
I did produce those debug files today in order to post them here, and I may have done proxy the first time...
I don't have the mp4 original debug file...
Here's the ProRes original debug:
Last edited by SupermanTV; 5th Aug 2019 at 22:59.
It works ok for apco here too . I think the vapoursynth part is ok; especially since your MP4 came out as expected. It might have something to do with that ffmpeg binary used by hybrid (you can try replacing it by dropping in a replacement) or some threading issue that you're having
The log says ffmpeg -threads 8 (and that's what I used too, but I'm on Intel) , but frames out of order like that suggests maybe some threading problem, maybe some AMD issue ? According to the log, the MP4 used x264 , not ffmpeg to encode
But for other editors - even at very high bitrates , like low CRF values 1-4 maybe, will be significantly higher quality - than even prores XQ4444 or cineform filmscan 2
SD is usually not a problem for editing latency, but the timeline/scrubbing performance is "sluggish" compared to cineform , even with --tune fastdecode and --keyint 1 . Cineform is really good in that regard, even if the quality is "lower" (splitting hairs here at filmscan 2)
If you really wanted to use cineform, you could use the script generated by hybrid and use that as input into vdub2 . vdub2 supports both the cineform sdk (it has a filmscan 3), and the official go pro cineform implementation (filmscan 2 is the highest), as well as .vpy scripts . I still use cineform occasionally for UHD projects, when I'm not using proxy edits.
But the observations here suggest some issue with that ffmpeg binary since the x264 on your setup encoded as expected (vspipe was used for both the MP4 and MOV)
Indeed, seems like the issue is somewhre within the ffmpeg binary implementation (or ffmbc), though I did replace them..
I did try x264 lossless and it looks pretty good...
Question about it: Since I have SD, do I stay with i420, or go i422/i444?
* I don't use vDub, but I might look into it as well for the Cineform option...
Replacing the ffmpeg did not work ? Some work for selur to do then, because that should not happen since your x264/mp4 output did not have the issue (it can't be some global computer or hardware issue)
I would use i420, because for your source (if they are all similar) it already exceeds NTSC DV chroma . The rest is just upscaling the chroma, ballooning the filesize for nothing
The decoded lossless will be the exact output of the script processing - ie. no compression losses. If you used uncompressed video, it would look the same
But if you don't need truly lossless, you can choose any quality in between . There is a filesize tradeoff. It's difficult to see the difference even pixel peeping zoomed in on frames with low quantizers/high bitrates . Metrics, however, can see the difference
Same result with the 'other' ffmpeg file...
I did try lossless i420/i422 and it produces comparable ProRes regular and HQ sizes, so that's fine I guess..
(i444 is almost double the size..)
Thanks a lot for all your input and help, I really appreciate it..
Now I have a few options to play with..
Last edited by SupermanTV; 2nd Aug 2019 at 09:24.
In 2023 this situation still seems yet to be resolved. DV video -> Hybrid QTMGC bob -> ProRes 422 HQ = Frame order alternating 0, 2, 1, 3, etc.
DV flagged top field first by source. This is on an AMD multi-threaded CPU.
Tried with all 3 field order settings -1, 0, 1 and all gave the same result. Interestingly it doesn't look that bad when played in VLC, just looks 'jumpy'. But frame by frame in Vegas Pro shows improper frame ordering. Any fix on this other than just having to use x264 lossless for the export?
DV is normally BFF.
Sadly exporting as x264 did not solve my issue.
I'd really appreciate it if someone could take a stab at this. Here's a short clip from the video captured with WinDV. Lots of movement so the frame order problem is obvious.
# Imports import vapoursynth as vs import os import ctypes # Loading Support Files Dllref = ctypes.windll.LoadLibrary("C:/Program Files/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll") import sys # getting Vapoursynth core core = vs.core # Import scripts folder scriptPath = 'C:/Program Files/Hybrid/64bit/vsscripts' sys.path.insert(0, os.path.abspath(scriptPath)) # Loading Plugins core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/FFT3DFilter/fft3dfilter.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/EEDI3m.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/libmvtools.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/scenechange.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/Support/fmtconv.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll") core.std.LoadPlugin(path="C:/Program Files/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll") # Import scripts import edi_rpow2 import havsfunc # source: 'E:\VHS Recordings 2022 RAW\DV Tapes\Frame Order Test Source.avi' # current color space: YUV411P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: top field first # Loading E:\VHS Recordings 2022 RAW\DV Tapes\Frame Order Test Source.avi using LWLibavSource clip = core.lsmas.LWLibavSource(source="E:/VHS Recordings 2022 RAW/DV Tapes/Frame Order Test Source.avi", format="YUV411P8", stream_index=0, cache=0, fpsnum=30000, fpsden=1001, prefer_hw=0) # Setting detected color matrix (470bg). clip = core.std.SetFrameProps(clip, _Matrix=5) # Setting color transfer info (470bg), when it is not set clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5) # Setting color primaries info (), when it is not set clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=5) # Setting color range to TV (limited) range. clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1) # making sure frame rate is set to 29.97 clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001) clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=2) # tff # adjusting color space from YUV411P8 to YUV444P8 for VsQTGMC clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, range_s="limited") # Deinterlacing using QTGMC clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=True) # new fps: 59.94 # Making sure content is preceived as frame based clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0) # progressive # resizing using ZNEDI3 # current: 720x480 target: 2560x1440 -> pow: 4 clip = edi_rpow2.nnedi3_rpow2(clip=clip, rfactor=4) # 2880x1920 # adjusting resizing clip = core.fmtc.resample(clip=clip, w=2560, h=1440, kernel="lanczos", interlaced=False, interlacedd=False)# before YUV444P8 after YUV444P16 # adjusting output color from: YUV444P16 to YUV422P10 for ProResModel clip = core.resize.Bicubic(clip=clip, dither_type="error_diffusion", format=vs.YUV422P10, range_s="limited") # set output frame rate to 59.94fps (progressive) clip = core.std.AssumeFPS(clip=clip, fpsnum=60000, fpsden=1001) # Output clip.set_output()
Edit: I just noticed the 'overwrite input scan type to' option on QTGMC and I set that to BFF. This seems to be solving the issue of frame order. Additionally setting input order to 0 (BFF) solves all issues with smoothness. Thank you jagabo!
Any idea why WinDV is marking my catpures as TFF? MediaInfo lists my files as Scan Type: Top Field Only.
Here it is with the 'fixed' override input to BFF and set field order to 0 (BFF)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=1) # bff
# adjusting color space from YUV411P8 to YUV444P8 for VsQTGMC
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, range_s="limited")
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=False) # new fps: 59.94
Last edited by nightowl3090; 17th Sep 2023 at 14:17.