Merging Two Captures Into A Single Better Result?

[Okiba]

I'm done capturing the last tape today
It took me a while longer, as I was finishing my father's Camcorder, a lot of friends/family wanted me help in converting their important events in life (wedding etc) - and you can't say no to that

Anyhow, I don't have professional setup, but I was able to capture pretty good results (in my own opinion at least). The Camcorder smaller tapes were MUCH easier to capture properly compared to the big standard tapes family/friends gave me. That made me believe if I was lucky enough to have to the smaller camcorder tapes, capturing just bigger tapes would have been a nightmare without an internal TBC/DNR (timing issues, comets, green pixels and what not).

However, at the end of the Camcorder life, some tapes had had some major TBC issues (probably the camera getting old). Lucky me, It's only 4 tapes. You can see how the capture went by checking the 'Combo.avi" file. That's was captured with a Combo Deck. The signal is being digitized and while it means the signal isn't "raw",the digitized results are pretty good (in other videos at least). In this specifically, it made the errors more clearly visible (see the flicking black lines). I have another Deck with just VHS and Component out - the results is at "NonCombo.avi". As you can see, everything is more raw. It's not as sharp as the Combo image, much more noise (you can clearly see the digitized signal has been fiddling with to look better). However, those very visible black lines with error - are much less visible on when there is no correction being done by the VCR. You can still see the error, but not so harsh.

So since I'm about to starting converting all lossless files into de-interlaced versions (while keeping the source of-course). I wonder if I have to do anything with that problem. Can I use the two samples and create a third better one? the median filter I was checking require 3 samples. Should I keep the Combo sample because there's something I can do about the harshness of the black lines? Or maybe keep the Non Combo sample as I can pretty much bring it close to the Combo quality (minus the blinking black lines) with AviSynth plugin? Perhaps I should keep both samples (each is around 100G, so...)?

And saving opening another thread - my conversion to FFMPEG AviSynth is pretty basic. It has Brightness/Contrast correction, Chroma Offset correction, Cropping, Chroma sharpening and De-interlacing using the QTGMC. I wonder if there's anything else generic that can be safely applied to 95% of the videos making the end-results better? maybe some basic noise cleaning? sharpening? (I know QTGMC seems to do a bit of sharpening magic).

Thank you again everyone!

[jagabo]

flickering black lines? I didn't see anything like that in combo.avi. Maybe you mean the flickering brightness, especially in the sky?

[Okiba]

Yes. It's brightness that flickers? You can see the the non-combo.avi video too, but it's much harder to see.

[johnmeyer]

I've read your really long post twice, but still don't know what you're trying to do.

If you want to combine captures from two different sources, that is tough because captures from analog material don't always line up all the way through.

If you want to switch between two captures, taking video from one when it has the better video and then from the other when you like it better, that is easily done in any NLE.

The Combo.avi file has serious timebase errors. Also, the levels are not right at all as you can see from the attached histogram. The time/date overlay should show up as spikes on the right side of the display, going up to close to 100.

The NonCombo.avi file has very few timebase errors, and the levels are much closer to normal.

It is hard to believe that your combo deck's results look better on other videos because on this one, the result is massively worse.

Deinterlacing is not needed unless you are going to use restoration filters that require a deinterlaced source.

[Okiba]

I've read your really long post twice

It is tad long isn't it...
Being more to the point, I guess what I'm asking is:

a. Is there a way to perhaps create a third, corrected video, by analyzing each frame from both the samples (which If I understand you correctly isn't possible because analog material don't always line up).
b. If the that's scenario is not possible - which capture I have better chance to bring to a decent condition (that's the one I'm going to archive).

It's not very clear from the this section of the video, but on better captures - the Combo unit had better results all around compared to the non-combo deck. I assume that that's a side effect of the digitized signal. Everything is a bit more sharp, less noisy, and even more stable TBC-wise from the non-combo captures. So while the timing is very bad on THIS combo capture, I'm not auto-throwing it away in case I can salvage better results out of it.

The time/date overlay should show up as spikes on the right side of the display, going up to close to 100.

It is hard to believe that your combo deck's results look better on other videos because on this one, the result is massively worse.

Here's a video taken from the same capture setup, and from the same source (Camcorder). It's not perfect, and you can see the date is a bit wiggly, but it's not that bad. The Overscan areas are masked, so you won't see the wiggly edges, but I can assure you it was no where near the capture I shared earlier. One of the theories we spoke about in another post was that the Camcorder starting to introduce major TBC issues as It was getting older. I have 10 years of footage from the same Camcorder, and only the last 4 tapes (chronologically) suffer from that.

Also, the levels are not right at all as you can see from the attached histogram.

Ermm, your right. Could that be a side effect of that brightness flickering?

Deinterlacing is not needed unless you are going to use restoration filters that require a deinterlaced source.

Well, the original plan was not to deinterlace the compressed format (the lossless file will always stay interlaced ofcourse), but I was following jagabo tip here. And now that I de-interlaced couple of videos - I have to agree. It's probably a side effect of QTGMC doing much more then just de-interlacing, but everything looks much sharper and better with QTGMC. And the Camcorder footage seems to benefit from 50FPS instead of 25FPS.

Long post again. /sign off
Thanks johnmeyer!

[jagabo]

a. Is there a way to perhaps create a third, corrected video, by analyzing each frame from both the samples

I don't think so. All that time base wiggle makes it difficult. But it would have been more helpful if you had posted overlapping samples. I don't think any frame appears in both videos.

b. If the that's scenario is not possible - which capture I have better chance to bring to a decent condition (that's the one I'm going to archive).

I think you'll get better results with noncombo.avi. That severe incoherent flickering in combo.avi will require very heavy filtering which will otherwise damage the picture.

[Okiba]

But it would have been more helpful if you had posted overlapping samples

I'm sorry! I was sure I did! I probably uploaded the wrong samples! Attaching identical footage (to the count of frame).

I think you'll get better results with noncombo.avi. That severe incoherent flickering in combo.avi will require very heavy filtering which will otherwise damage the picture.

The Audio in the Combo capture is WAY better. I assume again, there's some cleaning happening on the digitizing process. I think I will apply the sound from the Combo audio, to the non-combo video. And then delete the Combo footage.

I uploaded images from a different section of the same video. The upper part is pretty much destroyed because of the TBC issues - but if you will check the lower part, you will see the digitized version is a bit better. For example, you can see the noise reduction in action on Goofy's pants, and the lower feathery part of Donald Duck is more sharp on the Combo unit. Also, some section suffer from "green pixels" sometimes. The differences are much clear on some scenes with a lot of noise for example. But I assume - everything the VCR did - I can do later with AviSynth - and that's no a reason to keep that video.

Before capturing the other footage, I compared the TBC spikes on the Combo and Non Combo video, and the Combo was much better. Almost flat. So I assume the digitized process also tries to stabilized the picture a bit. It just those 4 tapes that suffering heavily from TBC issue from the "Good" capture device. It's almost like it's a fault of not the setup - but the digitizing process who can't handle well tapes with very bad TBC issues.

Thank jagabo!

[jagabo]

I don't see the combo video having better resolution than the noncombo video. It just has more analog oversharpening halos. The noncombo video has more noise but only in the chroma channels. Fortunately, you can apply really heavy noise filtering to the chroma channels without hurting the picture quality much.

[Okiba]

Thank you for checking into that jagabo. In that case, It seems like I have nothing to worry about by keeping just the non-combo version. I will apply the combo audio into the non-combo video, and will only keep the non-combo for archiving and later de-interlacing.

Will start the recursive de-interlacing filter after the weekend is done. If you guys think of any filter that can all camcorder videos can benefit from anything else besides Brightness/Contrast correction, Chroma Offset correction, Cropping, Chroma sharpening and De-interlacing using the QTGMC (subtle noise remover maybe?) let me know!

Thanks again you two!

[Sharc]

Unless I want to resize I prefer to leave interlaced sources as interlaced - but that's perhaps just me.
For denoising of VHS sources I often use SMDegrain (which has an interlaced option) or MCDegrainSharp, applied to even and odd fields.

[johnmeyer]

Unless I want to resize I prefer to leave interlaced sources as interlaced - but that's perhaps just me.

Me too.

[Okiba]

So let's talk about it for a second. QTGMC De-interlacing is very time consuming (I have around 3T of Lossless videos I would like to create viewable FFMPEG from). I wrote a small Python script that takes a generic .avs script (that suitable for all footage from the same setup), and I plan to run it recursively for probably 2 weeks (with preset "Slow" on FFmpeg and QTGMC on "Slower") on all the lossless videos. It's a dedicated machine, so I don't mind much about the time I'm going to take - but, here's what I know:

De-interlacing will happen anyhow right? If your not going to do - your TV going to do it based on it own logic. Or - if you using PC (let's say VLC). VLC let you choose all kind of run-time deinterlacer like Yapif, Yapif x2 etc. I also know that de-interlacing hurts quality on some level - However, when I compare a a video that has been de-interlaced by QTGMC and one I let VLC interlace, QTGMC videos looks better. Now, assuming that De-interlacing does effect quality to the worse - I assume QTGMC does more magic, like sharpening, noise reduction and perhaps other things. Also, the fact the videos are 50FPS instead of 25FPS is huge. Everything is much smoother.So why you guys not using QTGMC? The time it takes to de-interlace? the filter it automatically applies are too much?

Thanks!

[Sharc]

The pros and cons have already been discussed to death, just search the forums.
It's also a matter of personal preference, and there is often a compromise between 'technical correctness' and 'subjective viewing experience'.
Simply said I like to keep the original field structure intact (main point, in the sense of 'archiving') and even accept the awkward 4:2:0 YV12 interlaced color format after re-encoding.
Also to mention that not all of QTGMC's salient features under the hood are beneficial in all cases. This is no criticism at all on the excellent QTGMC bobber, and I don't want to convince anyone of anything.

[Okiba]

Sure, sound interesting. Will search the forum

Simply said I like to keep the original field structure intact

Oh, I own two copies of each video. One is interlaced, lossless HuffYuv file. They range from 5G to 100G. It's a bit of pain to stream 100Gb of-course, so I create "view-able" copy - which is FFMPEG file that has been QTGMCed. So in case one day with more experience I will find the QTGMC under the hood stuff too over-whelming, I can always create a new lossy file.

Also to mention that not all of QTGMC's salient features under the hood are beneficial in all cases.

While the extra noise reduction, or sharpness is (often) great, I don't mind watching the footage without any of that. What scores highly on my list is the 50FPS. Playing a lot of video-games in the past, my eyes are very delicate to FPS. 25FPS looks very "jumpy" for me. 50 Is much better. Perhaps I should look into that. Disabling other QTGMC enchantments, and only use the actual de-interlacing part for 50FPS. Wonder if it's easy as adding a flag, or you need to manually turn each of the things it does. Will check the documentation.

[jagabo]

for me. 50 Is much better.

VLC's Yadif 2x will get you 50 fps (though, if I remember correctly, it always assumes BFF). Any TV will get you 50 fps with its deinterlacing (assuming it's encoded and flagged correctly). Motion should be just as smooth with those. But you will get a lot more aliasing artifacts and buzzing on sharp, near horizontal, edges . QTGMC does a good job of reducing those problems.

[Okiba]

VLC's Yadif 2x will get you 50 fps (though, if I remember correctly, it always assumes BFF).

That would be a problem. I have mixed footage of both BFF and TFF.

assuming it's encoded and flagged correctly

FFMPEG Flags?

[jagabo]

FFMPEG Flags?

Assuming TFF YV12 output from AviSynth, a minimal x264 MBAFF tff command line:

ffmpeg -i input.avs  -c:v libx264 -preset slow -crf 18 -flags +ilme+ildct -x264opts tff output.mp4

For bff video change tff to bff.

[Okiba]

That's pretty cool. Didn't know that. I assume VLC will also honor those. Will test it out. I'm interested to see how QTGMCed video looks against non QTGMCed video when both are 50FPS.

[Okiba]

Did couple of tests. First of all, you write. VLC Yadif x2 can only do BFF. However, the "-flags +ilme+ildct -x264opts tff" flag options didn't work for me. The end results are still 25FPS. +ilme exists according to the documentation, but I couldn't find anything about ildct. Am I'm missing something? Here's the full command with a very-fast preset (just so I can quickly create examples):

ffmpeg64 -i "E:\test\test.avs" -c:v libx264 -preset veryfast -crf 18 -pix_fmt yuv420p -vf setsar=12/11 -flags +ilme+ildct -x264opts tff -c:a libfdk_aac "test.avi"

By the way, which de-interlacer FFMPEG use?

[Sharc]

However, the "-flags +ilme+ildct -x264opts tff" flag options didn't work for me. The end results are still 25FPS.

The command line is for interlaced encoding of interlaced sources.
What is your script? Did you leave a deinterlacer in the script? What does MediaInfo tell about your output? Do you see 'Scan type MBAFF'?
The interlaced output will still be 25fps (=25 interlaced frames per second) but with 50 fields per second. VLC will bob-deinterlace the 25 interlaced frames per second (2 fields each frame taken from different points in time) to give 50 frames per second.
Same what your TV would do.

[Sharc]

deleted (double post)

[Okiba]

No, no de-interlacing happen in the AviSynth script. All the script do Level() and Crop() (QTGMC isn't being called).
According to MediaInfo:

Scan type: Progressive
Original Scan Type: MBAFF
Scan type, stored method: Interleaved fields

And your right. While VLC reports 25FPS (unlike QTGMC file where it reports 50), MediaInfo will report FPS is 50. And yet, the video is much more choppy compared to QTGMC. I will create some short clips and share examples later when I'm home. Perhaps the VLC settings are incorrect? There's an "De-Interlace" option on VLC. You can set it to on/off, and when you set it to on - your expected to select de-interlaced (or it use Blend as default IIRC).

EDIT: Just picked something jagabo wrote and I missed, assuming YV24. I think the video is YV16. I will try setting it up to YV24.

[Sharc]

.... and output to .mp4 rather than to .avi as it may reverse the field order again ....

[jagabo]

the video is much more choppy compared to QTGMC.

That's because VLC always assumes BFF (a longstanding bug in the program). Your video is TFF. Since VLC is assuming the wrong field order (rather than reading what's flagged in the video data) it delivers a two-steps-forward-one-step-back cadence -- hence the jerky/flickery playback.

If you want to convert YV12 TFF video to BFF you can use:

SeparateFields().Trim(1,0).Weave()

With 4:2:2 or 4:4:4 chroma (or RGB) you can use:

Crop(0,0,-0,-1).AddBorders(0,1,0,0).AssumeBFF()

[Okiba]

a longstanding bug in the program

It sounds like VLC (and I assume, any TV) do de-interlancing in real-time? where QTGMC change the actual video file? Any alternative player that can do BFF?

I'm attaching two videos. One is QTGMed, and the other one isn't with:

Crop(0,0,-0,-1).AddBorders(0,1,0,0).AssumeBFF()

Here's the AviSynth script:

SetFilterMTMODE("QTGMC", 2)
AviSource("F:\test.avi")
ConvertToYV24(interlaced=true) 
RoboCrop()
MergeChroma(Levels(16, 1.0, 230, 16, 235, coring=false), last)

AssumeTFF()
QTGMC(Preset="Super Fast", EdiThreads=3)
#Crop(0,0,-0,-1).AddBorders(0,1,0,0).AssumeBFF()
Prefetch(3)

(Note I un-comment and comment the QTGMC section). The jerky playback is gone indeed. But - I'm pretty sure the camera movement in smoother on the QTGMC video. Almost like the FPS is higher? or that's not FPS but another filter by QTGMC?

[jagabo]

Try this:

q = LWLibavVideoSource("qtgmc.mp4", cache=false, prefer_hw=2) 
nq = LWLibavVideoSource("non-qtgmc.mp4", cache=false, prefer_hw=2).Yadif(mode=1, order=0)
StackHorizontal(q, nq)

Motion smoothness is the same. I used AviSynth's Yadif because that's similar to VLC's Yadif 2x. And it's typical to what most TVs do.

[lordsmurf]

Also to mention that not all of QTGMC's salient features under the hood are beneficial in all cases. This is no criticism at all on the excellent QTGMC

I say this often.
For example, preset=Slower is overrated, and I use preset=Faster far more often.

QTGMC is best, but you don't have to load up on switches, nor extended slower presets that mostly just drag out encode time.

[johnmeyer]

Also to mention that not all of QTGMC's salient features under the hood are beneficial in all cases. This is no criticism at all on the excellent QTGMC

I say this often.
For example, preset=Slower is overrated, and I use preset=Faster far more often.

QTGMC is best, but you don't have to load up on switches, nor extended slower presets that mostly just drag out encode time.

I agree 100%.

You can end up with stupidly long processing times for no discernible improvement if you go with the slower presets. As always find a representative 10-15 second clip and try deinterlacing & denoising with QTGMC first using one of the fastest, and then one of the slowest presets. Then put them both in your NLE and "A/B" between them to see if you can find any differences. If you DO see a difference, you will then know what sort of artifact to look for when you try one of the intermediate settings.

Even if you are a perfectionist, if you can't tell a difference, then you'd be a fool to double or triple the time it takes to finish your work by using one of the super-slow presets.

[Okiba]

Motion smoothness is the same

I trust you eyes more then I trust mine. I could be the sharpening of everything create this smoothness I'm talking about. Thank you for checking it!

For example, preset=Slower is overrated, and I use preset=Faster far more often.

Yea, I remember you mentioning Slower blurs. I tested it a while ago, and my eyes couldn't see a difference. Another reason to test Faster more often I guess

If you DO see a difference, you will then know what sort of artifact to look for when you try one of the intermediate settings.

I think that's a problem as a newbie. While I can see difference between QTGMCed video and non QTGMCed videos, I can't point what artifacts were getting "modified". Perhaps if I would be able to point my finger more easily - it would have been faster just to apply a specific filter with AviSynth - and let TV/VLC de-interlace the rest for me for faster encoding. But I totally agree about QTGMCs preset. I currently testing settings for couple of videos (mostly presets), pin-pointing the sweet spot for me.

[Sharc]

You may also want to apply QTGMC() on your non-qtgmc.mp4, i.e. substitute the yadif() by QTGMC() in jagabo's script, and compare with the qtgmc.mp4.
IMO this preserves more details, removes rainbows and looks sharper than the qtgmc.mp4

I have no idea what kind of 'magic' (or 'cheap') bob-deinterlacers TVs are using. jagabo mentions yadif or equivalent. I have not found any specs on data sheets or manuals. I would assume however that TV bobbers should be quite good as many TV stations broadcast HDTV as 1080i. Hence I would suggest to view and compare the 2 files on your TV as well rather than just VLC on PC.

Just another exercise