QTGMC image vibration, motion?

[lordsmurf]

I have home movie footage with the onscreen date. The date vibrates, sometimes worse, especially at camera panning. I've never seen this before, and don't know why I am now.

I've tried pretty much every syntax I can think of, every preset.

Why is this happening?

What are the current QTGMC alternatives, if any?

I've tried the same footage with Yadifmod, and the vibration not present. But the aliasing in Yadifmod is pretty miserable, so not an option -- unless you know of a better AA that matches QTGMC.

Samples attached.
- longer interlaced source file
- tiny QTGMC'd vibrating clip

[jagabo]

The character generator in camcorders and other devices usually generate two identical fields. That confuses QTGMC and it produces a bob, just like you would see on a CRT. You can overlay the original image in parts of the frame that are unchanging, or maybe parts of the frame where the two fields are identical (though noise and compression artifacts might interfere with detecting that). Something like I did here:

https://forum.videohelp.com/threads/395672-Shimmering-problem-after-encoding/page2#post2594209

You'll probably want to leave out the nnedi3 that was used to fix other issues with that particular clip.

[lordsmurf]

What would be the syntax for that attached sample?

And noting that the data comes and goes. It's not static 100% of the time.

In a semi-related note, I'm still using QTGMC-3.32.avsi, and decided to update to QTGMC.avsi (3.375), and get "I don't know what AvsPlusVersionNumber means". Any idea on that, or just back out of trying to update?

This appears to be a limitation of QTGMC. Is there an alternative?

[lordsmurf]

For a quick test, I added in this:

function AbsSubtract(clip v1, clip v2) { 
    Subtract(v1, v2).ColorYUV(off_y=2)
    Overlay(last.ColorYUV(off_y=-128), last.Invert().ColorYUV(off_y=-127), mode="add")
}
diff = AbsSubtract(last, Trim(1,0)).mt_binarize(10).mt_expand(chroma="-128").Blur(1.0).SelectEvery(1,0,0)
dup = PointResize(width, height/2).nnedi3(dh=true).SelectEvery(1,0,0)
Overlay(dup, QTGMC(), mask=diff)

And I do see improvement.

I'm now trying to figure out
- what this does
- how to tweak, if needed
- what the consequence to encode time is
- is this can work with x64

Any guidance appreciated.

EDIT: Faster preset, some improvement. Slower preset, the above script has no effect. It's obvious (?) that this is a motion search issue, and will need advanced syntax/switches. But which?

(And for now, just rolled back that QTGMC update attempt, don't need to complicate matters if the older version is fine, and I think it is.)

[jagabo]

You can fix the AvsPlusVersionNumber error by updating AviSynth Plus. Or you can just put

AvsPlusVersionNumber = 0

at the start of your script. I think most the improvements in QTGMC are for working with higher bit depth video.

I didn't see the samples when I originally answered. A quick look at them makes me think some other ways of dealing with might work better. I'll play around with it... At the very least, I'd limit the "fix" to the area where the date code is.

[lordsmurf]

I'm starting to think that 59.94 is simply a mistake.

QTGMC(Preset="Faster")
SelectEven

appears to be perfect.

I've never been fond of 59.94 anyway.

My added question now is whether it's just the date/clock giving the artifact, or if the rest of the frame is also affected.

Is the counter throwing off the frame, or is the frame throwing off the counter?

Interestingly, I was about 15 files in before noticing this. I've having to re-check the others with a much closer inspection now.

[johnmeyer]

I see zero problems with the original (non-QTGMC) clip. I turned of deinterlacing on VLC and played it there. The date/time text was rock solid.

I then imported it into my NLE (Vegas Pro) and played it, both at normal speed, and then rocking back and forth, frame-by-frame. Again, the only thing I saw were the normal interlacing "teeth" in the lane flags as the camera panned back and forth.

Finally, I put the mouse cursor over the top of the "1" at the right side of the date/time and then played the video slowly. The mouse cursor provides a fixed reference so that any actual movement of the text you be seen. I saw no movement whatsoever.

So, what I see is a really good capture (your equipment really does produce amazing results) with no artifacts that I can see. The camera motion can be reduced with Deshaker or any other competent stabilizer.

[edit]I then looked at the QTGMC clip. One issue is that you changed the aspect ratio. That is basically a resize operation, and that is going to cause problems, depending on whether the resize was done prior to deinterlacing or not. Jagabo already provided a possible explanation for the degradation of the edges of the text. Basically what you have is a progressive overlay on top of interlaced video. QTGMC, in the process of deinterlacing inevitably introduces artifacts into the progressive text.

BTW, in my initial post above I mention using a stabilizer, but having done that with video which has a date/time code burned into the video, you cannot use stabilization because it will make the date/time bounce all over the place. You'd have to first remove the date/time display, then do the stabilization, and finally take a rectangle containing the date/time from the original video, and re-introduce it to the stabilized video, feathering the edges. This would avoid the fuzzy spot that you get when you remove the date/time.

[lordsmurf]

Thanks! Yes, that original capture should be as perfect as possible. Every drop of quality was excised from the tapes. The interlaced master is steady, zero flaws. (The only "flaws" are inherent to VHS, the recording camera, and shaky handheld shooting. But nothing can be done about that now.)

It's the deinterlaced copy where I'm unsure if the entire frame is shaking, or if just the date is shaking. The problem is only visible on motion, all handheld shots, and it's not always easy to tell the difference between motions. QTGMC and bobbing/59.94 seems to be the culprit here.

BTW: Do you get stuttering on VLC with Yadifx2 mode?
I don't know what causes that. It's random. Obvious TFF/BFF read issue. I don't believe that can be adjusted in setting anywhere. Common bug, for years, and never been fixed.

Sometimes x2 is pretty clean. I'm wondering if 59.94 Yadifx2 can be done in Avisynth. Hmmm....

29.97 is still looking to be the best solution. Still testing.

EDIT: The AR is not different. Both samples are 720x480, SAR+DAR 3:2

[johnmeyer]

Several things to report.

1. Your original is 720x480 with a PAR of 0.9091. The QTGMC output is 720x480 with a PAR of 1.000. Thus, you went from non-square to square pixels. This is, I'm pretty sure, equivalent to a resize operation.

2. I tried Yadifx2 in VLC and the video suddenly looked horrible. It had artifacts that cold be described as stuttering, although the artifacts were mostly in the vertical direction.

The results from #2 makes me wonder if there is some sort of field order issue (TTF vs. BFF). To get into that I need to get onto my main editing computer which has all my tools and also has a dozen years of use which makes it much more reliable for finding subtle errors like this. I'll report back when I get a chance to get back to the office.

[edit]I did a quick check using my laptop, and with the tools I have available it does appear that the problem is field reversal on the original video.

[johnmeyer]

I decided to fire up the main computer in the office and use my time-tested tools. Initially everything looked OK, but I then found that I still had my field order detection script set for checking HD video (TFF). Thus it was set for top field first, but your video is flagged as bottom field first. Thus, I now get the same results on both computers.

So, that is indeed the problem: you have field reversal.

What this means is that if you put it through any script which expects BFF, based on the flag in the video, it will give you bad results. The same is true of any player that does its own deinterlacing, which explains the Yadifx2 results.

[edit]In most cases you can fix and/or avoid the problem by specifying top field first. In any AVISynth script, use AssumeTFF() as the first line after opening the video.

[jagabo]

Since the text is white with black outlines you can find light areas surrounded by dark areas and replace those with something other than QTGMC, in this example a simple blend deinterlace.

LWlibavVideoSource("DC18-testclip deint mute.avi") 
AssumeTFF()
ConvertToYV12(interlaced=true)

boxx = 74
boxy = 396
boxw = 256
boxh = 44

box = Crop(boxx, boxy, boxw, boxh).SelectEvery(1,0,0)

darks = box.Sharpen(1.0, 0.0).mt_binarize(60, upper=true).mt_expand().mt_expand().mt_expand().mt_expand().mt_inpand().mt_inpand().mt_inpand()
brights = box.Sharpen(1.0, 0.0).mt_binarize(175).mt_expand().mt_expand()
tmask = overlay(darks, brights, mode="multiply")
blend = box.Blur(0.0, 1.0).Sharpen(0.0, 0.6)

Overlay(QTGMC(), blend, x=boxx, y=boxy, mask=tmask)

Let me know if you want further explanation. The masking isn't prefect because the text isn't very bright and the outlines not very dark (and other bright/dark stuff passes through the area), but the artifacts aren't too visible at normal playback speed.

[jagabo]

Actually, just overlaying Yadif(mode=1, order=1) in the text area (QTGMC for the rest) looks pretty good. Especially if you feather the edges of the box.

LWlibavVideoSource("DC18-testclip deint mute.avi") 
AssumeTFF()

boxx = 74
boxy = 396
boxw = 256
boxh = 44

box = Crop(boxx, boxy, boxw, boxh).SelectEvery(1,0,0)
yd = Crop(boxx, boxy, boxw, boxh).Yadif(mode=1, order=1)
bmask = box.Crop(2,2,-2,-2).ColorYUV(off_y=256).AddBorders(2,2,2,2).Blur(1.0).Blur(1.0).Blur(1.0)

Overlay(QTGMC(), yd, x=boxx, y=boxy, mask=bmask)

[lordsmurf]

Actually, just overlaying Yadif(mode=1, order=1) in the text area (QTGMC for the rest) looks pretty good.

Will try this soon. Thanks.

your video is flagged as bottom field first
So, that is indeed the problem: you have field reversal.

In any AVISynth script, use AssumeTFF() as the first line after opening the video.

Lossless AVI / Huffyuv doesn't have TFF/BFF flags. There's nothing to detect. VLC just guesses field order wrong with Yadif2x. And since there is no setting to tweak it, nothing can be done. It's a bug that's existed for years.

Yes, AssumeTFF() should always be used. In theory, specifying TFF in a .avs, then passed to VLC, should work. But lack of .avs support is yet another limitation of VLC, and for many years now, after many requests to support it.

VLC is often praised as the best player, but it's undeserved. Yes, it does a great job for MP4/MKV, and Xvid before it, but that's really it. The MPEG and ISO support is nice, but hit-or-miss compared to others. The lossless AVI support is pretty craptastic. The software is frequently updated, but I really cannot tell any difference between new released, and a release from 5-10 years ago. Almost nothing changes that I can see.

You know what software works better with lossless AVI and MPEG? Mplayer, namely MPC-HC. But no deinterlacing ("automatic" that's often wrong), so still not best. (Maybe some command-line options exist, but those are poorly documented.)

But remember: lossless is an intermediary format, for editing/etc, playback was never the intention. Sadly, that means player support is lacking.

[johnmeyer]

Yes, I keep MPC-HC around for the things VLC can't handle. On my system VLC often can't keep up with 4K video (although I seldom have reason to watch such hi-res stuff on a computer screen), but MPC-HC sails through it.

There are very few software programs that don't have frustrating limitations.

[lordsmurf]

Let me know if you want further explanation.

The plain Yadif does look good.

The boxx/y/w/h somewhat confuses me.
How large is the box?
Any way to see that visually? Ideally with an overlay on the video, so I can verify it's covering the area needed? (The box size will change some, due to due length variations, and standard placement variation. I'll need to later do some tapes where the date is on the opposite side of the screen, different camcorder.)

This is most helpful.

I've never seen this error before ... but I'm also not recalling footage with an on-screen date, when I was doing 59.94.

[jagabo]

Let me know if you want further explanation.

The boxx/y/w/h somewhat confuses me.
How large is the box?
Any way to see that visually?

The box is at 74,396 and is 256x44 pixels. One way to easily visualize the box is to Invert() the overlay. Change the Overlay line to:

Overlay(QTGMC(), yd.Invert(), x=boxx, y=boxy, mask=bmask)

Or, since the mask is the same size as the overlay, you can Overlay the mask instead of the yd image:

Overlay(QTGMC(), bmask, x=boxx, y=boxy)

The box is a little larger than it really needs to be because of the earlier code where I was detecting the text with mt_expand() and mt_inpand(). You don't want the expanded intermediate to hit the edge of the frame. And also because I was cropping 2 pixels around the edge of the mask and adding it back with black to feather it.

[oln]

In my experience QTGMC can sometimes err on the side of caution on date/time stuff if there is a lot of motion behind it and not smooth it out the jitter on it as it otherwise would, using a slower setting in QTGMC can sometimes help improve it a bit. I guess the more primitive yadif algorithm handles this specific thing a little better, looks like a useful solution to use that in this specific area of the image.

[lordsmurf]

What kind of NR can be applied for the Yadif? Because when the date turns off, there is a fuzzier blotch that reminds me of a delogo video. Ironically, can qtgmc be run here?

[jagabo]

What kind of NR can be applied for the Yadif?

Any NR you want.

Ironically, can qtgmc be run here?

QTGMC(InputType=1or2)

[lordsmurf]

I was looking at dfttest(), but not liking it, not sure how to tweak it stronger, more like the QTGMC usage. It needs some AA as well, but sangnom2 is ghastly, xaa won't run ("no function TryFTurnRight"). Anyway ... doesn't matter anymore.

I completely forgot about the QTGMC option. That looks perfect.