Duplicates Frames. Cause and means?

[manono]

I wonder why there is no filter for the "easy-to-use" VirtualDub
for the task of adaptive decimation ...
but one has to dip deep into something like avisynth scripts,
packing into AVI container, enabling VfW codec, ...
Big hurdle.

If you're serious about decimating duplicate frames, you use AviSynth. Period. Don't bother beating your head against a wall trying to do it through VDub or VDub2.

[nji9nji9]

I wonder why there is no filter for the "easy-to-use" VirtualDub
for the task of adaptive decimation ...
but one has to dip deep into something like avisynth scripts,
packing into AVI container, enabling VfW codec, ...
Big hurdle.

If you're serious about decimating duplicate frames, you use AviSynth. Period. Don't bother beating your head against a wall trying to do it through VDub or VDub2.

If one thing is for sure:

I won't "bother beating my head against a wall" called "manono from avisynth stone age".

Period.

[jagabo]

Good luck finding another tool as flexible and powerful as AviSynth.

[nji9nji9]

Good luck finding another tool as flexible and powerful as AviSynth.

Thank you.

Not to be misunderstood:
I do not doubt about the flexibility and power of AviSynth.
But - as I already stated - it is so complicate to handle (especially the scripting) that this will prevent most users to (be able) to use it.

Actually I wrote a CLI application (C++/opencv) that makes a thorougly analysis of the movie and lets the user decide between specificity and sensitivity.
The exported numbers of the frames to be removed have to taken by a VD-plugin that actually removes them (before any further processing).
Will announce as soon as in release state.

[poisondeathray]

Actually I wrote a CLI application (C++/opencv) that makes a thorougly analysis of the movie and lets the user decide between specificity and sensitivity.
The exported numbers of the frames to be removed have to taken by a VD-plugin that actually removes them (before any further processing).
Will announce as soon as in release state.

Wouldn't a CLI application be difficult to handle for most users ?

Can you preview your results and adjust the settings? Or do you have to run it, preview, adjust, run it again etc..

What about AV sync ? If you delete frames, does your code keep everything in sync adjusting timecodes and framerate ?

[nji9nji9]

Actually I wrote a CLI application (C++/opencv) that makes a thorougly analysis of the movie and lets the user decide between specificity and sensitivity.
The exported numbers of the frames to be removed have to taken by a VD-plugin that actually removes them (before any further processing).
Will announce as soon as in release state.

Wouldn't a CLI application be difficult to handle for most users ?

I focussed on doing it as simple/ comfortable as possible.

But - as alway - you did a point.
It would be best to do it into the VD filter.
But as I'm not (yet) familiar with sdk of VDF and Win32 ... I did it this way.
As what I'm actually am about: I want to use it. Now.

Can you preview your results and adjust the settings? Or do you have to run it, preview, adjust, run it again etc..

What about AV sync ? If you delete frames, does your code keep everything in sync adjusting timecodes and framerate ?

No preview (CLI you know )
You make decisions about some values, get resulting distribution of dupes, cadences, their coverage ... AND resulting desync.
You may let optimize for some values (i.e. desync), or try another set of values manually.
And again.

[johnmeyer]

I wonder why there is no filter for the "easy-to-use" VirtualDub
for the task of adaptive decimation ...
but one has to dip deep into something like avisynth scripts,
packing into AVI container, enabling VfW codec, ...
Big hurdle.

To echo what the two senior posters have said, AVISynth is a very unique tool that can do things that you simply cannot do in VirtualDub.

Why?

1. Because most editing programs do not give the user the ability to look at multiple frames.

2. Because most editing programs do not provide programmatic logic that lets you perform an action only if some criteria is met. This becomes exceedingly important when dealing with complex telecine that doesn't follow a set pattern. The program must adapt.

3. Because most editing programs do not let you have multiple filters interact with each other. In VirtualDub, you can load an inverse telecine filter, but can you have load another filter which will only take action on a field which follows the decimation of a field? There are millions of examples of this, but here is one of the best where I was able to remove the moving noise bars from a home-made "kinescope" of a 1950s football game. This required looking at every scan line and comparing it to adjacent frames and scan lines.

https://forum.doom9.org/showthread.php?t=167591

and here's where I asked for help to develop the AVISynth script that created this magic:

Bad 1950s Kinescope - Hopeless?

Bottom line: AVISynth is a very unique tool and many video improvements and fixes cannot be done as well -- or at all -- with any other software.

[poisondeathray]

Bottom line: AVISynth is a very unique tool and many video improvements and fixes cannot be done as well -- or at all -- with any other software.

And keep sync !

It's absolutely worth learning avisynth; a very useful tool in many situations

[nji9nji9]

Yes, only keep on all of you ...

2 more posts and I am convinced and will turn to avisynth.
(Just kidding - only having a look at the 1950's Kinescope script gives me vertigo )

To echo myself:
I do not doubt the abilities and power of avisynth.
But it's a BIG hurdle.
And this hurdle will keep the number of users small, and smaller, and smaller ...

But I have to admit:
If there is a more complex situation (as filters depending from each other)
VD can't do that.
Also - as far as I know - to act a filter only if a citerium is met.
(Nice suggestion for an improvement of VD2 )
Also - as someone who must know said to me - it is said there are tons of VD filters,
but the one you need (sometimes) doesn't exist yet.

Yet I also have to contradict:
Donald Graft's old VD decimate filter IS adaptive.
It needs avisynth, but only as to the time it was written VD wasn't able to delete frames.

I'm really sorry if I offended anybody in this thread.
I only wanted to understand what's technical going on
to adequate identify the cause.

... and I will not turn to this cryptic avisynth ,
but write my own VD filter.
In standard C++.

[jagabo]

I do not doubt about the flexibility and power of AviSynth.
But - as I already stated - it is so complicate to handle (especially the scripting) that this will prevent most users to (be able) to use it.

Actually I wrote a CLI application (C++/opencv) that makes a thorougly analysis of the movie and lets the user decide between specificity and sensitivity.
The exported numbers of the frames to be removed have to taken by a VD-plugin that actually removes them (before any further processing).

So to avoid writing a two line script

LWLibavVideoSource("Der Kommissar 28 Drei Tote reisen nach Wien - HD-L4kQdOWojeQ.mkv") 
TDecimate(Cycle=6, CycleR=1)

You wrote a whole program to detect dups. And you still need to figure out how to delete the dups...

[nji9nji9]

I do not doubt about the flexibility and power of AviSynth.
But - as I already stated - it is so complicate to handle (especially the scripting) that this will prevent most users to (be able) to use it.

Actually I wrote a CLI application (C++/opencv) that makes a thorougly analysis of the movie and lets the user decide between specificity and sensitivity.
The exported numbers of the frames to be removed have to taken by a VD-plugin that actually removes them (before any further processing).

So to avoid writing a two line script

LWLibavVideoSource("Der Kommissar 28 Drei Tote reisen nach Wien - HD-L4kQdOWojeQ.mkv") 
TDecimate(Cycle=6, CycleR=1)

You wrote a whole program to detect dups. And you still need to figure out how to delete the dups...

You haven't read the last paragraph of my post, right?
https://forum.videohelp.com/threads/394366-Duplicates-Frames-Cause-and-means/page2#post2562347

[johnmeyer]

This is a pointless thread.

1. The guy won't post his video.

2. The guy claims he can code C++ but then claims that AVISynth is too complicated. As everyone else in this thread knows, that statement makes zero sense.

3. Finally, he keeps posting, but doesn't appear to have actually done anything.

This is a pointless exercise. As I said almost a month ago, what is the point of continuing.

[nji9nji9]

This is a pointless thread.

1. The guy won't post his video.

2. The guy claims he can code C++ but then claims that AVISynth is too complicated. As everyone else in this thread knows, that statement makes zero sense.

3. Finally, he keeps posting, but doesn't appear to have actually done anything.

This is a pointless exercise. As I said almost a month ago, what is the point of continuing.

Not to forget:

4. Recently I took the wrong side's bicyle path.

[nji9nji9]

There is a strange observation I made during the dupe removal:

Saving an arbitrary movie (with no "artificial dupes") as avc1.

1. Without change or
2. With inserted dupes every 2nd (VirtualDub filter "interpolate") or
3. With inserted interpolated frames every 2nd (same filter).

I would have expected that the compression rate of the codec
would be highest at 2., then 3. and then 1.
While the file size would be smallest on 1., then 2. and then 3.

But in my examples the file size with respect to 1.
only increases (about 1%) when adding interpolated frames,
but always decreases (about 4%) when inserting real dupes.

Why is that?

[jagabo]

x264 (x265 too?) is frame rate aware. It uses fewer bits per frame at higher bitrates. The assumption being that you will see defects less when frames are displayed for shorter periods of time. This makes it difficult to predict what happens with the overall file size when increasing frame rate.

[nji9nji9]

x264 (x265 too?) is frame rate aware. It uses fewer bits per frame at higher bitrates. The assumption being that you will see defects less when frames are displayed for shorter periods of time. This makes it difficult to predict what happens with the overall file size when increasing frame rate.

Thank you!

In my mind there was that statement that crf-mode is best for archiving.
And so I (falsely) concluded that crf was constant quality.
But it is not.
It is constant perceived quality.

OK.
Still I would have expected that that frame rate awareness lets a doubled fps-movie increase by only ... say 1.5.
Never would have expected even a decreasing.
Still strange, isn't it?

P.S.
x265: I once checked, as compression was rather good.
But having a closer look at the frames (even with small movements) ... I turned away shuddering.
Fake quality

[jagabo]

In my mind there was that statement that crf-mode is best for archiving.
And so I (falsely) concluded that crf was constant quality.
But it is not.
It is constant perceived quality.

Yes, that's the point of CRF mode. As opposed to QP (constant quantizer) mode which is mathematically constant quality.

Still I would have expected that that frame rate awareness lets a doubled fps-movie increase by only ... say 1.5.

It depends on the content.

Never would have expected even a decreasing.
Still strange, isn't it?

No. You are dealing with something of a degenerate case -- every other frame is a duplicate.

P.S.
x265: I once checked, as compression was rather good.
But having a closer look at the frames (even with small movements) ... I turned away shuddering.
Fake quality

How long ago was that? x265 has improved over the years. Even x264 was very bad in its first few years.

[nji9nji9]

I better do not quote again, as it's getting confusing then.

My x265 tests have been a year ago.
But I have to admit that I have a mathematical/ quality look
The perceived quality might be ... ok.
On the other hand ... there are these neuro studies showing a "brain jogging"
when consuming these "new" kind of compressed formats (audio too).

Frame rate doubling by dupe frames
... a degenerated case, I agree.
On the other hand I noticed that some of our public tv stations offer their material
only in 50 fps. But having a closer look, it's only dupes every second frame (= 25 fps).
Why do they do that?

[jagabo]

On the other hand I noticed that some of our public tv stations offer their material
only in 50 fps. But having a closer look, it's only dupes every second frame (= 25 fps).
Why do they do that?

Because they only broadcast at 50p.

[nji9nji9]

On the other hand I noticed that some of our public tv stations offer their material
only in 50 fps. But having a closer look, it's only dupes every second frame (= 25 fps).
Why do they do that?

Because they only broadcast at 50p.

No.
I don't mean broadcast but also their download service.

Maybe it's because of the seemingly reduction of file size I do observe right now?
Or it's because the visual look of a duped movie is better?
BTW
A low fps movie seems to have a better look if I insert interpolated frames, right or illusion?

[jagabo]

Maybe it's because of the seemingly reduction of file size I do observe right now?

I doubt it. They most likely use hardware encoders rather than x264.

Or it's because the visual look of a duped movie is better?

If you're watching on a 50p TV 25p usually looks the same as 50p with dups. In the former case the TV performs the dups.

A low fps movie seems to have a better look if I insert interpolated frames, right or illusion?

It's a matter of personal preference.

[nji9nji9]

Thank's again for your support.

I do ask these questions because with this new observations
I got uncertain if it's a good idea at all to remove dupe frames.