Nero Video Grabber vs Hauppauge WinTV-USBlive2 vs August VGB300

[Alwyn]

While it is relative simple and reliable to discover duplicates by means of a script

Do you have a script for spotting dupes I could try?

[Sharc]

While it is relative simple and reliable to discover duplicates by means of a script

Do you have a script for spotting dupes I could try?

The DeDup avisynth plugin comes to my mind:
http://avisynth.nl/index.php/DeDup

The first pass 'DupMC' will create a log.
A very low difference (~0%) between 2 frames indicate a duplicate. The log can be evaluated in Excel for example, or evaluated and processed in pass2 ('DeDup') for removing the duplicates.

[jagabo]

AviSource("filename.avi"
WriteFileIf("Dups.txt", "YDifferenceFromPrevious<0.5", "current_frame", flush=true)

Open the AVS file in VirtualDub then use File -> Run Video Analysis Pass. When it's done you will have a file called Dups.txt that has a list of the duplicate frames. You can change the threshold to reduce false positives or false negatives. Also note that it only compares the Y channel (not U and V).

[Alwyn]

Thanks guys, I finally got DupMC working after seeing Jagabo's comment re running the video analysis pass. That isn't mentioned in the wiki or readme that I can see.

@Jagabo, I'm still experimenting with your script. I haven't worked out how the "difference" parameter affects things yet.

[jagabo]

YDifferenceFromPrevious is the average difference of the Y value of each pixel of the current frame and the previous frame. If the current frame and the previous frame are identical (as far as the Y values are concerned) the calculated value will be zero. It will be non-zero if any of the Y values is different.

Note that compression artifacts (when using lossy compression) or other noise may cause originally identical frames to no longer be identical. That's why a non-zero value is used in the sample code. It is no longer possible to detect for certain which frames were identical before the lossy compression.

[Alwyn]

Thanks, I'm working with analogue AVI captures.

I've set the "Diff from Prev" to 0.0 but am getting just a list of numbers (I assume the frame numbers, looks like very frame) with no other details, as per the attached.

The DupMC report does show the % difference between each frame.

[jagabo]

I've set the "Diff from Prev" to 0.0 but am getting just a list of numbers

Yes, that's all it shows, the frame number of frames that are duplicates of the frame before them. Oh, it also always shows frame zero as a duplicate, a false positive (as there's nothing to compare that frame to). It shouldn't show every frame is a duplicate.

[Sharc]

Thanks, I'm working with analogue AVI captures.

I've set the "Diff from Prev" to 0.0 but am getting just a list of numbers (I assume the frame numbers, looks like very frame) with no other details, as per the attached.....

The clip must be in planar format, so

AviSource("your.avi")
converttoYV16(interlaced=true) #convert to planar
WriteFileIf("Dups.txt", "(YDifferenceFromPrevious)<0.5", "current_frame",flush=true)

If you want to see the % as well:

AviSource("your.avi")
converttoYV16(interlaced=true) #convert to planar
WriteFileIf("Dups.txt", "(YDifferenceFromPrevious)<0.5", "current_frame",  """ ":" """, "YDifferenceFromPrevious",flush=true)

[Alwyn]

Excellent, thanks Sharc and Jagabo. That works.

[jagabo]

You can include the chroma channels using UtoY() and VtoY(). For YV16:

StackHorizontal(last, UtoY(), VtoY())

before WriteFileIf().

For YV12:

StackHorizontal(last, StackVertical(UtoY(), VtoY()))

[rgr]

While it is relative simple and reliable to discover duplicates by means of a script

Do you have a script for spotting dupes I could try?

The DeDup avisynth plugin comes to my mind:
http://avisynth.nl/index.php/DeDup

The first pass 'DupMC' will create a log.
A very low difference (~0%) between 2 frames indicate a duplicate. The log can be evaluated in Excel for example, or evaluated and processed in pass2 ('DeDup') for removing the duplicates.

A very unreliable script, like all the others. With compressed material and fast scenes, it can produce a high value for duplicate frames, while in still scenes, it can produce a low value for non-duplicate frames.

In fact, you should compare it with neighboring values, i.e.:
- 40, 3, 50 - in subsequent frames indicate a duplicate
- 3, 2, 3 - indicate a non-duplicate
- 3, 0.5, 3 - indicate a duplicate

Dfttest with high sigma (up to 64) increases the number of correct detections but eliminates easy recognition of perfect dups.

RT_FrameMovement (RT_Stats) is better than DupMC.

However, duplicates in lossless capturing should not differ at all (YDiff=0).

[rgr]

If you send me this file, I can check it with my program based on the above assumptions.

[rgr]

You can include the chroma channels using UtoY() and VtoY(). For YV16:

StackHorizontal(last, UtoY(), VtoY())

before WriteFileIf().

For YV12:

StackHorizontal(last, StackVertical(UtoY(), VtoY()))

Or use RT_FrameDifference (RT_Stats).

[Sharc]

A very unreliable script, like all the others....

It is not an "unreliable script". DupMC does exactly and reliably what it claims to be doing. It's up to the user how to interpret the data which it delivers.

[Alwyn]

DupMC is verifying that the AmarecTV report is correct WRT dupes (Sharc, I'm using = as the delimiter ).

And yes, I finally got AmarecTV to report two actual drops (not dupes). Never seen drops before in an Amarec capture report. And they were in the recording, at the frame count the report said. Were there other drops (without dupes)? I don't know, but with the audio in sync after 3 hours, I'm happy.

[rgr]

A very unreliable script, like all the others....

It is not an "unreliable script". DupMC does exactly and reliably what it claims to be doing. It's up to the user how to interpret the data which it delivers.

If it's supposed to show the difference between frames, then it can be considered effective.
If it's supposed to detect duplicates, then it performs average to good.

[rgr]

DupMC is verifying that the AmarecTV report is correct WRT dupes (Sharc, I'm using = as the delimiter ).

And yes, I finally got AmarecTV to report two actual drops (not dupes). Never seen drops before in an Amarec capture report. And they were in the recording, at the frame count the report said. Were there other drops (without dupes)? I don't know, but with the audio in sync after 3 hours, I'm happy.

Have you checked the frame counter between VDub and AmarecTV captures? For a 3-hour movie, AmarecTV should drop about 60 frames, or ~1 second.

[Alwyn]

For a 3-hour movie, AmarecTV should drop about 60 frames, or ~1 second.

?? 60 frames, even at 29.97, is 2 seconds. That would obviously be immediately noticeable if the audio wasn't corrected during capture. Since I've never had that sort of audio error, are you saying that Amarec adjusts the audio to compensate for the alleged masses of dropped frames?

[rgr]

Of course.
https://forum.videohelp.com/attachments/87807-1752419717/rating.png

Sync means nothing.

[Alwyn]

That's just your table, nothing particularly credible about, based on what I see.

What do you mean, "sync means nothing"? Are you suggesting that, throughout the capture, the audio just randomly drifts in and out of sync with all those 60 dropped frames but miraculously at the end it's perfectly in-sync?

If Amarec was dropping 20 frames in an hour without replacing them, the out-of-sync would be blindingly obvious.

[Sharc]

What do you mean, "sync means nothing"? Are you suggesting that, throughout the capture, the audio just randomly drifts in and out of sync with all those 60 dropped frames but miraculously at the end it's perfectly in-sync?

Hmm, what do these ~1s fluctuating fps in Amarec's status bar indicate? The variation of the duty cycle of ....-slow-fast-slow-fast-.... could be a strategy to keep A/V in sync. Just speculating.

[rgr]

That's just your table, nothing particularly credible about, based on what I see.

What do you mean, "sync means nothing"? Are you suggesting that, throughout the capture, the audio just randomly drifts in and out of sync with all those 60 dropped frames but miraculously at the end it's perfectly in-sync?

If Amarec was dropping 20 frames in an hour without replacing them, the out-of-sync would be blindingly obvious.

Apparently, it also leaves out parts of the audio. Or resamples audio (VDub can do this).
The Chinese grabber adds 66 frames, yet the audio is synchronized.

But you didn't answer the question. You haven't actually tested anything...

[Alwyn]

Latest test.

Workflow: HR-S5700AM>S Video>Pioneer 645H>S-Video>DVK-200>S-Video>IOData GV-USB2>Lagarith

Using VDub 2: 7 Duplicate frames, total frame count 86224

Using AmarecTV: 1 Duplicate frame, total frame count 86219.

I don't have the ability to detect dropped frames and there was no audio accurate enough to check the sync.