Old silent movie with weird duplicate frame pattern.

[dopethrone]

I want to remove the duplicate frames from the video (the movie is Limite (1931) and keep just the good frames, the source is 29.97fps, i noticed that it has a really weird pattern of duplicate frames and i wanted to restore the original framerate, on the brazilian cinemateca website, it says it's 16fps (https://bases.cinemateca.org.br/cgi-bin/wxis.exe/iah/?IsisScript=iah/iah.xis&base=FILM...detailed.pft#1) but i'm not sure if this can be trusted regarding the criterion restoration.
I think this could be achieved using a avisynth script, i tried using the tdecimate function using a first pass to generate a timecode file and then making a vfr encode, but it errors out saying something like "no direct access" or something similar, does anyone know how to achieve this? i'm afraid of trying something and ending up removing good frames. Also, i don't want to break sync with the audio.
From what i counted the dupes go like this: 11223344556678899AABBCCDDEEFGGHHIIJJKKLLMMNOOPPQQR RSSTTUVVWWXXYYZZ1122344
6 dupes, 1 ok frame afterwards then 7 dupes, 7, 6, 7
I got instructed that it would be better to just tfm() tdecimate() but i'm afraid this would end up removing good frames, so i'm not sure if this is valid.
Also, if i end up using a avisynth script to remove frames and encoding it with mulder's simple x264 launcher, if i take the original audio file from the 29.97 bluray source and mux it with mkvtoolnix, will it break sync or have problems? Thanks.

sample: https://mega.nz/file/cpxmEb6R#60OUsNHNkcUixk3nsChDvXYXJRzFwcgtlK7EDLD2BNo

[jagabo]

The film frame rate appears to be either 16 or 15.984 (16000/1001) fps. This works pretty well, leaving 15.984 fps:

LWLibavVideoSource("cut.mkv") 
TFM()
TDecimate(Cycle=15, CycleR=7)

Note, I didn't analyze the modern opening titles, only the film portion.

[lordsmurf]

Lots of old film was hand crank shot, so actual frame rate can vary. This is why films appear too fast (usually), or too slow (rarely). When forced to look "normal", you get weird frame rates. And those odd rates don't translate well to standard rates. So odd dupes, blends, etc.

But there's also other reasons you can see oddities.

[dopethrone]

The film frame rate appears to be either 16 or 15.984 (16000/1001) fps. This works pretty well, leaving 15.984 fps:

LWLibavVideoSource("cut.mkv") 
TFM()
TDecimate(Cycle=15, CycleR=7)

Note, I didn't analyze the modern opening titles, only the film portion.

Thanks Jagabo! Can the code you wrote have the chance of removing good frames? I imagine just analyzing frame by frame in some way right? though i don't believe the dupe pattern changes in the rest of the movie..
Is there another way where i can preserve the FPS of the modern titles and have the original frames of the film like a VFR manner?

[dopethrone]

Lots of old film was hand crank shot, so actual frame rate can vary. This is why films appear too fast (usually), or too slow (rarely). When forced to look "normal", you get weird frame rates. And those odd rates don't translate well to standard rates. So odd dupes, blends, etc.

But there's also other reasons you can see oddities.

In a documentary where the director talks about this movie, he says what cameras he used for the movie and he mentions a hand cranked one if i recall correctly.

[jagabo]

Can the code you wrote have the chance of removing good frames?

Yes. If I chose too low a frame rate some unique frames would be removed. Or if there are big variations in the distribution of the duplicate frames one will lose some unique frames and some duplicates will be left.

I imagine just analyzing frame by frame in some way right?

Basically, yes.

I first analyzed a 1500 frame sequence using DeDup(). From that I determined there were 802 unique frames in that 1500 frame sequence. That indicates a frame rate of about 16.024 fps. 800 would have indicated exactly 16000/1001 fps (15.984). I decided that was close enough. Getting that last one or two frames is nearly impossible to do accurately. You can't really use TDecimate(Cycle=1500, CycleR=698) -- it doesn't work with such large groups of frames.

Is there another way where i can preserve the FPS of the modern titles and have the original frames of the film like a VFR manner?

Yes. You can use DeDup to reduce the clip to all unique frames (well, some of the longer shots in the titles will still have some duplicates) and export the time codes of each of those frames. You encode the remaining frames and mux the time codes with the result.

http://avisynth.nl/index.php/DeDup

The result probably still won't be perfect because of compression artifacts in your source (ie, what should have been identical frames may be different because of compression artifacts) and other possible problems.

[dopethrone]

Can the code you wrote have the chance of removing good frames?

Yes. If I chose too low a frame rate some unique frames would be removed. Or if there are big variations in the distribution of the duplicate frames one will lose some unique frames and some duplicates will be left.

I imagine just analyzing frame by frame in some way right?

Basically, yes.

I first analyzed a 1500 frame sequence using DeDup(). From that I determined there were 802 unique frames in that 1500 frame sequence. That indicates a frame rate of about 16.024 fps. 800 would have indicated exactly 16000/1001 fps (15.984). I decided that was close enough. Getting that last one or two frames is nearly impossible to do accurately. You can't really use TDecimate(Cycle=1500, CycleR=698) -- it doesn't work with such large groups of frames.

Is there another way where i can preserve the FPS of the modern titles and have the original frames of the film like a VFR manner?

Yes. You can use DeDup to reduce the clip to all unique frames (well, some of the longer shots in the titles will still have some duplicates) and export the time codes of each of those frames. You encode the remaining frames and mux the time codes with the result.

http://avisynth.nl/index.php/DeDup

The result probably still won't be perfect because of compression artifacts in your source (ie, what should have been identical frames may be different because of compression artifacts) and other possible problems.

Could i get some help on what parameters to tweak in DeDup/a brief guidance?

[dopethrone]

Also, with the timecodes file generated by DeDup, when i mux this in mkvtoolnix, should i put the timecode just for the video or both audio and video? Or if i just use it in the video and drop the audio file to mux it without attaching the timecode it will be ok

[dopethrone]

Alright so i tried a test encode with DeDup:

DeDup(threshold=0.3, maxcopies=10, maxdrops=3, log="D:\blah.dup.txt", times="D:\blah.times.txt")

What i got was: around 22~ frames frames in the film and 12~ fps in the modern titles, that look to have 23.976/24fps, any ideas?

[jagabo]

Could i get some help on what parameters to tweak in DeDup/a brief guidance?

DeDup is a 2-pass filter. First you use DupMC() to build a stats file. Then you use DeDup() to do the actual duplicate removal. I usually put the two command in the same AVS script and comment out the one I'm not using. Something like:

LWLibavVideoSource("cut.mkv", cache=false, prefer_hw=2) 
Crop(240,0,-240,-0)
TFM(cthresh=4)
DupMC(log="duplog.txt")
#DeDup(threshold=1.10, maxcopies=12, maxdrops=12, show=false, log="duplog.txt", times="timecodes.txt")
AssumeFPS(16)

First I cropped away the pillarbox bars and used TFM() to eliminate combing. The lower than default cthresh was to make sure the slow fade ins/outs didn't have any residual combing.

The first pass must process every frame of the video so the script must be fully "executed". I usually use VirtualDub to view my scripts so I use its File -> Run Video Analysis Pass to accomplish this. VirtualDub will sequentially read every frame of the video as fast as it can; DupMC() will write a diff value for each frame to the log file. The frames don't necessarily have to be exactly the same as the frames you will later DeDup(). For example, if there is some video noise at the edges of the frame you may want to crop the edges away so that noise doesn't interfere with duplicate detection later.

Here's a small portion taken from a log file:

frm 11310: diff from frm 11311 = 4.2803% at (672,512)
frm 11311: diff from frm 11312 = 0.0769% at (1408,704)
frm 11312: diff from frm 11313 = 5.0698% at (704,512)
frm 11313: diff from frm 11314 = 0.2930% at (864,544)
frm 11314: diff from frm 11315 = 4.7469% at (768,544)
frm 11315: diff from frm 11316 = 0.1970% at (608,640)
frm 11316: diff from frm 11317 = 6.5284% at (672,512)
frm 11317: diff from frm 11318 = 0.2053% at (736,32)
frm 11318: diff from frm 11319 = 5.8610% at (1024,608)
frm 11319: diff from frm 11320 = 6.4328% at (544,384)

In that sample you can see that some frames have differences around 5 percent, others less than 1 percent. The larger values are the unique frames, the smaller values the duplicate frames (not exactly the same because of compression artifacts).

The second pass is the same except for the call to DeDup() rather than DupMC():

LWLibavVideoSource("cut.mkv", cache=false, prefer_hw=2) 
Crop(240,0,-240,-0)
TFM(cthresh=4)
#DupMC(log="duplog.txt")
DeDup(threshold=1.10, maxcopies=12, maxdrops=12, show=false, log="duplog.txt", times="timecodes.txt")
AssumeFPS(16)

The main variable you need to be concerned about is the threshold. The higher you set it the more frames are discarded. If it's too high unique frames will be discarded. The lower you set it the more frames will be passed through. But some will just have compression artifacts, not really unique frames. I picked 1.10 here because there were a few fade in/out frames (in the titles) with values around 1 percent.

MaxCopies is the maximum number of sequential frames that can be replaced by a single frame. For the body of the movie you could use 1 (I don't there's ever more than one duplicate in a row). But to eliminate more frames during the opening credits you'll want a higher value. I arbitrarily chose 12 (also the default). See the docs for the interaction between this and MaxDrops.

The show option prints stats on the video -- helpful for debugging.

You can use MkvToolNix to mux the timecodes along with the video and audio stream. You only need them for the video. Highlight the video stream and select the "timestamp file" option to load the timestamps.

Attached is the result from the above script at 1/4 frame size (to reduce the upload size), muxed with the original audio.

You can get slightly smoother results using a bob deinterlace to 60p, then applying DeDup() to that.


A bigger issue is why you want to remove duplicates? They don't cost much bitrate when encoding. One good reason is if you plan on applying a temporal noise reduction filter -- duplicates poison the analysis of that.

[dopethrone]

Could i get some help on what parameters to tweak in DeDup/a brief guidance?

DeDup is a 2-pass filter. First you use DupMC() to build a stats file. Then you use DeDup() to do the actual duplicate removal. I usually put the two command in the same AVS script and comment out the one I'm not using. Something like:

LWLibavVideoSource("cut.mkv", cache=false, prefer_hw=2) 
Crop(240,0,-240,-0)
TFM(cthresh=4)
DupMC(log="duplog.txt")
#DeDup(threshold=1.10, maxcopies=12, maxdrops=12, show=false, log="duplog.txt", times="timecodes.txt")
AssumeFPS(16)

First I cropped away the pillarbox bars and used TFM() to eliminate combing. The lower than default cthresh was to make sure the slow fade ins/outs didn't have any residual combing.

The first pass must process every frame of the video so the script must be fully "executed". I usually use VirtualDub to view my scripts so I use its File -> Run Video Analysis Pass to accomplish this. VirtualDub will sequentially read every frame of the video as fast as it can; DupMC() will write a diff value for each frame to the log file. The frames don't necessarily have to be exactly the same as the frames you will later DeDup(). For example, if there is some video noise at the edges of the frame you may want to crop the edges away so that noise doesn't interfere with duplicate detection later.

Here's a small portion taken from a log file:

frm 11310: diff from frm 11311 = 4.2803% at (672,512)
frm 11311: diff from frm 11312 = 0.0769% at (1408,704)
frm 11312: diff from frm 11313 = 5.0698% at (704,512)
frm 11313: diff from frm 11314 = 0.2930% at (864,544)
frm 11314: diff from frm 11315 = 4.7469% at (768,544)
frm 11315: diff from frm 11316 = 0.1970% at (608,640)
frm 11316: diff from frm 11317 = 6.5284% at (672,512)
frm 11317: diff from frm 11318 = 0.2053% at (736,32)
frm 11318: diff from frm 11319 = 5.8610% at (1024,608)
frm 11319: diff from frm 11320 = 6.4328% at (544,384)

In that sample you can see that some frames have differences around 5 percent, others less than 1 percent. The larger values are the unique frames, the smaller values the duplicate frames (not exactly the same because of compression artifacts).

The second pass is the same except for the call to DeDup() rather than DupMC():

LWLibavVideoSource("cut.mkv", cache=false, prefer_hw=2) 
Crop(240,0,-240,-0)
TFM(cthresh=4)
#DupMC(log="duplog.txt")
DeDup(threshold=1.10, maxcopies=12, maxdrops=12, show=false, log="duplog.txt", times="timecodes.txt")
AssumeFPS(16)

The main variable you need to be concerned about is the threshold. The higher you set it the more frames are discarded. If it's too high unique frames will be discarded. The lower you set it the more frames will be passed through. But some will just have compression artifacts, not really unique frames. I picked 1.10 here because there were a few fade in/out frames (in the titles) with values around 1 percent.

MaxCopies is the maximum number of sequential frames that can be replaced by a single frame. For the body of the movie you could use 1 (I don't there's ever more than one duplicate in a row). But to eliminate more frames during the opening credits you'll want a higher value. I arbitrarily chose 12 (also the default). See the docs for the interaction between this and MaxDrops.

The show option prints stats on the video -- helpful for debugging.

You can use MkvToolNix to mux the timecodes along with the video and audio stream. You only need them for the video. Highlight the video stream and select the "timestamp file" option to load the timestamps.

Attached is the result from the above script at 1/4 frame size (to reduce the upload size), muxed with the original audio.

You can get slightly smoother results using a bob deinterlace to 60p, then applying DeDup() to that.


A bigger issue is why you want to remove duplicates? They don't cost much bitrate when encoding. One good reason is if you plan on applying a temporal noise reduction filter -- duplicates poison the analysis of that.

I see, what's the reason for the AssumeFps() at the end of the script?

[jagabo]

I see, what's the reason for the AssumeFps() at the end of the script?

Oh, I forgot to mention that. I encoded with x264 which is frame rate aware. At slow frame rates is uses higher quality, at higher frame rates it uses lower quality -- on the assumption that frames that are displayed for a longer period of time give you more time to notice defects. DeDup doesn't change AviSynth's frame rate flag so the output appears to be the inputs files 29.97 fps. Since know the frame rate of the film portions of the clip is really about 16 fps I decided to use that.

[dopethrone]

Say, i wanted to use a simple TFM().TDecimate(), would this end up removing good frames?

[dopethrone]

The film frame rate appears to be either 16 or 15.984 (16000/1001) fps. This works pretty well, leaving 15.984 fps:

LWLibavVideoSource("cut.mkv") 
TFM()
TDecimate(Cycle=15, CycleR=7)

Note, I didn't analyze the modern opening titles, only the film portion.

I figured it would be a interesting idea to tweak TDecimate values to get a FPS close enough to 16.024, i tried with TDecimate(Cycle=29, CycleR=13) and got 16.535, even though its quite higher it would avoid losing any good frames i assume, does this make sense? Though how do i come up with the values other than trying them out, like a formula?

[jagabo]

I already answered that question.

[dopethrone]

I apologize for the confusion.

[dopethrone]

Say, i wanted to use a simple TFM().TDecimate(), would this end up removing good frames?

Though i'm not sure about this question Jagabo.. Anyways thanks for all the help.

[manono]

I do it differently although I came up with the same as jagabo's first post. First, you provided way too much useless video. The text screens and more-or-less static scenes are useless. This one is absurdly easy for a silent film.
First put on TFM to make it progressive again. Then find a place with movement. Then start counting. The idea is to find the "grand cycle". Within that cycle are "mini-cycles". I did it in several places and the result was always the same. Plus, this is Criterion, and they know what they're doing.
Here's the result:
2 2 2 2 2 2 3
2 2 2 2 2 2 3
Each number represents a "mini-cycle" where the total (in each line) is the grand cycle. The 2 represents a mini-cycle of 2 frames with one being a duplicate. the 3 represents a mini-cycle of 3 frames with one being a duplicate. The grand cycle is the total number of frames before the cycle repeats again. So, within a cycle of 15 frames are 7 duplicate frames. This is represented as:

TFM()
TDecimate(Cycle=15, CycleR=7)

or 15.984fps, as jagabo stated in his first response. It took all of about 5 minutes to discover and then confirm. Assuming the pattern is the same throughout the film (a pretty safe bet, but you can confirm it yourself), you can be pretty sure of getting back unique frames without dupes or missing frames..

[dopethrone]

I see, thanks for the response manono. I'm do not have much experience using avisynth so i appreciate all the help.

[jagabo]

Say, i wanted to use a simple TFM().TDecimate(), would this end up removing good frames?

Though i'm not sure about this question Jagabo.. Anyways thanks for all the help.

First of all, when dealing with lossy codecs it's always possible that TDecimate() will remove a non-duplicate rather than a duplicate -- because compression artifacts lead to duplicates that are no longer exact duplicates. It's not very common though. This can also be an issue with movies telecined onto VHS tapes because of the low signal to noise ratio of VHS.

If you literally mean "TFM().TDecimate()" that will leave you with a lot of duplicates. The default for TDecimate() is Cycle=5, CycleR=1 -- which removes 1 frame out of every group of 5. With a 30000/1001 fps source hat will leave you with 24000/1001 fps, not ~16000/1001 fps. So still 8 duplicates every second. If you mean using other values for Cycle and CycleR -- if you remove too many frames you will lose some unique frames. If you remove too few frames you will be left with some duplicates. Missing frames or duplicate frames will lead to jerks in the video when played back. They will be most noticeable in smooth panning shots, less so in character motion shots, and not at all in still shots.