Dropped frames. How to create a log?

[salvo00786]

Hi... Please let me know, if exist a procedure for creating a log file that enlist all the dropped frames.

For example I have this file (the same I used in another thread) for learning reason:
https://drive.google.com/file/d/1tiqyLmlTFuI1Nnx_JXB0-EYKk2woQIRU/view?usp=sharing

With the help of Jagabo I used:

ShowFrameNumber(x=10, y=20)

If my assumption is correct, there is one dropped frame every 20, right? This is part of the concert, but I have seen that the pattern is the same for the entire concert.

I was using this old thread
https://forum.videohelp.com/threads/407054-Duplicate-Frames-or

but I don't understand this post:
https://forum.videohelp.com/threads/407054-Duplicate-Frames-or#post2668081

Why if in the old thread there are two dropped frames every 50, with selecteven, are selected 61 and 63 instead of 51 and 53?

Please if you can give me a script to fix this, thanks.

[johnmeyer]

Dups are easy to find but dropped frames are difficult because detection relies on sensing a jump between frames that is larger than the normal temporal difference between nearby frames.

I did write an AVISnyth script which attempted to deal with this:

Automatically fix dups followed (eventually) by drops

Note that it also talks about duplicates. That is because, in most situations, a dropped frame is usually "made up for" by adding a duplicate frame shortly after the drop. If this weren't done, the audio would be out of sync after a few drops.

Here is another thread about the same topic, but with different solutions offered.

Dropped frames

[salvo00786]

Hi johnmeyer. Thanks I read these threads, and I have tried your scripts. After using your script, the problem decreases but remains. Sometimes ago, Jagabo gave me a completely smoother file, with the problem totally resolved. Your script is very useful because I have tried it on other files, but this file is very difficult to repair.

[johnmeyer]

Sometimes ago, Jagabo gave me a completely smoother file, with the problem totally resolved.

I would love to see Jagabo's script. This is a common problem and my attempt works exactly as you describe: it improves the video, but does not find and fix every last drop.

[poisondeathray]

The pattern is set in your example, every 20th frame is missing - between 3,4 , 23-24, etc...

orig = LWLibavVideoSource("002 - Copia.mkv")

orig.selectevery(20,3,4)
z_convertformat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f")
RIFE(model=5, sc=true , sc_threshold=0.11).selectevery(8, 1,5)
z_convertformat(pixel_type=orig.pixeltype, colorspace_op="rgb:709:709:f=>709:709:709:l")
replace=last 

interleaveevery(orig,  replace,  21,4)

[johnmeyer]

poisondeathray, thanks for the script. I somehow missed RIFE. When did that get developed? I'll have to try it out!

[poisondeathray]

poisondeathray, thanks for the script. I somehow missed RIFE. When did that get developed? I'll have to try it out!

RIFE was ported to AVS+ Aug 21, 2022
https://github.com/Asd-g/AviSynthPlus-RIFE

This is the NCNN version - that means it runs on vulkan and is quite a bit slower than the CUDA or Tensor version in vapoursynth if you have a Nvidia card. But vulkan version can run on all modern GPU's , so it's more accessible to everyone

[salvo00786]

Hi... I was using this script, but probably I am wrong because this doesn't resolve completely my problem.
FrameRateConverter(FrameDouble=true)
SelectEvery(20, 0,2,3,4,6,8,10,12,14,16,18).


Is rife better or there is an error in my script?

[poisondeathray]

Hi... I was using this script, but probably I am wrong because this doesn't resolve completely my problem.
FrameRateConverter(FrameDouble=true)
SelectEvery(20, 0,2,3,4,6,8,10,12,14,16,18).


Is rife better or there is an error in my script?

It would be

SelectEvery(20, 0,2,4,6,7,8,10,12,14,16,18)

Because the 3,4 intermediate is being inserted according to original frame numbers. But the framenumber changes after FrameRateConverter


RIFE is a frame interpolator, just like FramerateConveter . It uses machine learning, the results are generally better - there are fewer artifacts, less blending. But it's much slower. Newer versions of FrameRateConverter have the option to use RIFE as well .

EDIT: that's not right either for SelectEvery... It should be every 40 for 1 cycle, because you're selecting from the FRC output in that script . I'll look at it later, but the script in post #5 works

[jagabo]

Hi... I was using this script, but probably I am wrong because this doesn't resolve completely my problem.
FrameRateConverter(FrameDouble=true)
SelectEvery(20, 0,2,3,4,6,8,10,12,14,16,18).

The pattern repeats every 20 frames. After doubling the frame rate those 20 frames have increased to 40 frames, with the even numbered frames the original frames, and the odd numbered frames the interpolated frames.

SelectEvery(40, 0, 2, 4, 6, 7, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38)

The missing frame is between frames 3 and 4 (starting with 0) before doubling the frames. After doubling the frames it's between frames #6 and #8.

Is rife better or there is an error in my script?

RIFE usually gives better quality motion interpolation. But it's much slower and only works on planar RGB.

[salvo00786]

Ok, thanks for all your replies. I have not considered the fact that after doubling frames, the frames are not in the same position as before.

Is Rife the best frame interpolator that I can use with Avisynth?

How is the quality of SVPflow?

[poisondeathray]

Ok, thanks for all your replies. I have not considered the fact that after doubling frames, the frames are not in the same position as before.

Is Rife the best frame interpolator that I can use with Avisynth?

Overall, yes . The 2.3 and 2.4 models are generally the best for general use, but slower. The 4.x models are about 2x faster

Look on your video - the images are cleaner, less ghosting and lblending

How is the quality of SVPflow?

Generally slightly worse than mvtools2, so the worst.

Newer versions of SVPFlow use RIFE too now, but not avs version has not been updated


You can use RIFE with SelectEvery only. It looks simpler, and the end results are the same as the RIFE + InterleaveEvery script

z_convertformat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f")
RIFE(model=5, sc=true , sc_threshold=0.11)
z_convertformat(pixel_type=orig.pixeltype, colorspace_op="rgb:709:709:f=>709:709:709:l")
SelectEvery(40, 0, 2, 4, 6, 7, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38)

However - it's slower. The InterleaveEvery version is about 20% faster - it has to do with the way avs+ caches and processes frames. You end up processing more frames using RIFE then SelectEvery, and RIFE is a relatively slower filter

[salvo00786]

I think that I can use safely the 4.6 model because after that I am taking 1 new interpolated frame every 40, and I will discard all the new interpolated frames.

But with this, you are telling me that 2.3 and 2.4 models are better than the last 4.6 model?

[poisondeathray]

I think that I can use safely the 4.6 model because after that I am taking 1 new interpolated frame every 40, and I will discard all the new interpolated frames.

I would still use the InterleaveEvery script, because it's faster . It's interpolating fewer additional frames that you are only discarding later

But with this, you are telling me that 2.3 and 2.4 models are better than the last 4.6 model?

In general, overall, yes . There might be some frames slightly worse, but that's the consensus if you look around various forums. There are comparisons posted on various forums, reddit, github. Everyone pretty much agrees. The 4.x models emphasize speed over quality

You can test it yourself, examine a few frames, maybe 4.x is good enough for on this source (it probably is). But FRC using mvtools2 is clearly worse on your source

A certain model might "solve" a given frame almost perfectly , but another might fail on the same frame, you might have to switch to say 3.0 for one frame. Maybe 2.8 for another. It's just the way machine learning models work. Or maybe mvtools2 might solve one frame better- I've seen this happen and can post examples where 99% of the frames RIFE solves , but a few frames it misses (artifacts), but mvtools2 works on a given frame, but fails the others. You might have to mix/match if you want to improve results. On some types of content, they all fail miserably. If you want to use a general "best" model for live action content, set and forget - it would be 2.3 or 2.4. If you care more about speed, use a 4.x model

[salvo00786]

Ok thanks. Another question, please can you tell me the commands for convert the colors to RGB48le and to BT709 and what is the best plugin for the conversion?

[poisondeathray]

Ok thanks. Another question, please can you tell me the commands for convert the colors to RGB48le and to BT709 and what is the best plugin for the conversion?

Do you mean convert your Source video , to RGB48 (16bit packed RGB) using Rec709 matrix ?

In avisynth ,

ConvertToRGB48(matrix="rec709")

[salvo00786]

I have to convert a file to RGB48le, upscale it in Topaz and after topaz reconvert it in BT709. This is my question.

Is this the right way?

ConvertToRGB48(matrix="rec709")
Use Topaz
ConvertToBt709(matrix="rec709")

II know that there are two rgb48, rgb48le and rgb48be, but please can you explain me the differences? le and be stands for? Rgb48 is a 16 bit depth file or is a 8 bit depth file?

[poisondeathray]

I have to convert a file to RGB48le, upscale it in Topaz and after topaz reconvert it in BT709. This is my question.

Is this the right way?

ConvertToRGB48(matrix="rec709")
Use Topaz
ConvertToBt709(matrix="rec709")

II know that there are two rgb48, rgb48le and rgb48be, but please can you explain me the differences? le and be stands for? Rgb48 is a 16 bit depth file or is a 8 bit depth file?

ConvertToRGB48(matrix="rec709")

Are you using image sequence ? Some versions of VEIA can load avs scripts directly (it got broken, then fixed)

rgb48le is little endian, rgb48be is big endian

RGB48 is 16bit packed RGB . gbrp16le is 16bit planar RGB, little endian


After topaz , if you load an RGB file (such as image sequence) and want to convert to 8bit YUV 4:2:0 (you didn't specify the end format you wanted)

ConvertToYV12(matrix="rec709")

[salvo00786]

I am not using image sequence. What container I have to use because of RGB48? Probably not a h264 mp4, because is 8bit, right? Ok and if I convert to rgb48 is not important if the file is a be or le? After Topaz i want to convert it to 10bits bt709 into a prores mov file

[poisondeathray]

I am not using image sequence. What container I have to use because of RGB48? Probably not a h264 mp4, because is 8bit, right?

h264 doesn't support 16bit RGB


16bit RGB PNG or TIFF image sequence, always works

16bit RGB FFV1 in MKV used to work, but got broken in some versions

AVS input used to work, but got broken in some versions

EXR image sequence used to work, but got broken in some versions .

You can ask on Topaz forum too

Ok and if I convert to rgb48 is not important if the file is a be or le?

No

After Topaz i want to convert it to 10bits bt709 into a prores mov file

Using topaz to export it or something else ?

What format are you exporting from Topaz?

I use PNG image sequences in/out

[jagabo]

II know that there are two rgb48, rgb48le and rgb48be, but please can you explain me the differences? le and be stands for?

Little endian and big endian. Multibyte values (16 bits is two bytes) can be stored with the low order byte first, followed by the high order byte, little endian; or the high order byte first, followed by the low order byte, big endian.

https://en.wikipedia.org/wiki/Endianness

Rgb48 is a 16 bit depth file or is a 8 bit depth file?

16 bits per channel. Ie, 16 bits for each red, 16 bits for each green, 16 bits for each blue.

Then there are interleaved vs. planar formats. In interleaved formats the channels are interleaved: RGBRGRGB.... In planar formats all the R are stored first, then all the G, then all the B: RRRRRR...GGGGGG....BBBBBB...

[salvo00786]

Ok, thanks, this is very informative.

For poisondeathray, usually I export in h265 mkv.

Now I am experimenting with Iris and I like it very much (not too sharp, very natural). After that with Staxrip, I use CASm (the upgraded CAS) for sharpening and sometimes f3dkbneo for add grain.

Sometimes I give a file two passage, one in TOPAZ VIDEO ENHANCE AI 2.3.0, when I use the Alq-8 at 100% or 120% (one of the best models of all Topaz version for me). After that i go on with Proteus model or now with the Iris model

[poisondeathray]

usually I export in h265 mkv.

What kind of h265 ?

Assuming it's 8bit or 10bit YUV h265, assuming everything was done correctly up to that point (709 matrix in/out) . To convert that to 10bit 422 for Prores HQ ,

z_convertformat(pixel_type="YUV422P10")

But don't assume, there are some point releases that do the RGB/YUV conversion incorrectly, some used 601 for example. RGB is always done correctly (internally VEA works in RGB), if you control the conversion yourself - that's why I tend to use image sequence. There is less room for VEIA to screw up. Also, if it crashes, your work is saved up to that point

Now I am experimenting

It's all about experimenting and always changing...what works now can get "broken" later... unfortunate but true... And newer versions of particular models can often be worse.

[jagabo]

Ok and if I convert to rgb48 is not important if the file is a be or le?

No

Some programs may support both -- in that sense it doesn't matter. But to clarify, it matters that it's flagged correctly. If you save big endian but flag as little endian (of vice versa) you will get very bad results. And of course, some programs will only support one of the two.

[salvo00786]

Yes poisondeathray, this is very true. I reported a bad bug. The latest version without this bug was the 3.0.11. After that this bug is present in all versions. When the software can't reconstruct detail go in full dehaloing mode and this is true for proteus and for dione robust 4.

[poisondeathray]

If you plan to export 16bit TIFF or PNG from VEIA, and you are using avisynth to load image sequence with ImageSource , don't forget to use pixel_type="RGB48" . It's not automatically detected as 16bit and will truncate to 8bit

[salvo00786]

Ok, thanks for the info

[salvo00786]

Hi... Please can you help me with this file? I post an example, I can't find the right pattern for restoring the correct rotation without dropped frames.

FrameRateConverter(FrameDouble=true)
SelectEvery(100, 0,2,4,6,8,10,12,14,16,18,\
                 20,22,24,26,28,30,32,34,36,38,\
                 40,42,44,46,48,50,52,54,56,58,\
                 60,62,64,66,68,70,72,74,76,78,\
                 80,82,84,86,88,90,92,94,96,97,98,99)

After this script the rotation is better, but there is a point that isn't perfect

This is the file before my script

https://drive.google.com/file/d/1tkKApUzx2qiMTiXOFU7QyV5djZ0xziyn/view?usp=sharing

[jagabo]

80,82,84,86,88,90,91,92,93,94,96,98)

[salvo00786]

Thank you Jagabo, but I necessary have to use 97 and 99, because after the initial rotation circle the pattern change, but If I use 97 and 99 with your script, the circle rotation is not good. Anyway, probably the easiest solution is to split the file and after the first 16 seconds of the circle with your script, merge the file with my script.