Remove Pulldown in Avisynth with manual mode

[Megafox]

How do I remove pulldown from the selection of the frame positions in which the search will be performed and with the transmitted template? For example: Pulldown (int start, int end, String template). This means that in a certain part of the video, several groups of I, P, B frames (the first letter in the template) will be searched, the second letter (I - interlaced, P - progressive) will be indicated in the template for each frame. Example: Pulldown (100, 3000, "ii-bi-bp-pp-bp-bi-pi-bp-bp-pp-bi-bi-pp-bp-bp"). The template for several identical groups next will be me. This is necessary in order to remove Pulldown in a changing template, but here the deletion will not be blind, but with the search for one or more groups of frames in a certain part of the video. After this, frames marked with "* i" will receive a field from the next frame. The search will be more accurate than any IVTC filter because it is a template. I want to know if there is such a filter for Avisynth or a program that allows you to make manual video processing?

[jagabo]

SeparateFields().SelectEvery(...).Weave()

[Megafox]

SeparateFields().SelectEvery(...).Weave()

Explain how to use it? The first function is clear to me, the third one too, but how does the second work? The syntax there is completely different, and there is no option that would check the frame type (I, P, B) and would understand that a certain frame in this group is interlaced. And this option generally can search for frame types? I want: only if the function has found a group identical to that described in the template, you can begin to remake frames and delete duplicates in this group. This operation without separating, or frame types will be changed. I do not need a blind template, sorry.

[poisondeathray]

Frame type (I,B,P) has nothing to do with interlaced vs. progressive. Frame type deals with compression ; interlaced vs. progressive refers to the content . They are independent

If you want to do everything manually, try yatta

https://forum.doom9.org/showthread.php?t=64440
https://github.com/hatt/Yatta

[Megafox]

Frame type (I,B,P) has nothing to do with interlaced vs. progressive. Frame type deals with compression ; interlaced vs. progressive refers to the content . They are independent

If you want to do everything manually, try yatta

https://forum.doom9.org/showthread.php?t=64440
https://github.com/hatt/Yatta

I know, I say that by types of frames he is looking for the identical groups with my template, and when he found, already according to the template I prepared, certain frames get the fields. Types are needed only for searching. And searching in certain range.

[poisondeathray]

Then what does this mean ?

Pulldown (100, 3000, "ii-bi-bp-pp-bp-bi-pi-bp-bp-pp-bi-bi-pp-bp-bp")

Internally, avisynth has no notion of i,b,p . Everything is "I" because everything is decompressed .



For frametypes -
If you are using dgindex with mpeg2 sources, you can use the parse d2v tool (tools => parse d2v) to look at frame type and more info

If you are using other file types, other compression formats - if you use ffms2 , you can use ffinfo()

Not all frametypes are correctly identified. For example you can have a non keyframe "i" frame vs. a true keyframe "I" frame , but ffinfo() will not distinguish between them

[Megafox]

Then what does this mean ?

Pulldown (100, 3000, "ii-bi-bp-pp-bp-bi-pi-bp-bp-pp-bi-bi-pp-bp-bp")

Internally, avisynth has no notion of i,b,p . Everything is "I" because everything is decompressed .



For frametypes -
If you are using dgindex with mpeg2 sources, you can use the parse d2v tool (tools => parse d2v) to look at frame type and more info

If you are using other file types, other compression formats - if you use ffms2 , you can use ffinfo()

Not all frametypes are correctly identified. For example you can have a non keyframe "i" frame vs. a true keyframe "I" frame , but ffinfo() will not distinguish between them

I can solve this problem with the help of a program, not Avisynth, which writes to the file the numbers of all frames and their type. And then you can enter a search for identical groups by template, after which in each group some frames will be marked, as interlaced, this data is taken from my template. After that, in Avisynth with the help of a special script, using this file to get its fields only those frames that were marked there. For creating template I use VirtualDub, check several groups, so as not to be mistaken because of an unobtrusive comb with insufficient movement and write right types. Just a lot of manual work. Does such a program exist or not?

[poisondeathray]

I can solve this problem with the help of a program, not Avisynth, which writes to the file the numbers of all frames and their type. And then you can enter a search for identical groups by template, after which in each group some frames will be marked, as interlaced, this data is taken from the template's memory. After that, in Avisynth with the help of a special script, using this file to get its fields only those frames that were marked there. For creating template I use VirtualDub, check several groups, so as not to be mistaken because of an unobtrusive comb with insufficient movement and write right types. Just a lot of manual work. Does such a program exist or not?

You can't do this completely accurately this way - you can't do it by "frames"

For example, a "combed" frame that looks "interlaced", might actually be progressive. (e.g. it might just be field shifted) . Or it might just be frame warping or just displaced

[Megafox]

I can solve this problem with the help of a program, not Avisynth, which writes to the file the numbers of all frames and their type. And then you can enter a search for identical groups by template, after which in each group some frames will be marked, as interlaced, this data is taken from the template's memory. After that, in Avisynth with the help of a special script, using this file to get its fields only those frames that were marked there. For creating template I use VirtualDub, check several groups, so as not to be mistaken because of an unobtrusive comb with insufficient movement and write right types. Just a lot of manual work. Does such a program exist or not?

You can't do this completely accurately this way - you can't do it by "frames"

For example, a "combed" frame that looks "interlaced", might actually be progressive. (e.g. it might just be field shifted) . Or it might just be frame warping or just displaced

I tell you that I will not be mistaken because I check several groups (10, start from end of previous template). And if I make a mistake (very unlikely), I'll see the remaining interlaced frames as a result. So is there a way to do this? I have already said that it is possible

[poisondeathray]

I tell you that I will not be mistaken because I check several groups (10, start from end of previous template). And if I make a mistake (very unlikely), I'll see the remaining interlaced frames as a result. So is there a way to do this? I have already said that it is possible

Maybe an excel macro ?



Sure you might not make a mistake in what you think is "checking." But you're not doing the proper check. You're going to have mistakes or worse end result (very likely). Unless you have only very simple sources (which can be handled automatically anyways)

Because you're only looking at frames. But different field patterns will screw up your identification. Sometimes a matching field is farther down, or farther up, and you will miss it by looking at frames . You will get lower quality because you will have not identified the proper field pair.



Unless I misunderstand what you're trying to do, you're going to have problems with some sources. It has to be done by fields, not frames.

[Megafox]

I tell you that I will not be mistaken because I check several groups (10, start from end of previous template). And if I make a mistake (very unlikely), I'll see the remaining interlaced frames as a result. So is there a way to do this? I have already said that it is possible

Maybe an excel macro ?



Sure you might not make a mistake in what you think is "checking." But you're not doing the proper check. You're going to have mistakes or worse end result (very likely). Unless you have only very simple sources (which can be handled automatically anyways)

Because you're only looking at frames. But different field patterns will screw up your identification. Sometimes a matching field is farther down, or farther up, and you will miss it by looking at frames . You will get lower quality because you will have not identified the proper field pair.



Unless I misunderstand what you're trying to do, you're going to have problems with some sources. It has to be done by fields, not frames.

First I'm looking for a template, rewriting the types from each frame, I find repeated rows of letters (from i). Then I check using several of the same groups to which position the interlaced frames belong. I'm looking for the moments where two interlaced frames are coming in succession, which exactly they are, and I determine their position by their type and type of neighboring frames related to the template. When all the frames are marked, the template is ready, and you can apply it to a text file, test it. But at me at manual processing in VirtualDub always all coincided at different situations. Now I'm looking for a program that can create a file with frame numbers and types that can search by the group template and mark the desired frames. And I'm looking for the Avisynth script, which will match the fields for the selected frames.

[jagabo]

As was pointed out, whether a frame is encoded as I, B, or P has nothing to do with pulldown patterns. You don't need how frames are encoded. So:

ii-bi-bp-pp-bp-bi-pi-bp-bp-pp-bi-bi-pp-bp-bp

becomes

iippp iippp iippp

And I already told you how to select fields for a manual IVTC in post #2. For this pattern it's:

SeparateFields().SelectEvery(10, 1,2,4,5,6,7,8,9).Weave()

[Megafox]

As was pointed out, whether a frame is encoded as I, B, or P has nothing to do with pulldown patterns. You don't need how frames are encoded. So:

ii-bi-bp-pp-bp-bi-pi-bp-bp-pp-bi-bi-pp-bp-bp

becomes

iippp iippp iippp

And I already told you how to select fields for a manual IVTC in post #2. For this pattern it's:

SeparateFields().SelectEvery(10, 1,2,4,5,6,7,8,9).Weave()

Read all my messages, I have already explained what I need.

[Megafox]

I tell you that I will not be mistaken because I check several groups (10, start from end of previous template). And if I make a mistake (very unlikely), I'll see the remaining interlaced frames as a result. So is there a way to do this? I have already said that it is possible

Maybe an excel macro ?



Sure you might not make a mistake in what you think is "checking." But you're not doing the proper check. You're going to have mistakes or worse end result (very likely). Unless you have only very simple sources (which can be handled automatically anyways)

Because you're only looking at frames. But different field patterns will screw up your identification. Sometimes a matching field is farther down, or farther up, and you will miss it by looking at frames . You will get lower quality because you will have not identified the proper field pair.



Unless I misunderstand what you're trying to do, you're going to have problems with some sources. It has to be done by fields, not frames.

First I'm looking for a template, rewriting the types from each frame, I find repeated rows of letters (from i). Then I check using several of the same groups to which position the interlaced frames belong. I'm looking for the moments where two interlaced frames are coming in succession, which exactly they are, and I determine their position by their type and type of neighboring frames related to the template. When all the frames are marked, the template is ready, and you can apply it to a text file, test it. But at me at manual processing in VirtualDub always all coincided at different situations. Now I'm looking for a program that can create a file with frame numbers and types that can search by the group template and mark the desired frames. And I'm looking for the Avisynth script, which will match the fields for the selected frames.

[johnmeyer]

I'll admit that I do not completely understand what you are trying to do, but the suggestion of using an Excel macro is excellent and is what I would recommend.

I do this quite frequently and it is a very good way to deal with complicated pulldown removal.

I use TVITC. I enable all the TFM metrics and output them to a text file. I do this by opening a one-line AVISynth file in VirtualDub and then have VirtualDub do a "video analysis pass." This creates a text file containing the metrics for each frame. I import that text file into Excel. I have an Excel template which contains formulas which look at the multiple columns of TFM metrics for each frame, and also at the same metrics for the five frames before and after that cell. The formulas create the logic that determines which fields should be decimated. Because it is really easy in Excel to see patterns and, from those patterns, determine which fields need to be decimated, it is actually pretty easy to write Excel formulas which convert those patterns into

I then wrote a macro which does nothing more than export the text from the resulting formulas into a "decimate" file. The macros are not necessary and this could be easily done by hand, but since I use this all the time, I wanted something more automatic.

The final step is to import that text file into MultiDecimate which actually does the pulldown removal.

Here is the TFM line from the first script that exports the metrics:

tfm(display=false,micout=2,mode=0,cthresh=45,mi=120,pp=1,metric=0,field=1,mmsco=false,micmatching=2,slow=2,output="e:\tfm.txt")

This is for a very different situation than yours, so you should ignore all the metrics and use your own. However, the key thing is the "tfm.txt" code which is what outputs the metrics to a text file.

I then use the MultiDecimate EXE to convert the metrics output by my Excel file into the format needed by the MultiDecimate AVISynth filter. I guess I could have done this in my Excel macro, but this saved me some programming time. Here is the MultiDecimate code that actually performs the pulldown:

tfm(display=false,input="E:\film\input.txt")
MultiDecimate(pass=2)

Note that "input.txt" is a different file than what was exported in the first pass. That is because "input.txt" is the file created by MultiDecimate, and is in the format that MultiDecimate requires.

[johnmeyer]

Here is what the metrics look like in Excel. The columns way over on the right (you have to scroll horizontally) that have headers are the columns where my formulas reside. The columns on the left are the ones output by TFM. The reason the formulas show "#VALUE!" is that I just copied this from the Excel template and have not yet imported the complete values from TFM.

#TFM	v0.9.12.2	by	tritical														
field	=	top															
crc32	=	de7884a2										Match	Frame		Delete	Frame	mfile
0	c	-	[23]	(23	23	12	23	19)	-	c	#VALUE!	c	0	c	-	0	1
1	c	-	[10]	(19	10	10	12	16)	#VALUE!	#VALUE!	#VALUE!	c	1	c	#VALUE!	1	#VALUE!
2	c	-	[18]	(16	18	14	10	14)	#VALUE!	#VALUE!	#VALUE!	c	2	#VALUE!	#VALUE!	2	#VALUE!
3	c	-	[12]	(14	12	12	14	14)	#VALUE!	#VALUE!	#VALUE!	#VALUE!	3	#VALUE!	#VALUE!	3	#VALUE!
4	c	-	[18]	(14	18	19	12	17)	#VALUE!	#VALUE!	#VALUE!	#VALUE!	4	#VALUE!	#VALUE!	4	#VALUE!
5	c	-	[10]	(17	10	14	19	23)	#VALUE!	#VALUE!	#VALUE!	#VALUE!	5	#VALUE!	#VALUE!	5	#VALUE!
6	c	-	[27]	(23	27	12	14	15)	#VALUE!	#VALUE!	#VALUE!	#VALUE!	6	#VALUE!	#VALUE!	6	#VALUE!
7	c	-	[11]	(15	11	10	12	16)	#VALUE!	#VALUE!	#VALUE!	#VALUE!	7	#VALUE!	#VALUE!	7	#VALUE!
8	c	-	[15]	(16	15	15	10	19)	#VALUE!	#VALUE!	#VALUE!	#VALUE!	8	#VALUE!	#VALUE!	8	#VALUE!
9	c	-	[10]	(19	10	15	15	17)	#VALUE!	#VALUE!	#VALUE!	#VALUE!	9	#VALUE!	#VALUE!	9	#VALUE!
10	c	-	[11]	(17	11	13	15	16)	#VALUE!	#VALUE!	#VALUE!	#VALUE!	10	#VALUE!	#VALUE!	10	#VALUE!

[poisondeathray]

probably some language barriers here...

And I'm looking for the Avisynth script, which will match the fields for the selected frames.

Did you also look at TFM ovr file ? You can manually specify ranges or override decisions . You can tell it what ranges to look at or ignore in a text file. You can tell it what pattern is actually there over what range of frames . You can override manual field matches (this field with that field over there)

[Megafox]

probably some language barriers here...

And I'm looking for the Avisynth script, which will match the fields for the selected frames.

Did you also look at TFM ovr file ? You can manually specify ranges or override decisions . You can tell it what ranges to look at or ignore in a text file. You can tell it what pattern is actually there over what range of frames . You can override manual field matches (this field with that field over there)

Thanks for the help, I'll take your method into consideration. I do not know English well, so I use an interpreter. As with any program, you can output data from mpeg2 video in this format in a text file: the number of lines in the text - the number of frames in the video, in each line the information about the frame: "i; p". The first value is the frame type I / P / B, the second is the default p (progressive), this value will be edited using a special program that processes the templates. Or in xml format:
<video>
<frame>
id="18"
compressType="b"
type="p"
</frame>
</video>
Or do I have to learn programming languages ​​for creating program and filter?