Convert VFR to Image Sequence (Keeping VFR)

[Vaengence]

I am not sure if that title explained well, and I will put my question upfront in TLDR format first:

TLDR
- How would I go about converting a VFR file that contains progressive in 3:2 pulldown and interlaced content into an image sequence, preserving the individual frames (best quality) in the process?

Long info.

I am trying to work with cleaning up old DVD's, like old Star Trek, Stargate, etc that have the wonderful VFR original files and have been experimenting with methods to work with them to find an end result I like. I have read enough articles on google searches on this process to know there is no perfect solution.

I am trying, on this attempt, to work instead with essentially this process:
1. Convert the VFR video to an image sequence
2. Work on the images standalone
3. Convert them back into the same VFR sequence as original but with (hopefully) cleaned up images.

The closest I have come to finding someones steps on the process is here:

https://github.com/queerworm/ds9-upscale/blob/master/guide.md

Which does pretty close to exactly what I am trying to do BUT converts it to frames by using TVEAI, which not only is the last thing I want to do initially, but also retains the interlacing artifacts and makes the issue worse.

Is there a relatively straightforward process in Avisynth to instruct the output of the frames in a way that preserves the correct frame counts for re-importing it back into a VFR file, but lets me work on the images separately?

NOTE: I have been using Avisynth in AVSPmod then opening it in VirtualDub to do the image export so far, so all I am trying to accomplish is the Avisynth commands to do this process. I have tried using TFM with TDecimate in Hybrid mode (to create a time codes file) but it errors immediately with the error "Non linear access detected in Mode 3". The info guide says the access must be linear or it will generate this error - but I am still relatively new and it doesn't explain what that means, or how to get around it (or what linear access is meaning in the first place)

Any assistance would be much appreciated, Thanks

[davexnet]

I am not sure if that title explained well, and I will put my question upfront in TLDR format first:

TLDR
- How would I go about converting a VFR file that contains progressive in 3:2 pulldown and interlaced content into an image sequence, preserving the individual frames (best quality) in the process?

Long info.

I am trying to work with cleaning up old DVD's, like old Star Trek, Stargate, etc that have the wonderful VFR original files and have been experimenting with methods to work with them to find an end result I like. I have read enough articles on google searches on this process to know there is no perfect solution.

I am trying, on this attempt, to work instead with essentially this process:
1. Convert the VFR video to an image sequence
2. Work on the images standalone
3. Convert them back into the same VFR sequence as original but with (hopefully) cleaned up images.

The closest I have come to finding someones steps on the process is here:

https://github.com/queerworm/ds9-upscale/blob/master/guide.md

Which does pretty close to exactly what I am trying to do BUT converts it to frames by using TVEAI, which not only is the last thing I want to do initially, but also retains the interlacing artifacts and makes the issue worse.

Is there a relatively straightforward process in Avisynth to instruct the output of the frames in a way that preserves the correct frame counts for re-importing it back into a VFR file, but lets me work on the images separately?

NOTE: I have been using Avisynth in AVSPmod then opening it in VirtualDub to do the image export so far, so all I am trying to accomplish is the Avisynth commands to do this process. I have tried using TFM with TDecimate in Hybrid mode (to create a time codes file) but it errors immediately with the error "Non linear access detected in Mode 3". The info guide says the access must be linear or it will generate this error - but I am still relatively new and it doesn't explain what that means, or how to get around it (or what linear access is meaning in the first place)

Any assistance would be much appreciated, Thanks

Post a Mediainfo report (text view) of the source file,
or a short clip of the file itself

[Vaengence]

Post a Mediainfo report (text view) of the source file,
or a short clip of the file itself

I have uploaded a short clip - I tried to upload the associated d2v file but the forum told me it was an invalid file, so if you need this as well let me know. I also made sure that the clip crosses one hopefully clear and distinct change between the two embedded framerates.

[badyu17]

No need for hybrid mode, I see no true interlaced sequences, it's all 3:2 pulldown. TFM with TDecimate will give you progressive 23.976 fps output that you can later export to whatever you like.

[jagabo]

I agree that the clip is a mix of hard and soft 3:2 pulldown and can be restored to the original 24p film frames with TFM+TDecimate. But knowing shows of that era there will be parts that are 30p and 30i.

[Vaengence]

If thats the case, then I have screwed up with the selection. I know with certainty (of several months now playing with these episodes) that they are mixed frame rate footage -DS9 is notorious for it. I do not, however, fully understand how to interpret the D2V output files to select the right sections - I "thought" I had done....

If it is the case that this episode doesn't have that issue, I certainly don't mind as it simplifies everything enormously - but I am still left with the issue when it does occur (and I know it will).

I am wondering if using the first movie length episode would be a better candidate for looking at.

The D2V file parse of the clip above has the following sections in it:

[GOP: open]
810 [B]: 1012,1013,1013.........1 *
811 [B]: 1014,1014..............2 *
812 [I]: 1015,1015,1016.........3 *
813 [B]: 1016,1017..............0 *
814 [B]: 1017,1018,1018.........1 *
815 [P]: 1019,1019..............2 *
816 [B]: 1020,1020,1021.........3 *
817 [B]: 1021,1022..............0 *
818 [P]: 1022,1023,1023.........1 *
819 [B]: 1024,1024..............2 *
820 [B]: 1025,1025..............2
821 [P]: 1026,1026..............2
[GOP: open]
822 [B]: 1027,1027..............2
823 [B]: 1028,1028..............2
824 [I]: 1029,1029..............2
825 [B]: 1030,1030..............2
826 [B]: 1031,1031..............2
827 [P]: 1032,1032..............2
828 [B]: 1033,1033..............2
829 [B]: 1034,1034..............2
830 [P]: 1035,1035..............2
831 [B]: 1036,1036..............2
832 [B]: 1037,1037..............2
833 [P]: 1038,1038..............2

I was "told" (through guides ie https://www.afterdawn.com/guides/archive/digital_video_fundamentals-ivtc_page_5.cfm), the bottom sequence was interlaced frames and the top a sequence of frames with a 3:2 pulldown, but I am now told here that this clip doesn't have any - can I ask, if not too much issue, what I need to look for then to identify what it is a mix of?

As I seem to be not reading the output correctly, if I can identify definitely an example, I can then upload a sample that can be looked at without wasting anyone's time.

Thanks.

[badyu17]

Watch the footage by your own eye frame by frame in AVSPmod instead of interpreting d2v which is index file for dgmpegdec that tells you nothing. Sequence with ship flying is also 3:2 pulldown. If this was real interlace then all frames would be interlaced (combed artifacts) or maybe even progressive 30p where every frame is unique and without interlacing.

[jagabo]

For simplicity I'm going round the rates from 23.976, 29.97, and 59.94 to 24, 30 and 60...

The clip is mixed soft telecine (24 fps progressive frames with pulldown flags that tell the player how to output 60 fields per second from those frames) and hard telecined (24p film with pulldown applied then encoded as 30 interlaced frames per second -- that the player outputs as 1 field at a time at 60 fields per second). Both can be restored to the original 24 fps film frames.

But as I said, other parts of the video will have hard telecined film with effects overlaid at 30p or 30i. If you reduce those to 24p the effects will get jerky. If you treat it as 30p the film will get jerky. The only way to retain the smoothness of both is to convert those sections to 60p. That means the underlying film will have a 3:2 repeat pattern, the 30p effects will have a 2:2 repeat pattern, and the 30i effects will have all unique frames.

Oh, and there will probably be some sections where the film was hard telecined, and 24p effects where hard telecined but overlaid out-of-phase. So the film may have field repeats like:

AAABBCCCDDEEEFF

but the effects:

AABBBCCDDDEEFFF

[Vaengence]

But as I said, other parts of the video will have hard telecined film with effects overlaid at 30p or 30i. If you reduce those to 24p the effects will get jerky. If you treat it as 30p the film will get jerky. The only way to retain the smoothness of both is to convert those sections to 60p. That means the underlying film will have a 3:2 repeat pattern, the 30p effects will have a 2:2 repeat pattern, and the 30i effects will have all unique frames.

Hello ^^. I will apologise, I am still learning the terminology but this paragraph here is exactly what I was referring to by the VFR of the original files in the first post and is the issue I have been trying to work around. I am guessing I am calling it the wrong thing. The jerkiness is precisely what I am trying to avoid.

The problem is that I have been experimenting with various upscaling and if I create a 60p source "before" that process I double and more the frames that I need to work on. I also think its a waste of time if by creating the 60p source, it is artificially guessing and/or duplicating frames as all that extra processing is really wasted. It will also move the whole process from up to a day to complete, to 2-3 days per episode which would likely take me three years to complete just one full series - so I am trying to reduce the processing time.

My thoughts were, to extract as much of the original content as I can in source frames(de-interlace or IVTC) to get as much original coded images as possible, process those and then recombine them at the end back into either a VFR video to go back to the original mixed footage, or else take those frames, process them, then recombine them into a 60p file conversion using the original timecodes.

I "thought" this would be relatively straightforward, but I am finding it is not.

I suppose the best way to describe it is I want to "take out" the original images, play with them, then re-import them back to their original sequence to avoid the jerkiness issue of going back into a VFR, or else finishing he initial conversion up to 60p to smooth out the issues.

Is this possible or is this going beyond the plugins Avisynth has generally?

That original link to a different guide dealing with the same thing had the same process I am trying to re-create - but they used TVEAI to convert the file directly, which attempts to upscale and work on images that have interlace like bars in them and is really what I am trying to avoid.

[smrpix]

if I create a 60p source "before" that process I double and more the frames that I need to work on..

No, it means you will have duplicate and triplicate frames. If you are working on them in say Photoshop you can simply substitute the one you fixed for the next one or two.

And yes, you are using the term VFR incorrectly. What you have is mixed frame rates in a constant 30i timebase (CFR.) VFR means the actual playback rate changes.

[jagabo]

I'd work in sections. Since the vast majority of the video is going to IVTC back to 24p I'd start with that. Then locate the sections where that didn't work right and treat those as necessary. Save each of those sections as image sequences in their own folder. Then you can upscale the folder fill of images. Finally turn the images back into videos, each at their appropriate frame rate. You can the append all the videos as a VFR video.

[jagabo]

Oops, wrong thread.