DVB-T .ts file "Ghost Effect" remove

[provato]

Hi all, I am new to video encoding, but I have read a lot about it.
However, I find it difficult to remove the ghost effect from the dvb-t captures of my tv-tuner card....
Here is a screenshot:


I have tried several deinterlacers and programs to solve this but none seems to do the trick
I prefer using MeGui, but any other software that works is welcome
Here is a video part in case someone wants to experiment on it:
TVmario.ts

[Selur]

Have your tried sRestore and the typical Fieldblending and Frameblending removal-filters?
sRestore(23.976) looks like a good start,...
additionally I would crop to:
Crop(24,0,678,570)
than resize to:
Spline16Resize(704,504)
and encode with an NTSC 4:3 PAR

Cu Selur

Ps.: Your content is flagged top field first, but is really progressive.

[provato]

Thanks for your reply Selur!
You really gave me great food for thought and experiment. So this originally was an ntsc video that was transformed into a pal video and the change in framerate created double frames (with a little help from the deinterlace filter that was used in this transcoding I suppose...)

But what if I want to keep it a pal video? Then I am stuck with this ghost - double frame effect?

And another question: How does someone install avisynth filters in megui? You just put them in the avisynth tools folder?

[Selur]

But what if I want to keep it a pal video? Then I am stuck with this ghost - double frame effect?

That's why I posted the link to the Fieldblending and Frameblending removal-filters,...

How does someone install avisynth filters in megui? You just put them in the avisynth tools folder?

can stuff them in whatever folder you like as look as you load them through (LoadPlugin/Import/...) in the Avisynth script MeGui uses.

[sanlyn]

The video is already 720x576. It looks originally progressive+telecine improperly re-encoded as interlaced. Not certain, though, because so many people are finding more creative ways to screw up video. The .ts was encoded at a miserly 2300 bitrate and mp2 audio (if you want this to go NTSC, mp2 won't work). Looks like this was part of a project that tried to stuff 4 hours of bad data-starved video onto 1 disc.

[sanlyn]

So this originally was an ntsc video that was transformed into a pal video and the change in framerate created double frames (with a little help from the deinterlace filter that was used in this transcoding I suppose...)

If it was originally NTSC, it would have been film-based, 23.97 fps progressive with 3:2 pulldown for 29.97 fps playback, not interlaced. The 3:2 pulldown was not removed, so you have a field blend at regular cycles. Open the video in VirtualDub and move it one frame at a time, you'll see the blend cycle in frames where objects move.

[Selur]

if you want to stay with 25fps and don't mind a few duplicates:
srestore(omode=3)
is probably a good choice (or another omode < 6)

[provato]

Well... I've tried both your recommendations Selur but still I get videos with duplicates...
In the first case the script looked like this:

ColorMatrix(hints=true, threads=0)
srestore(23.976)
#deinterlace
crop(28, 0, -22, -8)
Spline16Resize(704,504) # Spline16 (Neutral)
FluxSmoothST(7,7) # Medium Noise

on the second:

ColorMatrix(hints=true, threads=0)
srestore(omode=3)
#deinterlace
crop(28, 0, -22, -8)
FluxSmoothST(7,7) # Medium Noise

[Selur]

What sRestore does is:
a. replace blended frames with neighboring frame (this causes duplicates)
and
b. if omode is 6 (the default), decimate the frame and frame count to whatever is specified.

so srestore(omode=3) will create duplicates (that's why I wrote: 'if you want to stay with 25fps and don't mind a few duplicates')
srestore(23.976) will create duplicates and than decimate the frame rate and count to 23.976, but if your source does not have 23.976 real frames you will still end up with duplicates.
-> Only proper way to end up without duplicates and blended frames would be to 1st replace the blended frame by neighboring frames and 2nd remove the duplicates. This requires you to either configure sRestore to decimate to a specific frame rate that is low enough or create a vfr stream. Especially old cartoons were often not created with 25 or 24 frames per second in mind (17 or 18 fps are not uncommon), so they often simply don't have enough frames to create a duplicate free stream with 25 or 24 frames per second.


Cu Selur

[provato]

well, I realise that I simply don't have the knowledge to remove the duplicates through srestore... But let me note this: Even newer cartoons/anime coming from my dvb-t captures have the same problem, so I really don't think it's the cartoon's age.

Moreover, I see no difference in the number/frequency of duplicate frames after I use srestore It's like it has done nothing!
Why don't you have a try on the file I posted to see if you can at least decrease the number of duplicates?

[Selur]

Leaving sRestore aside, have you counted roughly how many of 25 frames are blended or duplicates in your source and therefore how many frames you want removed?

I attached a sample where I lowered the frame rate (frame count accordingly to 12.5fps) for general amusement.
-> it might be a good idea to first denoise, deblock, etc. the source so sRestore and other tools can work more reliable on the material

[provato]

where there is movement, I'd say 1/3 of the frames are duplicated. But duplicates appearance doesn't follow a pattern. I mean, it's not good-good-duplicate-good-good-duplicate... etc
I tested with media player classic

[Selur]

where there is movement, I'd say 1/3 of the frames are duplicated.

Which would be around the 17fps I suspected.

But duplicates appearance doesn't follow a pattern.

That in itself is probably not such a problem. Normally sRestore(17) does quite a good job, problem here is that it still makes a lot of mistakes in guessing which are duplicates and later which so throw away and which to keep due to the sheer amount of compression and other artifacts.

If you got the drive to really dive into the whole Avisynth filtering you probably could first create an intermediate file in which you do a lot of work to denoise/deblock/clean etc. the source and later feed this clip to sRestore to decimate the amount of frames and assuming.
If sRestore still couldn't produce a good file, you probably could use it to replace most of the blended frames with duplicates of a neighbor and later use dedub or a similar filter to a. remove the duplicates and b. create a timecode file which is needed to create vfr output file.
(No, I will not try to do this, since it is far more work than I'm willing to send on such a source. )

Cu Selur

[provato]

Thanks for everything Selur, I guess this is really a hard problem as I thought it to be. Unfortunately all my dvb captures of anime/cartoon are like that, so this means that they are broadcasting screwed up video...