Interlacing artifacts

[Randombit]

I have a file here which apparently had interlaced source, but was encoded as non-interlaced. At least that's what I think it is. Anyhow, when playing it back deinterlacing does not help, it still has those lines around moving objects. Is there a filter I can apply to get rid of this when re-coding? I can post a link to a piece of this file if needed.

[johnmeyer]

There is no way you can get any help with this unless you post a short clip. That clip should not be re-encoded. Use ffmpeg or other utility to do the cutting. Make sure to include a section that has lots of movement. Very often things like this are not what is claimed, and there is a good chance that this is not simply "encoding an interlaced video as non-interlaced," something that shouldn't produce something that is too bad.

[Randombit]

Thanks for reply! Please see attachment, mediainfo tells me it is progressive. No wonder the players don't know how to deinterlace it.

Video
ID                                       : 1
Format                                   : MPEG-4 Visual
Format settings, BVOP                    : Yes
Format settings, QPel                    : No
Format settings, GMC                     : No warppoints
Format settings, Matrix                  : Default (H.263)
Codec ID                                 : V_MPEG4/ISO/ASP
Codec ID/Info                            : Advanced Simple Profile
Duration                                 : 13 s 914 ms
Bit rate                                 : 1 079 kb/s
Width                                    : 720 pixels
Height                                   : 480 pixels
Display aspect ratio                     : 3:2
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Standard                                 : NTSC
Color space                              : YUV
Bit depth                                : 8 bits
Scan type                                : Progressive
Compression mode                         : Lossy
Bits/(Pixel*Frame)                       : 0.104
Stream size                              : 1.79 MiB (83%)
Writing library                          : DivX 5.1.0 (UTC 2003-09-02)
Default                                  : Yes
Forced                                   : No
DURATION                                 : 00:00:13.913000000

[LMotlow]

Thanks for the clip, but it's not interlaced. It was progressive video with 3:2 pulldown applied (telecine). The idiot who processed it then encoded it as progressive so that 2 telecined frames of every 5 frames are field-blended. In other words, and considering the awful quality, it's borked.

You can try inverse telecine, although some frames will look fuzzy:

AssumeTFF()
TFM(order=1,pp=0).TDecimate()
QTGMC(InputType=2)

The attached was a quickie. It needs lots of cleanup.

[Randombit]

So you are saying nothing can be done to smoothen it?

[LMotlow]

Yeah, the attached mp4 in my previous post is a sample of the script output, but it needs far more work. Why would anyone encode 720x480 at such a stingy bitrate. It has enough block noise to sink the U.S.S. Nimitz.

[Randombit]

Thanks for your help. Actually I have a bunch of those old movies, all encoded the same way, I'd love to watch them. Of course, I can watch them as they are, as you already mentioned there is no quality to begin with.

[LMotlow]

You could clean them up somewhat, but but they're in bad shape to start with if they all look like this. Tip: Hollywood movies are usually progressive video (film) with some form of pulldown applied for 29.97 playback. The mp4 posted runs at the original film speed of 23.976 fps.

The attached is another quickie using the earlier script but with dfttest, DeBlock, and gradfun3 added. Slightly cleaner, but you can see that adding more filters brings you pretty fast to the point of diminishing returns.

[johnmeyer]

The block noise is indeed awful.

Some sort of motion adaptive algorithm was applied to what is otherwise a standard 3:2 pulldown. If you use AVISynth, and run the video through a one-line "separatefields()" script, what you see at the beginning of the clip is that the soldier's entrance through the door looks like nothing more than 3:2 pulldown. In fact, when I first looked at just that part of the frame, I couldn't figure out what LTMotlow was talking about, and that standard IVTC was going to be all that was needed.

However ...

I then went back to the first frame and this time looked at the woman on the far left as she turns, and during portions of her rotation, there is movement in every single field (note: field, not frame), even though there is no movement in the soldiers forward progress. I'll bet that those fields were motion estimated.

I tried several variations of what LT did, using a lot more of the TFM and TDecimate parameters. I did not look at his results, but I am sure that mine are no better. The problem is that the motion estimated fields create an intermediate point in time that didn't exist in the original, but only for certain sections of the frame, and not on all frames. Thus, parts of the frame are behaving according to the standard 3:2 pulldown, and all you have to do is remove the duplicate field (which is what TFM/TDecimate does). However, a portion of the frame has a field that is moving relative to the other field (something that should not happen with progressive). This is what gives you fringing artifacts, but only on certain portions of the frame.

I suppose that, if this were for a big paid project, one could spend half a day trying to reverse engineer the original adaptive motion estimation, and try to recreate the original fields.

Another possibility would be to go ahead and do the TFM/TDecimate to get what should be a clean progressive result, then bob the result. Then use YDifference to look for frames where there is movement where there shouldn't be any. You could then go back and duplicate a field that better matches, in time, the field that has been "borked" and use that, even though it is an even field rather than odd (or vice versa). You'd then shift that field to the proper spatial position and use it instead. This would lead to a very subtle loss of detail on fields where this substitution was made, but with all the other noise, you'd never notice it.

I did take the first step towards doing this, but I don't have time to see it all the way through.

Here is some really ugly "proof of concept" code I cobbled it together from code I have lying around. It is not pretty looking, and I am not proud of it, but it does put metrics on the screen that show you how this might work.

loadPlugin("c:\Program Files\AviSynth 2.5\plugins\TIVTC.dll")

AVISource("E:\Documents\Dnload\UNPACK\7z\ffmpeg-20150311-git-2e5605f-win32-static\ffmpeg-20150311-git-2e5605f-win32-static\bin\output.avi")
AssumeTFF()
converttoYV12(interlaced=false)

tfm(display=false,field=1,mode=5,pp=0,cthresh=255) 
tdecimate(cycleR=2,cycle=10,mode=2,rate=23.976)

global source=bob()
ShowMetrics(source)

function ShowMetrics (clip c) 
{
fixed=c.ScriptClip("Subtitle(String(YDifferenceToNext(source) ) ) ") 
return fixed
}

What you will see is the the YDiff metrics are really large (>4) when the next bobbed frame (i.e., field) is truly from a different moment in time. This is exactly what you would expect with normal progressive video. When a field that truly matches comes next (i.e., a normal progressive field), the YDiff number is <1. All this is what one would get with clean progressive video.

However, when the field which should match instead has motion, you get a YDiff number that is >1.

What you'd need to do is refine the difference metric (a simple YDiff may work, but may need something a little more sophisticated) and then perform a conditional operation that, when YDiff > 1 for a field that should match the other field, you that bobbed field with a duplicate of the previous (matching) field. You then would undo the bob, re-weaving everything back together, with the proper spatial alignment, and you'd have the result you want.

It is, however, more work than I'm willing to do at the moment.

[Randombit]

Thank you all very much, I'll see what I'm able to do with this information. Some of it is certainly over my head. BTW, I haven't used Windows for 17 years, is there avisynth for Linux?

[Randombit]

Any hope I could use the wisdom above in Linux?

[johnmeyer]

I have no idea about AVISynth in Linux. However, I assume you are able to use Google in the country where you live:

AVISynth Linux

[edit] I didn't realize that Acadiana is not a country, but the Cajun region of Louisiana. How did I make it to social security age and not know that?

As for my script, there are actually only about three lines of additional code needed to implement what I described, if one is willing to live with the detection code I wrote. All that is required is a conditional statement that replaces a bobbed frame with the previous frame when a glitch is detected, and then two lines of reweave code.

And, if your original really has this horrible encoding noise, it might not hurt to try to filter a bit of that, while you're at it.

[Randombit]

Thanks! I did some search, but I'm afraid I didn't notice it, I gave up after I found someone writing it is possible only using Wine. Will dig into it ...

[Randombit]

The avisynth code is not maintained, it is too old to build with current GCC. My system GCC is 7.1, there was an ABI change between 4.x and 5.x versions. However, I found vapoursynth, does you code work with vapoursynth?

[johnmeyer]

I have no idea whether it works with vapoursynth, but I doubt it. That is a newer branch of AVISynth and I think the code is quite different.

If you ever can find a Windows machine, here is a complete script that implements the entire fix that I outlined in my earlier post. I've also provided a link to the resulting video that this script creates.

I think I fixed pretty much everything having to do with the weird fields, and also applied some denoising. I probably should have added a little sharpening to punch up the result a little. Also, I made no attempt to deal with the macroblocks that are quite evident on big flat areas (like the walls).

#Script to IVTC telecined film (normal IVTC), but then find a replace 
#any bottom (2nd) field that doesn't "match" top field.
#Use "return metrics" to get a sense of what a bad field metric number is for your clip
#John Meyer 2017

loadPlugin("c:\Program Files\AviSynth 2.5\plugins\TIVTC.dll")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

#Set this threshold based on what you see when you return the metrics variable
thresh=1.6

AVISource("E:\Documents\Dnload\UNPACK\7z\ffmpeg-20150311-git-2e5605f-win32-static\ffmpeg-20150311-git-2e5605f-win32-static\bin\output.avi")
AssumeTFF()
converttoYV12(interlaced=false)

#Recover original progressive frames (i.e., standard IVTC)
tfm(display=false,cthresh=255) 
source=tdecimate()

#Bob the source to create frames from fields
global bobbed=bob(source)
metrics=ShowMetrics(bobbed)

#Replace 2nd field with 1st field, when bad field is detected.
fixed = ConditionalSelect (bobbed, "(current_frame %2 == 0) && (YDifferenceToNext(bobbed) > thresh) &&(current_frame<>0)? 0 : 1", trim(bobbed,1,0), bobbed )

#Change bobbed video back into progressive video
separatefields(fixed)
SelectEvery(4,0,3)
final=Weave().MDegrain(8,4,0)

return final

#Optional: enable this return and comment out return above to see metrics
#return metrics

#Remove comments from this next section in order to see "before" followed by "after."
/*
return Interleave(
\    source
\  , final
\ )
*/


#Function to show metrics
function ShowMetrics (clip c) 
{
#fixed=c.ScriptClip("Subtitle(String(YDifferenceToNext(bobbed) ) ) ") 
fixed=c.ScriptClip("Subtitle(String((current_frame %2 == 1) && (YDifferenceFromPrevious(bobbed) > thresh) ) ) ") 
return fixed
}

#Function to reduce noise
function MDegrain(clip source, int "blksize", int "overlap", int "dct") {
  overlap=default(overlap,0) # overlap value (0 to 4 for blksize=8)
  dct=default(dct,0) # use dct=1 for clip with light flicker

  super = source.MSuper(pel=2, sharp=1)
  backward_vec1 = super.MAnalyse(isb = true, delta = 1, blksize=blksize, overlap=overlap, dct=dct)
  forward_vec1 = super.MAnalyse(isb = false, delta = 1, blksize=blksize, overlap=overlap, dct=dct)
  backward_vec2 = super.MAnalyse(isb = true, delta = 2, blksize=blksize, overlap=overlap, dct=dct)
  forward_vec2 = super.MAnalyse(isb = false, delta = 2, blksize=blksize, overlap=overlap, dct=dct)

  MDegrain2(source,super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=800) 
}

Here is a link to the result. I used HuffYUV to encode, so the file is stupidly big (100 MB). I wanted to use an encoder that is common.

Fixed Video

[Randombit]

Thanks alot. Unfortunately I promised myself not to install Windows ever again, I was a sysadmin for a decade and simply cannot stand Windows any more. Honestly, I think MS Windows is an insult to a computer enthusiast. I used to have a Windows box at work, but I'm disabled now.
I need to learn how to use vaporsynth, then I may be able to use your code with it if it is only matter of different syntax.

[johnmeyer]

I promised myself not to install Windows ever again, ...

OK, although I think you are being a little silly, and perhaps shooting yourself in the foot. However, we all do have our personal preferences, so I won't say any more than that, because I understand how some computer conventions can rub a person the wrong way. I too have my own peculiar computer software dislikes.

I started using computers in the 1960s with paper tape and punch card inputs. I've programmed lots of computers and have used just about every personal computer operating system including CP/M (I worked at Digital Research), DOS, Windows (all versions, from the May 1990 3.0 onward), Mac, Linux, BeOS, and even Job's Next computer. They all do pretty much the same thing, and in the end, as long as they don't get in the way, it is the application you are using that matters. In this case, that would be AVISynth, which is one of the most amazing video restoration tools I've every used.

I'm typing this on Firefox, and I've use the Linux version and they work pretty much the same way (i.e., I don't even know what operating system I'm using while in Firefox, and don't have to do much of anything different).

However, as the French say, À Chacun Son Goût.

P.S. Do download and look at the video and let me know if you think it looks any better.

[johnmeyer]

Here is a very tiny section from the original frame 25:



Here's the same small section after:

[Randombit]

I downloaded the video and I think it is greatly improved.

Off-topic now.
Speaking generally of cars for instance one should notice cars without brakes should not be on public highways, this is a safety issue.
Speaking generally of computer operating systems one should notice operating systems with thousands of unpatched security holes should not be connected to the internet, because this is what computer viruses use - vulnerabilities. Doing so would cost billions ... of wait, they are doing it and it does cost many, many billions annually.

[LMotlow]

P.S. Do download and look at the video and let me know if you think it looks any better.

Kinda noisy and fuzzy, ain't it?

[johnmeyer]

They all have security holes, but Windows has 10x the installed base so that's where the hackers go. As Willie Sutton, a famous bankrobber once said, when asked why he robbed banks: "Because that's where the money is." If you are going to spend "x" amount of effort to screw with a computer, you will get 10x the return for your time, compared to going after other O/S platforms, if you go after Windows.

Here is a more spectacular before/after comparison:

Before (full frame)


After (full frame)

[Randombit]

You know, this is a movie from 1936, there really is not much hope I could get my hands on a better copy. And I absolutely love those pre 1950 movies. I'm really looking forward to have some fun with this vapoursynth and your code. I'm not very strong and I will be slow, though. Thanks again.

[johnmeyer]

Which movie is it? It looks like Ronald Colman. I know lots of sources for old movies, and a huge number have gone through a similar restoration process that I use for the movie transfers I do with my own equipment. If I know the title, I might be able to help find a better copy, or at least one that hasn't been re-rendered several times. I can guarantee that this is NOT how it looked when released by the studio.

[johnmeyer]

P.S. Do download and look at the video and let me know if you think it looks any better.

Kinda noisy and fuzzy, ain't it?

Compared to the original? Not at all. All the field issues are fixed, and much of the mosquito noise is gone.

I did say, in my earlier post where I provided my script, that I did not apply any sharpening, which could easily get rid of any "softness" that you perceive (the buttons on the jacket would probably look better with a little sharpening). Also, I made no attempt to deal with the issues on the flat areas, like the walls. That problem is akin to contouring, and although technically not that problem, it is something that stems from the same issue of insufficient bit depth. I can only do so much on a ten second clip for a stranger, for no money ...

However, I did completely fix, on every frame, the original "interlacing artifacts" that were the topic of the thread.

[Randombit]

This one is The Charge of the Light Brigade.

[LMotlow]

You know, this is a movie from 1936, there really is not much hope I could get my hands on a better copy.

Eh? The Charge Of The Light Brigade is a classic that's been in print and in high demand for years. I have a much better copy I recorded on my old DVD recorder off TCM in 2002, as well as the retail DVD. I don't know how people watch downloaded crap. http://shop.tcm.com/the-charge-of-the-light-brigade/888574513436

Which movie is it? It looks like Ronald Colman.

I think you mean Errol Flynn.
Michael Curtiz did a bangup job mounting that movie. And no computers back then, either.

[Randombit]

Shop, yeah. I'm not a whiner, and $20 used to be nothing to think about for me, but times have changed. 2016 was the worst year in my life. Cancer, disability, Louisiana flood took my home. I'm about $60k short to restore my house. I'm rebuilding it myself, but the progress is very slow as I'm real weak from cancer treatment. I simply cannot justify for myself spending $20 on this kind of entertainment.

[LMotlow]

Yep, that's a rough row to hoe. I'm on chemo myself. But, lordy, those downloads are a nightmare. The websites have certainly done a job changing the definition of "digital quality". It usually means digital garbage. Anyway, hope something better comes along.

[johnmeyer]

Which movie is it? It looks like Ronald Colman.

I think you mean Errol Flynn.
Michael Curtiz did a bangup job mounting that movie. And no computers back then, either.

Hey, that's cheating: you already knew it was Charge of the Light Brigade.

I have access to a good copy. I'll look at it and PM in a week or so if I have something to report.

[Randombit]

Thanks! (I can open FTP port in my router for upload.)