I need help with removing two weird image anomalies.

[Freelancer]

Hello

My source is 5mbit MPEG-2, 25fps interlaced PAL. I am attempting to convert the source to 2mbit XviD using VirtualDub. After standard deinterlacing via blending both fields, there are still two anomalies present. Note that these anomalies are present in the source itself. Changing the output codec won't make any difference.
But what puzzles me the most is that both unwanted effects are not always present in the very same 20-minute video clip. Most of the time, the image seems to be OK:

Sample: OK



But sometimes, this strange (annoying!) vertical effect is present:

Unknown effect sample 1
Unknown effect sample 2
Unknown effect sample 3
Unknown effect sample 4


And sometimes, certain areas of the image are displaced. This seems to be the most visible on faces.

Displacement sample 1
Displacement sample 2
Displacement sample 3
Displacement sample 4

So first of all, can someone identify these effects? I don't even know what to look for. The vertical effect seems to be somehow connected to interlace...? I've tried several methods but it won't go away. The quick fix is 2:1 reduction, but that will ruin the image clarity . Smoothing seems to eliminate the displacement... but unfortunatelly, image details are gone as well .
Any feedback is appreciated!

Regards,
Freelancer

[Freelancer]

Doesn't anyone have some idea what the problem could be?

[Gazorgan]

M'kay, I'll take a stab.

What you have there is a PAL DVD of what was originally a 29.97 fps NTSC Video. Simpsons DVDs are what, seasons 1-3? That makes the video 10-13 years old. I do not believe there is a digital master (pregressive) of those seasons.

Personally it looks like a 29.97 NTSC-to-PAL issue. The source isn't that high a quality. Anyone have these in NTSC and can look at similiar scenes? There is a huge difference between season 1 and season 10 in terms of quality. I believe sometime after the 2nd or 3rd season they changed animation companies (after they were a hit).

[flaninacupboard]

the logo on the "OK" sample suggests it's not from DVD
The errors in set one of your images looks to me like dot crawl. this is related to the way your TV/capture card seperates brightness from colour. there's plenty of info available online about this. basically, don't capture with a composite or RF cable. if the source (i.e. TV station) has composite only video, there's not a lot you can do.
The second set of errors looks like animation. i wouldn't worry about them.

Gazorgan,
A progressive form of the show will exist somewhere, afterall they don't draw interlaced footage by hand
I assume they animate at 24fps, telecline to 29.97fps, and probably put that through a standards convertor to get 25fps. i'd much rather they just sped the 24 up to 25 (and i think this is what the bbc do, compared to Sky's interlaced setup) but that's just me being a PAL fanboy

[Freelancer]

First of all the source is not from DVD, I captured it via digital satellite. This particular episode is from season 13.
Could something be wrong with the video being transmitted itself? I've captured other shows and movies from different stations (widescreen and 4:3) and there were no such problems! There is some noise present in this clip, which must be somehow connected to the original video being transmitted. Some 4:3 sources have this noise, 16:9 sources don't have a single spot on them...
Anyway what I need to know is if this problem is solvable by editing or post-processing the video? Again, thanks for the feedback.

[Freelancer]

I've had some more time today and I took a closer look at the clip. It seems that the "vertical effect" is only present during the intro. I still have no idea what it is. The episode and the outro are OK.
The displacement... It's not always present, and sometimes it's very slight... Could it be a part of the cartoon? Though this seems far-fetched.

edit:
If anyone feels like checking this out: Here is a zip file (2,20MB) containing screenshots of every frame of a short scene (less than two seconds ). The displacement comes and goes. I wonder if this is encoding or cartoon related.

[SaSi]

I don't feel I'm quite helping here, but, for the life of me, I can't see anything wrong with the frames. I've opened every one of them. And then I re-opened them. I cannot find any displacement effect or vertical effect.

Now, I know I'm not blind and I also know we are both sane people.

There are some "artifacts" with interlaced video that when played on the computer screen appear rubbish but on the TV screen show picture perfect. I'd suggest that before you spend any more time on this, record the footage on a rewritable and view it on the TV. If it still displays bad, go back to investigating.

Of course, this doesn't explain why I can't see a problem on the frames.

[flaninacupboard]

Sasi, check "unknown image 1" and look at the top of the cloud just aboe the "N" it has a serrated look. i'm pretty sure that it's dot crawl.
Freelancer, how did you capture? RF? Composite? S-Video?
As for the banding round the characters, if it were some kind of noise/ghosting, then the band (especially around Marge) would not be so even and well defined. i thnk it's purposeful, either to look artistic, or to stop there being too high a colour contrast between characters and nearby objects (a cheap way of hiding, you guessed it, dot crawl!)

[Freelancer]

SaSi:
In the first set of pictures (unknown effect) there is considerable jagging on the letters and clouds and later on on objects.
I guess you were looking at the pictures of the judge when you said there's nothing wrong with them. Here's to Vulcan wisdom. I guess I wanted my clips to look "too perfect" and then I found a problem where there never was one . I'm beggining to believe that the displacement is artistic.

flaninacupboard:
Upon further inspection of several simpsons' clips I have established that the "dot crawl" is only preset during the intro (from the clouds to (and including) the living room scene).
I capture using RF. Nothing else will fit on my DVB-S card anyway
I believe that the problem is not me, but the station that is transmitting. In all other movies or series I've captured, there was never such dot crawl. If there was, I never noticed any. But this particular one can be spotted from a mile away And every simpsons episode they transmit has this effect. Only during intro. Is there any way to tone it down besides blurring?

[flaninacupboard]

Ah, so you're not really capturing, it's a digital transmission into your computer. This leads me to believe that fox, when putting together the newer series of simpsons, simply used an old composite copy of the titles to stick on at the beggining: hence dot crawl present in the credits and no where else. You won't get dot crawl by capturing in this way, as the transmitted program is MPEG already.
Blurring it would work, but is there any point? that's how the episode looks.

[Freelancer]

Ah, so you're not really capturing, it's a digital transmission into your computer.

Exactly! Sorry that I was not able to express myself more clearly. It's just that I'm not very familiar with all these terms

This leads me to believe that fox, when putting together the newer series of simpsons, simply used an old composite copy of the titles to stick on at the beggining: hence dot crawl present in the credits and no where else. You won't get dot crawl by capturing in this way, as the transmitted program is MPEG already.

I agree and I did suspect the source

Blurring it would work, but is there any point? that's how the episode looks.

Yeah, you're right . It's just that I've never seen a Simpsons episode with such dot crawl in the credits. But then again, the episodes I've seen in the past weren't very detailed (except for DVDs). All the popular VCDs have too poor a quality for dot crawl to be seen.
Unless there's a way to get rid of this without too much loss to the quality, I'll just leave it alone.
It's just me... Whenever video editing is concerned I always want things to look too perfect
Thanks for all the help. At least now I know what dot crawl is and when not to make a problem when there isn't one!


All I need to do now is to convert the source. I was thinking of XviD at first since I use a computer to decode, but I might need DVD player compatibility in the future so I leaning towards XSVCD. Something like 800kbps min, 2750 kbps average, 4000 kbps max (TMPGENC settings). With these settings I should be able to squeeze 2 eps onto a 90 min CD.
What do you think? Also I'm don't know how good DVD players tolerate 90min+ CDs. I've been hearing that some no-so-good 90+min CDs will stop playing at about 87-88 minutes?

[SaSi]

SaSi:
In the first set of pictures (unknown effect) there is considerable jagging on the letters and clouds and later on on objects.
I guess you were looking at the pictures of the judge when you said there's nothing wrong with them. Here's to Vulcan wisdom. I guess I wanted my clips to look "too perfect" and then I found a problem where there never was one . I'm beggining to believe that the displacement is artistic.

flaninacupboard:
Upon further inspection of several simpsons' clips I have established that the "dot crawl" is only preset during the intro (from the clouds to (and including) the living room scene).
I capture using RF. Nothing else will fit on my DVB-S card anyway
I believe that the problem is not me, but the station that is transmitting. In all other movies or series I've captured, there was never such dot crawl. If there was, I never noticed any. But this particular one can be spotted from a mile away And every simpsons episode they transmit has this effect. Only during intro. Is there any way to tone it down besides blurring?

Well, on second view (third actually) you are right. There are some microscopic displacements of pixels in the vertical axis. Very hard to see, after all they are like single pixel size.

Well, at 2:00 am in the morning at 40 degrees celcious outside temperature, I guess I shouldn't trust my eyes too much.

The displacements are very interesting. They appear like the interlace artifacts, only in the wrong axis.

Did you check on the TV? Do they show ?

[Freelancer]

Well, at 2:00 am in the morning at 40 degrees celcious outside temperature, I guess I shouldn't trust my eyes too much.

40 degrees? Man, that's hot Where exactly from southern Europe are you from? I'm from Slovenia and I'm enjoying a chilly 15°C right now (just after midnight). I just love this lousy rainy weather. A few weeks ago there was an entire week with temperatures at about 35°C - which isn't much - and I thought I was going to die already

The displacements are very interesting. They appear like the interlace artifacts, only in the wrong axis.

Exactly! I was extremely confused by them. At first I was searhing for info on vertical deinterlacing and I found nothing (figures). Then I tried all the deinterlacing or de-telecining techniques I could find.... no success at all... well apparently it's no interlacing, but dot crawl. I guess there's a first time for everything.

Did you check on the TV? Do they show ?

Well my TV isn't exactly state of the art, but it adequate . It's much less noticeable, but still noticeable (especially if you know what to look for ). But it kind of blends in, you know... on the monitor it just stands out. Yea, they're very noticeable on the monitor. Those screenshots I gave you were taken at the original 720×576 resolution. You have to look closely to really notice. But when I go full screen it just stands out
Since it's only during the credits and not throughout the whole clip I won't even worry about it. Mysterious are the ways of digital video...

[flaninacupboard]

Freelancer,
I'f you're converting them to MPEG but not keeping them 720X576, i'd reccomend 352X576. The slighty lower horizontal resolution than 480X576 SVCD will make them slightly better quality, and if you ever get a DVD burner you'd be able to use the video at 352X576 without re-encoding, wheras 480X576 needs a re-encode. in this way you could squeeze about 11eps per DVD-R. the resolution 352X576 is often referred to as 1/2-D1 (half D1) or CVD resolution. go look CVD on the site, there's plenty of info about...

[Freelancer]

Thanks for the info. I've done some testing and I'll be encoding with CCE using multipass VBR. It gives MUCH better results than TMGENC for my desired filesize. There's just one problem though . I need to process the video before I encode it with CCE (since cce doesn't offer much editing ).
Anyway - I need to deinterlace, clip the video and then resize it.
Now there are several options...
The biggest problem is that CCE won't open MPG files!

I could frameserve using DVD2AVI and avisynth, thus saving space and time. There's clipping... but nothing else

Virtualdub. I've tried frameserving but it will serve my MPEG-2 file in MPG format? So CCE won't open it... The only option left is to output to a codec that isn't more lossy than mpeg. Huffyuy comes to mind...
PRO: I can do it the way I want it and it doesn't take much time.
CON: A 20 minute clip will waste almost 10 gigs. Ouch!



I either need:
- A near lossless codec that won't take as much space as huffyuy... what about DV? Or MJPEG? Are those any good?
OR
- A different way to do it altogether
OR
- a way to frameserve with virtualdub in avi format (without re-encoding the mpg clip of course because then it would be the same as option 1)
OR
- a kick in the CCE's author's ass for making such a great MPEG encoder without the option to open MPEG files!





edit:
Well after fiddling with various techniques, I've decided to do it with virtualdub & huffyuy. It gives the best results. The only partial alternative would be to first edit & encode @ 10mbit CBR in TMGPENC and then frameserve with DVD2AVI to CCE. However with TMPGENC I can't edit the thing as well as with virtualdub. I've tried lossless MJPEG but it takes up even more space than huffyuy. So unless anyone has a better idea, VirtualDub & Huffyuy it is! I'll be a little short on HDD space, but I can manage.