Quality obtainable in PAL/NTSC conversion for DVDs.

[bugster]

@opoman. As I said earlier in this thread, I have not used avisynth but have often intended to try it out simply to see what all the fuss is about from those that swear by it. So I dowloaded and installed 2.55 and started reading and experimenting. My source file was an mpeg ripped from a DVD recorded on a Phillip DVDr75. It was a computer animated program (tripping the rift) so it is not quite the same test as a Rugby game but it was all I had. After lots of reading and some experimenting I came up with this script for PAL to NTSC conversion.

MPEG2Source("J:\Movies\DVD\Tripping\tripping.d2v") 
SeparateFields() 
BilinearResize(720,240)
SelectEvery(8, 0,1, 2,3,2, 5,4, 7,6,7)
Weave()
AssumeFPS (29.97)

lets take each line at a time.

1) Source file - nuff said
2) Seperate fields. The source is interlaced, so lets handle each field individually for the moment.
3) Resize to 720 * 240. Only 240 vertical as this is now working on each field, only half the final frame. (note to Lordsmurf!)
4) SelectEvery... I got this pattern from a page on the avisynth.org website for perfoming 3:2 pulldown (or is that 2:3 pulldown never sure about that!). I also found this page that helped me understand what this line of the script was doing.
5) re form the fields into frames
6) Set frame rate.

I used this to encode to NTSC with Tmpgenc and played the result on my TV. It looked ok to me and played smoothly to my eyes. Not sure how a rugby game would look but it might be worth a try.

[adam]

No, it did not move fields. The field moved.

At any rate, data does not change fields. Again, the whole "line 50", "line 55" thing implies some sort of non-existant rigid "place" for the interlace to exist.

This is just utter nonesense. I'm sorry, but you are making all of this up and yes, there is no other way to put it except that you are wrong.

Its not a matter of the "size of" interlace, or the "place" of interlace, or whatever other label you are suggesting we are attributing to interlacing. It is simple. The first scan line of Field A must be followed by the first scan line of Field B, or vice versa depending on field order. If it doesn't, then the fields are not plaiying in the order that they were sampled, and this means the temporal order is wrong.

If you resize this image, lets say to double its resolution, then the first scan line of Field A has now been interpolated into two distinct scan lines. How else do you make 1 row of pixels fill the area of 2? These scanlines now take up the first two sequential lines of your whole image. See what happens now? The FIRST line is physically stored in Field A and the SECOND line is now stored in Field B. All of this data was originally only in Field A, but now it is split between Field A and B.

The fields are not "moving," we're not dealing with silly putty here. This is digital data and to change the resolution means to resample the image. To increase it by a factor of 4x you have to create 4 new and distinct pixels which approximate the initial single pixel. Do an extreme zoom into any still image and you can clearly see this. What was once a single pixel of red is now a 40 by 40 block of pixels of various shades of reds and browns. It is in all respects a new digital representation of our image, and it is now being spread over a series of scan lines as opposed to just being contained on one.

When you resize interlaced footage horizontally this is fine because the newly interpolated pixels are still contained on the same scan line. When you resize vertically the newly interpolated pixels are on different scan lines, and thus some will remain in that field and others will be displayed in the following field. You ARE moving temporal information from one field to another and this WILL destroy the temporal order. This has nothing to do with framerate.

[opoman]

Thanks Bugster, I am currently trying to sort out the most enormous jumble round me and in my computer files - I have gone at this thing lickety split and need to spend some time getting order into the surrounding chaos.

I am currently tidying up a comparison of the results of scripts by groyal, dphirschler and yourself.

I will return.

[junkmalle]

Bugster, I think that pulldown pattern

SelectEvery(8, 0,1, 2,3,2, 5,4, 7,6,7)

is wrong for an interlaced video source. You are sometimes moving an earlier field to a later position (eg, 2,3,2). It would work fine for a film source though.

One can also use a more complicated pulldown pattern to go directly from 50 fields per second to 60 fields per second. I've seen this pattern in a few european TV movies broadcast on NTSC TV: 2:3:2:2:3 which works pretty well. I think the correct AVISynth command is:

SelectEvery(10, 0,1, 2,3,3, 4,5, 6,7, 8,9,9)

[lordsmurf]

You guys are just getting silly with this. You keep saying "interlace has a size" regardless of how you think you're not saying it. That IS what you're saying, like it or not.

I give up. Do what you want. You're wrong, but so many people on this board seem to be fine with passing around bogus info and living in a bubble.

I guess every tv station and editor in the world has been doing it wrong according to your made-up resize/crop rules.

When I get some time, I'll put up some ISO files for you to burn onto RW discs.

In the meantime, have fun running from the interlace boogeyman.

[adam]

I give up. Do what you want. You're wrong, but so many people on this board seem to be fine with passing around bogus info and living in a bubble.

[groyal]

I have to congratulate you for being such a stalwart helper in the face of some pretty fierce fire.

I appreciate that, but mad dogs and Englishmen go out in the midday sun.

I've made a frame by frame comparison of a few fast action frames between the non-interlaced and the final result from Pulldown. I can't see anything which gives a clue.

As junkmalle said, 3:2 pulldown introduces visible picture judder, but it's very subtle. You have to be right in front of the tube and know what you're looking for to see it, and you'd probably have a hard time convincing someone who wasn't aware of it that it's there. Certainly at an average viewing distance it's imperceptible.

But it's also an artifact of the 23.976-to-29.97 frame rate change, so it should be visible in the pulled-down MPEG but not the AVS script, especially when played back on a PC monitor. Have you tried playing them full-screen through MediaPlayer, and comparing those renditions to what you see on the television?

I'm skeptical of the suggestion that throwing away one of the fields is all that harmful to the picture, but it's an easy theory to test. I would create a second version of the AVS and TMPGenc template, i.e.

LanczosResize(720,576)
AssumeFPS(25,false)

and encode it as progressive PAL,

Video
Size: 720 x 576
Frame rate: 25
Video format: PAL

and compare the way it looks on TV. If the field/frame rate decimation is part of the problem, the result should exhibit a similar defect.

[opoman]

Right now, having spent time tidying up the mess made by the initial storm, and having become a little more familiar with all the new bits of software and concepts, I have three scripts - named groyal, dphirschler and bugster. They've all reached the stage where I am about burn them to DVD-RW for testing via my TV recorder/player. I will also examine how they play on the computer and will make the comparisons that have been suggested. I hope to find time to really absorb the 'theory' debate. I've come a long way in less than a week, starting from total ignorance of all the issues surrounding this topic, but I am concerned that I am going way too slowly to properly fuel the debate.

But I'll do my best - at least until Sunday when real life takes over and I have to go away for a few days.

One aspect of the tests is worrying me to death at the moment and that is that since I shortened the clip, to cut down process times, I get the following message from DGIndex - 'Opening GOP is not closed. The first few frames may not be decoded correctly' I've checked the clip in TMPGEnc DVDAuthor and it indicates that the start of the original VOB is suitable for the injection of new material. VirtualDubMod shows the first frame to be I followed by BBP BBP BBP BB before restarting the GOP sequence. This looks to me to be as it should be but I still worry.

Groyal - The 'shaking' effect that I reported during rapid movement is not *that* subtle - you certainly don't need to closely approach the screen to be aware of it as a watcher - and you'd certainly get fed up with it if you wanted to just watch the game.

Don't give up on me!

[bugster]

You guys are just getting silly with this. You keep saying "interlace has a size" regardless of how you think you're not saying it. That IS what you're saying, like it or not.

I give up. Do what you want. You're wrong, but so many people on this board seem to be fine with passing around bogus info and living in a bubble.

I guess every tv station and editor in the world has been doing it wrong according to your made-up resize/crop rules.

When I get some time, I'll put up some ISO files for you to burn onto RW discs.

In the meantime, have fun running from the interlace boogeyman.

If you are so knowledgable, why don't you provide opoman with an an avisynth script that will convert interlaced PAL to NTSC and prove us all wrong. Unlike some, I am happy to learn and to be proved wrong, but so far all you have done is continually shouted 'wrong' without explaining what is right, other than saying you let your encoders do the work for you.

[lordsmurf]

Because AVISYNTH is part of the problem.

And our discussion to date was resize/cropping. We removed the FPS part.

Changing framerates is a temporal adjustment, artifacts and errors are inevitable. Moreso in software. Use hardware for best results.

[opoman]

Because AVISYNTH is part of the problem.

Use hardware for best results.

I would find it helpful if you, lordsmurf, would explain how hardware does it. I'd always imagined that the hardware processed algorithms just like my computer does.

[lordsmurf]

You'd have to find a device and read the manual and/or contact that company. They all work differently.

[dphirschler]

lordsmurf, I examined you sample image and did a few tests. First and foremost, your sample image is flawed. It appears to be an interlaced image that was sized down.

Your test image.

Observe that every twentieth (or so) line is repeated. This will result in screwed up image upon output to TV.

Your two fields look like this (resized horizontally to preserve aspect)

I don't understand how you think this is acceptable.

I corrected your image as best I could by removing the doubled scan lines.

Now the scan lines are in the proper order. Here is how it would appear with the fields split (again resized horizontally to preserve aspect).


Finanly, I deinterlaced, resized, and re-interlaced. This is the only way to properly resize interlaced material.


Here is how it looks with the fields split (again resized horizontally to preserve aspect).

Each field is in its correct place and the temporal order is not screwed up. There is no other way.

Darryl

[lordsmurf]

I don't know what you did there, but that's some kind of nonsense "example".

[bugster]

I don't know what you did there,

You are making that more obvious with every post!

but that's some kind of nonsense "example".

nonsense or not, it looks a lot better than your example!

[junkmalle]

I examined you sample image and did a few tests...

Nice work. Obviously someone who knows what he's doing. How did you fix Smurf's original "interlaced" picture?

[dphirschler]

It was a pain. I manually removed every extra scan line in Paint Shop Pro. It was like so: start a section box just under the doubled line and extend it all the way to the bottom of the image, then move the selection up one pixel. Deselect and do it again on the next doubled line. There were about 25 doubled lines that had to come out.

I am certain the aspect got screwed up but a) it didn't matter for this example, and b) I don't know the original aspect. It was obviously resized before I got ahold of it.


Darryl

[lordsmurf]

Like I said, when I get time, I'll post some ISO's for everybody, along with explanation of how results were achieved. It won't be anytime this month, most likely.

[junkmalle]

It was a pain. I manually removed every extra scan line in Paint Shop Pro. It was like so: start a section box just under the doubled line and extend it all the way to the bottom of the image, then move the selection up one pixel. Deselect and do it again on the next doubled line. There were about 25 doubled lines that had to come out.

I was afraid of that. You have a lot of patience!

[bugster]

Bugster, I think that pulldown pattern

SelectEvery(8, 0,1, 2,3,2, 5,4, 7,6,7)

is wrong for an interlaced video source. You are sometimes moving an earlier field to a later position (eg, 2,3,2). It would work fine for a film source though.

One can also use a more complicated pulldown pattern to go directly from 50 fields per second to 60 fields per second. I've seen this pattern in a few european TV movies broadcast on NTSC TV: 2:3:2:2:3 which works pretty well. I think the correct AVISynth command is:

SelectEvery(10, 0,1, 2,3,3, 4,5, 6,7, 8,9,9)

I think you are right. I re-did my encode with your suggested pattern and it did look a bit better. I also took a closer look at the properties of the two results, the one using the pattern I suggested was longer than the source by approx 1%, so something else was wrong too which I don't yet understand.

[groyal]

Groyal - The 'shaking' effect that I reported during rapid movement is not *that* subtle - you certainly don't need to closely approach the screen to be aware of it as a watcher - and you'd certainly get fed up with it if you wanted to just watch the game.

Right, I gather the effect you're seeing is anything but subtle. You're familiar with FTP, right? Would you mind sending me a copy of the VOB so I can duplicate the experiment? Check your PM.

Don't give up on me!

Not at all.

[junkmalle]

Groyal - The 'shaking' effect that I reported during rapid movement is not *that* subtle - you certainly don't need to closely approach the screen to be aware of it as a watcher - and you'd certainly get fed up with it if you wanted to just watch the game.

Right, I gather the effect you're seeing is anything but subtle. You're familiar with FTP, right? Would you mind sending me a copy of the VOB so I can duplicate the experiment? Check your PM.

Any chance this could be posted were we all could see it?
<edit>
Oh, I think you're talking about the source. I wanted to see the "jumpy" final result.

[dphirschler]

Try this:

function pal2ntsc_3(v)
{
	v= assumeBFF(v)
	v= bob(v)
	v= v.bilinearresize(720,480)
	v= v.SelectEvery(10, 0,1, 2,3,3, 4,5, 6,7, 8,9,9)
	v= v.separatefields.selectevery(2, 1)
	v= weave(v)

	return v.assumefps(29.970)
}

This one gives an interlaced output. Pretty smooth too.

Or this gives a progressive output at ~30fps. No sence going ~24fps if you can do ~30fps.

function pal2ntsc_film(v)
{
	v= assumeBFF(v)
	v= bob(v)
	v= v.bilinearresize(720,480)
	v= v.SelectEvery(10, 0,1, 2,3,3, 4,5, 6,7, 8,9,9)
	v= v.selectevery(2, 1)
	v= weave(v)

	return v.assumefps(29.970)
}

Darryl

[opoman]

Here are the results I've obtained so far.

I've made three versions of the conversion, according to the recipes of bugster, groyal and dphershler. I've also done the conversion comparison test suggested by groyal.

I've spent about two hours watching the various versions of the 3 minute clip, played with my Pioneer 5100 recorder/player and also in a 35 uk pounds Yamada player. I found no difference in the results with the two players.

Its remarkable how easy it is to decide if the clip is comfortable to watch - or not - and how hard it is to convert the subjective response into statements about what is happening, in detail, on the screen.

In recognition of that fact, I am going to get a friend, who is rather good at sorting the subjective from the factual detail, to watch the four clips tomorrow morning. He will also be relatively free from the 'fault paranoia' from which I am definitely suffering after hours of normal watching and close scrutiny.

However, conscious of how overdue is my feedback, I'll give you my own observations and will adjust them, as necessary, after my friend has added his conclusions.

I observed two distinct effects, both of them quite subtle, the first distinctly irritating.


1) A steady, very short 'blink'effect, at close to 2Hz.

2) A rapid jitter, at tens of Hz, which occurred round the periphery of players' images when they moved rapidly across the field of view. Particularly noticeable in close-up and when viewed out of the corner of one's eye (peripheral vision has faster response and is more sensitive to flicker).

One other effect, a marked break up in images moving rapidly across the field of view, may simply be a more extreme example of 2)

Now for the four tests:

Bugster's: This was suggested on (page 3) Feb 2nd timed at 21:46. This was the least acceptable to view. The low frequency blink was present and there was marked stuttering when the camera was swung round and during rapid zooming. Slight stuttering was frequently observed throughout the clip.

Groyal's: Page 2 timed at 01:25 on Feb 2nd. This was the most acceptable to view. The steady blinking effect was absent and the impression was of smooth motion most of the time. The player silhouettes showed some slight rapid jittering, almost too fast to detect and some slight ghosting. The jitter was easier to detect when I viewed the screen from the corner of my eye.

Dphirschler's: Page 1, Feb 1st, 14:32. Some peripheral blurring, slight blink effect, jitter very noticeable when close up figure passed across the field of view. Quite watchable.

Groyal's suggested test conversion: P3, Feb 3rd 05:37. Steady pulsing blink, particularly strong when camera being swung round. Otherwise the picture was quite good. The marked difference between the test and Groyal's original suggestion was the presence of the pulsing blink in the test conversion.

That is as much as I can make of it. I hope that my friend can tighten up the observations - it will be a help to discuss it with another ex-scientist.

Thus, the best is not bad but I'd like to improve on it if possible.

Now, the matter of sending out copies of the rugby game clip. I asked for advice on how to do this at 10:00 on Feb 2nd (page 2) but have had no forum-published suggestions. The VOB-size is about 125 Meg. I am happy to spend a few pounds on sending a limited number of CDRs to people who give me an address to send to. I have broadband at 1Meg but a 1 gig limit per month on what I download - I presume that's an upload limit too, but I'm not sure. One solution may be for me to ftp it to one person who could then pass it on to one more and so on. Please forgive my naivete on this matter.

More when my friend has seen the outputs.

[junkmalle]

www.yousendit.com is designed for emailing files up to 1GB but the files aren't really emailed, a message is emailed and the file sits on their server for a while. When you post the file they give you a link to retrieve it. You can post that link here and everyone will be able to download it.

I don't think you need to send all 125 MB. A 10 second segment with the worst bits will probably do. Others may have a different opinion on this.

[opoman]

Yes, I'd like to shorten the clip but the situation is pretty complex. To see all the effects, and the way they change with scene and conversion script, you need to see the effect of player movement, fast and slow with still camera, zooming, and camera-swing, and combinations of zoom and camera-swing - its complex, I'm afraid.

An expert would know immediately what to keep and what to leave out but, by now, you know I'm not an expert. I could still edit out bits but this would take hours and I just can't spare that at the moment because life outside this project has piled up this week and I have to go away on Sunday. When I return on Tuesday, perhaps I'll have time to do this.

I'm sorry.

A word or two more about the clip. It was recorded at what the Pioneer Manual refers to as 'Standard Play.' Doesn't sound to me like a technical standard but maybe it is. They also refer to it as MN21.

SP was my choice for recording rugby even though Extended Play, MN1, serves fine for movies (I'm really not picky) and LP (MN9) for other sports. Rugby really is the toughest test of DVD and it can make a fool of digital where VC was perfectly acceptable.

Thus, I've had to repeatedly remind myself of the limitations in the original clip by playing it along with the tests of the three (plus one) scripts I've referred to above.

After sleeping on it, I regret that I didn't summarise the scripts at the top of my report last night. When I update, as promised, after my friend has helped out with assessment, I'll put the whole thing up as a comparison table, with the scripts included.

In response to Groyal's request for ftp upload, I've explained to him that I want to make the original clip generally available - I'd love to learn how you all get on with it - but my broadband usage is limited to 1 gig per month so I suggested a round robin scheme. However, uploading to the site you mentioned will be even better but should I wait for views on whether the full clip is acceptable? - I don't want to waste the broadband time.

[midnightsun]

You can't use assumeframebased if your source is actually interlaced. If you want to go this route you need to run an actual deinterlacer. FieldDeinterlace works well, SmoothDeinterlace is probably better.

It took me some time to get the grasp of the assumeframebased and assumefieldbased functions in avisynth.
Basically if you import an interlaced source and DON'T separate the fields you have to specify assumeFRAMEbased (or don't specify anything and hope avisynth gets it right, I can't recall off the top of my head how it works by default), meaning that every frame that avisynth sees is an actual frame and not one of the fields that have been separated.

Once you separate the fields you have doubled the number of "frames" that avisynth sees and that's when assumefieldbased comes into play: to tell avisynth that each frame is in reality a field coming from a frame whose fields have been separated. I think I remember reading that you don't need assumefieldbased after a separatefields command since it's obvious that the fields got separated by that and avisynth knows what it has to deal with.

[opoman]

junkmalle suggested an access site for the clip, yousendit.com.

After 45 minutes of waiting for my upload to complete, I got round to checking the upload speed - 17kB/s - uggghhhh.

I'm afraid I stopped it there and then so we will have to go with groyal's suggestion - I will ftp it to him, hopefully more quickly(!) and then leave it to him to distribute/edit as he sees appropriate.

[opoman]

The ftp transfer to groyal has been completed, successfully as indicated at my end. It took just over an hour to complete.

[dphirschler]

Let me know when/where to get it. Hopefully today because I am on dialup at home and will be there all weekend.


Darryl