Help me undestand - Same Videos, one PAL one NTSC, same length?

[Killer3737]

Ok i know the title is a bit cryptic but i couldn't really think of something short and appropriate. Basically the situation is thus:

I have two videos of the same thing. They are both identical in length in terms of the content. One is PAL recorded from SD (so 720x576, 25fps). The other is a HD recording from NTSC (so 1920x1080, 29.97fps).

The runtime of the PAL SD version is 43:27.
The runtime of the NTSC HD stream is 43:50. It has slightly less of the adverts trimmed out, so a tiny bit more content.

Now last night i watched the PAL video and played back the audio from the NTSC HD source over it, and they synced up perfectly, apart from the slightly longer advert cuts. I was doing this to check they did have the same content, which it seemed they did. I actually thought the HD file was PAL originally, but it is not.

Thing is, if i were to downsize the HD video to PAL (720x576, 25fps) rather than NTSC (720x480, 29.97fps), the video would come out at 52:33 (because of the lower fps), and therefore i would have to stretch the audio to that length aswell, but if i keep it at NTSC, it would still be 43:50.

So how can two videos with the same length in terms of content be the same time length in PAL and NTSC video format, but if i made them both PAL, one would be much longer, despite the same content?

I'm just going to convert to an NTSC DVD, but i still would like to understand this as it has me confused.

Thanks

EDIT: If it helps, here are the MediaInfo results on the two files:

HD:

General #0
Complete name : C:\Documents and Settings\Simon\My Documents\05-11-05\Foo Fighters Live 1080i DD5.1\Foo Fighters Live 1080i DD5.1.ts
Format : MPEG-2 Program
Format/Family : MPEG-2
File size : 5.33 GiB
PlayTime : 43mn 50s
Bit rate : 17 Mbps

Video #0
Codec : MPEG-2 Video
Codec/Family : MPEG-V
Codec profile : Main@High
Codec settings, Matrix : Standard
PlayTime : 43mn 50s
Bit rate mode : CBR
Bit rate : 16 Mbps
Nominal bit rate : 39 Mbps
Width : 1920 pixels
Height : 1080 pixels
Display Aspect ratio : 16/9
Frame rate : 29.970 fps
Chroma : 4:2:0
Interlacement : Bottom Field First

Audio #0
Codec : AC3
PlayTime : 43mn 50s
Bit rate mode : CBR
Bit rate : 384 Kbps
Channel(s) : 2 channels
Channel positions : L R
Sampling rate : 48 KHz

And one of the VOB's from the PAL DVD:

General #0
Complete name : D:\VIDEO_TS\VTS_01_1.VOB
Format : MPEG-2 Program
Format/Family : MPEG-2
File size : 1024 MiB
PlayTime : 9mn 31s
Bit rate : 15 Mbps

Video #0
Codec : MPEG-2 Video
Codec/Family : MPEG-V
Codec profile : Main@Main
Codec settings, Matrix : Standard
PlayTime : 9mn 31s
Bit rate mode : CBR
Bit rate : 14 Mbps
Nominal bit rate : 9500 Kbps
Width : 720 pixels
Height : 576 pixels
Display Aspect ratio : 4/3
Frame rate : 25.000 fps
Standard : PAL
Chroma : 4:2:0
Interlacement : Top Field First

Audio #0
Codec : MPEG-1 Audio layer 2
PlayTime : 9mn 31s
Bit rate mode : CBR
Bit rate : 384 Kbps
Channel(s) : 2 channels
Sampling rate : 48 KHz
Resolution : 16 bits

[jagabo]

There are many ways you can perform PAL/NTSC conversions. Some involve throwing away or duplicating frames or fields (often including blending what's left to smooth things out) to change the frame rate. These can maintain the running time of the original.

The following post contains an AviSynth script that converts interlaced NTSC to interlaced PAL without changing the running time:

https://forum.videohelp.com/topic353257-90.html#1868682

And an explanation of the rate changes:

https://forum.videohelp.com/topic353257-90.html#1868868

[minidv2dvd]

if you make a video pal from ntsc it's not just a matter of changing the fps from 29.97 to 25. that would make the video play in slow motion. what needs to happen is the number of frames has to be reduced and the fps lowered to 25.

then they play in the same amount of time and at the proper speed in both pal and ntsc.

[Killer3737]

Ok thanks i think i get that now. Just one more question, would converting the HD file to a PAL SD-DVD result in lower picture quality than just downconverting to an NTSC SD-DVD? Or would they look pretty much the same since it's HD in the first place?

[manono]

If the NTSC version was created from a PAL master (explaining the similar lengths), then almost certainly field-blending was involved, and the PAL source will have much better quality, no matter what you do to the NTSC one. To be sure, though a sample would be necessary.

[edDV]

1080i/29.97 -> 480i/29.97 or 480p/59.94 is just a field resize. Converting frame rates from 29.97i to 25i ideally requires field interpolation. This will be lossy. Deletion of fields will cause jerky motion.

[Killer3737]

If the NTSC version was created from a PAL master (explaining the similar lengths), then almost certainly field-blending was involved, and the PAL source will have much better quality, no matter what you do to the NTSC one. To be sure, though a sample would be necessary.

The video was shot in HD in Denmark, so i assume it would be PAL, but because the PAL source i have is just from standard cable to DVDR the picture quality is not very sharp and clear, whilst the HD version obviously is.

I've already converted the HD file to PAL 720x576 and it looks better than the original SD-PAL version, but i was just wondering if it would look any better if i just kept it at NTSC.

[manono]

I've already converted the HD file to PAL 720x576

I'd be curious as to how you did that.

i was just wondering if it would look any better if i just kept it at NTSC.

Like I said, an untouched sample of the source would be necessary. Just a few seconds would be enough.

[Killer3737]

I've already converted the HD file to PAL 720x576

I'd be curious as to how you did that.

I actually did it accidentally because i forget to check the file, i wrongly thought it was PAL. I just used a super simple AVIsynth script w/ CCE SP2. I deleted it once i realised the source was actually NTSC.

MPEG2Source("xxxxxx.d2v")
LanczosResize(720,576)
ConvertToYUY2()

I keep trying to find better scripts for HD > SD conversion using AVISynth and CCE SP2 but i can't find anything.

i was just wondering if it would look any better if i just kept it at NTSC.

Like I said, an untouched sample of the source would be necessary. Just a few seconds would be enough.

Ok sure, here is a 16 second sample of the HD source, cut and demuxed with DGIndex (video only):

http://www.mediafire.com/?z2xyzjmnpak

As you can see it doesn't actually look that good in HD (macro blocking) because it's from the MHD channel in 2005 which i gather were very hit and miss with their HD broadcasts and still are.

[manono]

Yeah, it got field-blended in the conversion from PAL to NTSC. You can separate the fields or (better) put on a smart bobber and check for yourself. You should see blends/ghosts/double images in the fields. But that explains the similar lengths. They used a PAL master for the NTSC video, used the original untouched audio, and blended the heck out of it to go from 25 to 29.97fps. This is a fairly common practice, unfortunately, as it saves the expense of preparing a proper NTSC master.

Interestingly (for me), it seems to unblend better to 29.97fps, while I was expecting 25fps, although I couldn't get it to unblend properly to any framerate. Maybe the PAL version was shot as interlaced video, which would explain it, or they performed some other mumbo-jumbo I don't quite understand.

If you plan on converting it to NTSC DVD, I'd still use the PAL source (assuming it's not blended in some way), resize for 720x480, encode at 25fps (unchecking the "For DVD" box in CCE), and then run the resulting MPV through DGPulldown set for 25->29.97fps. That'll make it compliant for NTSC DVD, keep it the same length as the original PAL source, and you can use the original untouched audio. Thanks for the sample. It was interesting.

Also, if you decide to use the Hi-Def NTSC source for the standard-def NTSC DVD, you'll want to convert to Rec.601 from the BT.709 source (as seen in DGIndex). Load the ColorMatrix.dll and add this to the script:

ColorMatrix(mode="Rec.709->Rec.601")

That way the colors shouldn't get changed.

[Killer3737]

Yeah, it got field-blended in the conversion from PAL to NTSC. You can separate the fields or (better) put on a smart bobber and check for yourself. You should see blends/ghosts/double images in the fields. But that explains the similar lengths. They used a PAL master for the NTSC video, used the original untouched audio, and blended the heck out of it to go from 25 to 29.97fps. This is a fairly common practice, unfortunately, as it saves the expense of preparing a proper NTSC master.

Interestingly (for me), it seems to unblend better to 29.97fps, while I was expecting 25fps, although I couldn't get it to unblend properly to any framerate. Maybe the PAL version was shot as interlaced video, which would explain it, or they performed some other mumbo-jumbo I don't quite understand.

If you plan on converting it to NTSC DVD, I'd still use the PAL source (assuming it's not blended in some way), resize for 720x480, encode at 25fps (unchecking the "For DVD" box in CCE), and then run the resulting MPV through DGPulldown set for 25->29.97fps. That'll make it compliant for NTSC DVD, keep it the same length as the original PAL source, and you can use the original untouched audio. Thanks for the sample. It was interesting.

Also, if you decide to use the Hi-Def NTSC source for the standard-def NTSC DVD, you'll want to convert to Rec.601 from the BT.709 source (as seen in DGIndex). Load the ColorMatrix.dll and add this to the script:

ColorMatrix(mode="Rec.709->Rec.601")

That way the colors shouldn't get changed.

Thanks very much for all that information, interesting stuff. So what you're saying is i should output the source as a 720x480 25fps non-dvd compliant file, then use DG Pulldown to make it think it is 29.97fps? That would give me the best results, and still in it's original length?

I'm certain that whatever i down convert it two, NTSC or PAL, it will look better than the best pure SD version i have (for a start it's 16:9 in a 4:3 frame, windowboxed).

Thanks.

[manono]

So what you're saying is i should output the source as a 720x480 25fps non-dvd compliant file...

Yes, if you'd like to use the PAL DVD as a source and don't want to mess with the audio.

...then use DG Pulldown to make it think it is 29.97fps?

Hehe, it won't just think it's a 29.97fps video; at that point it will really be a 29.97fps NTSC DVD compliant video file.

That would give me the best results, and still in it's original length?

It'll definitely be in it's original length (if done properly). As for whether it's the best result, that may depend on whether or not it was created at 25fps and if the audio is in pitch, without the usual PAL speeded up and higher pitched audio. All indications point to it having been shot at 25fps, so the audio should be OK. If so, you'll probably want to keep it at the 25fps length.

I'm certain that whatever i down convert it two, NTSC or PAL, it will look better than the best pure SD version i have (for a start it's 16:9 in a 4:3 frame, windowboxed).

Am I misunderstanding you, or do you really have a third version of the same concert? You must really like these guys. Yeah, widescreen 4:3 videos really suck, especially if you have a big 16:9 TV on which to watch them.

[jagabo]

Ok sure, here is a 16 second sample of the HD source, cut and demuxed with DGIndex (video only):

http://www.mediafire.com/?z2xyzjmnpak

As you can see it doesn't actually look that good in HD (macro blocking)

Deblock()

[Killer3737]

So what you're saying is i should output the source as a 720x480 25fps non-dvd compliant file...

Yes, if you'd like to use the PAL DVD as a source and don't want to mess with the audio.

...then use DG Pulldown to make it think it is 29.97fps?

Hehe, it won't just think it's a 29.97fps video; at that point it will really be a 29.97fps NTSC DVD compliant video file.

That would give me the best results, and still in it's original length?

It'll definitely be in it's original length (if done properly). As for whether it's the best result, that may depend on whether or not it was created at 25fps and if the audio is in pitch, without the usual PAL speeded up and higher pitched audio. All indications point to it having been shot at 25fps, so the audio should be OK. If so, you'll probably want to keep it at the 25fps length.

I'm certain that whatever i down convert it two, NTSC or PAL, it will look better than the best pure SD version i have (for a start it's 16:9 in a 4:3 frame, windowboxed).

Am I misunderstanding you, or do you really have a third version of the same concert? You must really like these guys. Yeah, widescreen 4:3 videos really suck, especially if you have a big 16:9 TV on which to watch them.

Right thanks. No i just have two versions of the show, the SD PAL version and the HD NTSC version. The only reason i want to downconvert the HD version is to replace the SD version. If it was already a decent DVB>HDD 16:9 capture i wouldn't bother, but it was shown on MTV2 Europe, so it's not. Also yes, i do love Foo Fighters.

I'll try both options and post back here with the outcome.

Thanks very much for all of your help, it's greatly appreciated.

[Killer3737]

Sorry for the double post, but i just realised something. Am i misunderstanding here, or are you saying i should convert the 1080i HD 29.97 NTSC file to 25fps 720x480, then take it back to 720x480 NTSC. Wouldn't it be easier to keep it in NTSC all the time? Or am i missing something here in my tired state?

EDIT: Actually looking back i think i may have confused somewhere. I don't want to use the PAL source at all. I want to downconvert the HD NTSC version to replace it. I'm not sure if i made that clear or not, i'm thinking probably not.

[manono]

Am i misunderstanding here, or are you saying i should convert the 1080i HD 29.97 NTSC file to 25fps 720x480, then take it back to 720x480 NTSC.

I was saying start with the PAL 25fps DVD and convert it to NTSC by resizing to 720x480 and encoding at 25fps. But if you just said that the PAL DVD is widecreen 4:3, then I might change my mind about the recommendation. I was assuming it to be 16:9. And if you do decide to use the PAL version as a source, and if you decide to reencode as 16:9, then you do more than just resize. You crop 72 from top and bottom and then resize to 720x480. I think that's how it goes.

If you're going to start with the Hi-Def blended NTSC version, then you either have to do an interlaced resize (which is tricky if you don't know how), or make it progressive by either unblending it or deinterlacing it, at which point there's no problem resizing. Based on what I saw of the NTSC video, I'd do my best to unblend it to 29.97fps progressive, resize for DVD and encode. You have several choices here and I can't make them for you. And maybe others have other ideas.

I'm not sure if i made that clear or not, i'm thinking probably not.

I'm only now realizing that's what you want. Probably my fault because I hate blended stuff so much. That comes from working with it too many times.

[Killer3737]

Ok so as a first test i converted the HD file to 720x480 29.97fps with no other advance settings in the script. That gave me a file at the right length but i guess because i didn't try to deblend it or deinterlace it, it has horizontal lines in the picture. Hard to describe, so i'll upload a sample in a hour or so.

So, can you or someone else suggest what i should add to my avisnyth script for deblending or deinterlacing to avoid this problem? I just used the one i posted earlier but with (720,480) of course.

[manono]

Hard to describe, so i'll upload a sample in a hour or so.

No need. I know what improperly resized interlaced material looks like. If you want to keep it interlaced, here's a script requiring only a Smart Bobber as a 3rd party filter:

AssumeTFF() #if TFF
LeakKernelBob(Order=1) #or your favorite bobber
LanczosResize(720,480)
ChangeFPS(59.94)
AssumeTFF()
SeparateFields()
SelectEvery(4,0,3)
Weave()
ConvertToYUY2(Interlaced=True) #for CCE

If you wish to unblend it as best as possible (keeping in mind it'll be improved greatly, but not perfectly unblended), I use MRestore:

LoadPlugin("C:\Path\To\DGDecode.dll")
LoadPlugin("C:\Path\To\MT_MaskTools.dll")
LoadCPlugin("C:\Path\To\Yadif.dll")
LoadPlugin("C:\Path\To\TIVTC.dll")
MPEG2Source("C:\Path\To\Video.d2v")
Import("C:\Path\To\MRestore.avs")
Yadif(Mode=1,Order=1) #or your favorite bobber
MRestore(Numr=1,Denm=2) #for 29.97
LanczosResize(720x480)
ConvertToYUY2() #for CCE

MRestore is part of the R_Pack which you can get here:

http://forum.doom9.org/showthread.php?t=95924&highlight=R_Pack

[Killer3737]

Thanks very much, i'll try those over the next two nights (it's a 14 hour encode on my PC). IS there any settings i should adjust within CCE apart from the obvious?

Thanks again.

EDIT: Oh just one more thing, when i run the TS through DGIndex to create my .d2v file, at the end i get the message:

A field order transition was detected. It is not possible to automatically decide whether this should be corrected.

Should i be correcting that or not? This is what it the fix says it's doing:

D2V Fix Output

Field order transition: 2 -> 0
800 1 0 1446299648 0 0 0 32 92 b2 a2
d00 1 0 1446467584 0 0 0 90 a0 a0
corrected...
800 1 0 1446299648 0 0 0 32 92 b2 a3
d00 1 0 1446467584 0 0 0 90 a0 a0

Field order transition: 0 -> 2
d00 1 0 4208896000 0 0 0 90
d00 1 0 4208953344 0 0 0 92 a2 a2 a2 a2
corrected...
d00 1 0 4208896000 0 0 0 91
d00 1 0 4208953344 0 0 0 92 a2 a2 a2 a2

[manono]

IS there any settings i should adjust within CCE apart from the obvious?

I don't know which script you're using or which CCE settings you're using, so I don't know what should be adjusted. And I'm not much interested, either. Maybe someone else is. I'm just here to help with the original subject of this thread, and then with the scripting.

Should i be correcting that or not?

I don't know for sure. If it's places where you cut out stuff, then yes, I think, especially if you decide to keep it interlaced.

[Killer3737]

Apologies for so many questions, i'm sure i'm annoying by now, but i just tried to use the unblending script you posted above and load it with CCE SP2. If in the options i just leave it at 29.97fps it tells me the runtime of the output file is halved. However if i tick the "Rate conv" box just below it says the output is the same as the original, as it should be.



Now i'm not sure if i should be ticking the rate conversion box or is CCE just saying the output is halved because it isn't aware of what the script is doing? Just seems odd to need to tick rate conversion box.

I don't want to do a 24 hour+ encode and end up with a bad output. I swear once i've got this right i'll get out of your hair.

Thanks.

[manono]

First, open the script in VDub(Mod) and go File->File Information to find out the framerate. it's always a good idea to check out a script in VDub before sending it to the encoder.

Second, note that I made a mistake in the script and, indeed, the framerate is wrong. I screwed you up. Sorry. In the Yadif line, it should be:

Yadif(Mode=1,Order=1) #or your favorite bobber

I'll go back and change it in the original post so anyone else reading this in the future doesn't get tripped up. Thanks for bringing it up. You'd have been real mad if you'd done a 24 hour encode only to find you had a lousy jerky-playing video.

[Killer3737]

Ahh thanks for that. I never knew about loading the scripts with VirtualDub/Mod, i'll remember that for the future.

Thanks again for all your help on this, i am very grateful.