DePALifying TV Episodes

[koberulz]

I have a lot of TV series on PAL DVDs, which I've backed up with MakeMKV. Is there a way to bulk-dePALify them?

[DB83]

Not really my subject but I guess you mean a process whereby an original NTSC series has been 'converted' to PAL.


I doubt if such a process exists since I believe there are several ways to do that. And if one involves frame-blending that can not be reversed.

[jagabo]

I have a lot of TV series on PAL DVDs, which I've backed up with MakeMKV. Is there a way to bulk-dePALify them?

It can be done with a batch file, and AviSynth. But why bother? Any player will play the MVK files directly.

[koberulz]

But why bother?

PAL speedup.

[koberulz]

So to clarify yes, they're US shows and thus originally NTSC, but I'm in Australia so my DVDs are PAL conversions.

[DB83]

Well my understanding of 'PAL speed up' seems different to yours.


PAL speed up typically occurs for film sources captured to PAL. So the PAL is 4% shorter in run time than the original film. A properly converted NTSC source should have exactly the same run time on the PAL and, again, a properly converted PAL source back to NTSC will still have the same run time else the result will be faster playback and not slower.

[manono]

PAL speed up typically occurs for film sources captured to PAL.

That's what he's talking about.

But as you also mentioned earlier, there are several ways to convert film to PAL and some of them don't involve PAL speedup. So, first he has to study what he has before deciding whether or not to slow them down.

If there's a way to change them all at once I don't know. But he doesn't want to do that for the reason I mentioned above. One-by-one.

[johnmeyer]

I wonder if there is a container into which you can put the video that has a playback speed flag. With AVI files, all I have to do is patch that header, and I can get the video to play at any speed I want (AVIFrate is the name of the ancient utility/hack). Does MKV allow the same thing? I haven't used it so I don't know, but if it does, you could do the whole thing, for all your files, in a matter of seconds.

[koberulz]

I think it's safe to assume that if the first episode of a season of a TV show is sped up, the remaining 21 are also sped up.

Not sure how I'd be able to tell though?

[DB83]

Maybe I miss something (and one does not know the series and the production method).


But surely if the production was recorded on videotape and not to film the argument about 'speed up' fails.


But again, I have dvds of the same film on NTSC and on PAL. The run time may vary but on PAL playback I, personally, do not notice anything. But I do understand that if someone in the US plays a PAL disk he can notice the 'speed up'

[manono]

Not sure how I'd be able to tell though?

If purely progressive and 25fps, it's speeded up and both the video and audio have to be slowed to film speed.

If interlaced or field blended, most likely it's the same length as the film source (but not 100% of the time) and while you might want to make it progressive again, you wouldn't slow it down and the audio can be used as-is. If you can find the original lengths at IMDB or elsewhere, you can compare with yours to help determine if PAL speedup has been used.

While it's usually true that all episodes of a series would be done the same way, that's not always the case. Different years of a series might be released at different times, be converted by different companies, and be done in different ways.

[koberulz]

Years, maybe. But each season would be consistent within itself.

And even if there were differences, I'd still want to bulk-convert everything even if I had to manually check each episode first, rather than having to push the button over and over and over.

We're talking interlaced footage, but there's no film source as they're TV shows, and would have been shot on tape. Or digital, or whatever. Interlaced NTSC either way.

[DB83]

But just because they are TV shows do not assume there is no film.


Apart from Black Adder, I am a big Monty Python fan and much of that was shot on film.

[johns0]

Easy way to tell if a tv episode was speeded up for pal is to look up the ntsc version,if your pal version is 39 minutes long and the ntsc is 43 minutes then you know for sure.

[koberulz]

What sort of difference would there be over an hour of TV? And is there a reliable source for such things?

And...how do I actually fix it? Somehow we've not covered that.

[DB83]

manono covered your first Q. in reply #11. IMDB is like anything else on the net - not always accurate.


But there may also exist fansites or even dedicated forums for the series or topics in forums dedicated to tv series etc. . You need to look. Even then, the topics could only discuss the original NTSC which leaves you little the wiser about PAL conversions.


And your second Q. can only really be answered when you have a full understanding of the source. Right now you are making a broad assumption.

[jagabo]

I wonder if there is a container into which you can put the video that has a playback speed flag. With AVI files, all I have to do is patch that header, and I can get the video to play at any speed I want (AVIFrate is the name of the ancient utility/hack). Does MKV allow the same thing? I haven't used it so I don't know, but if it does, you could do the whole thing, for all your files, in a matter of seconds.

MKV has that. I believe MP4 has it too. But few players will obey the flags. Especially outside a computer. And with those player that do, they vary in how the audio is adjusted. Some keep the pitch, some don't.

[manono]

And...how do I actually fix it? Somehow we've not covered that.

That's because we've been trying to talk you out of doing them all the same way using some sort of a BAT file. We also assume you know basic AviSynth and none of the fixes are difficult.

If progressive 25fps, slow to film speed and slow the audio to match. If phase-shifted, a simple TFM followed by slowing to film speed and slowing the audio to match. If field-blended then a bobber followed by SRestore and the audio remains unchanged. If hard telecined (pretty rare), IVTC back to film speed and the audio stays unchanged.

If the source was truly interlaced before being converted to PAL, you'd have to figure out how it was converted before deciding what to do. As DB83 already mentioned, many many television shows are and were shot on film. The technical aspects of the show's creation are usually shown in IMDB.

[jagabo]

What sort of difference would there be over an hour of TV?

Modern "hour" long TV show are about 45 minutes. Older shows were a little longer (fewer ads were allowed back then), about 52.5 minutes. PAL speedup is a factor of 24000/1001/25, about 0.959. So a 45 minute show would be 43.155 minutes. But many TV shows would not go through a PAL speedup. Even if they were shot on film (and most were) most are simply put in a realtime analog NTSC/PAL converter that uses field blending and results in exactly the same running time. You'll have to analyze them individually.

And...how do I actually fix it? Somehow we've not covered that.

In AviSynth, to remove for PAL speedup:

WhateverSource() # 25p source
AssumeFPS(24000, 1001, sync_audio=true) # lowers audio pitch
ResampleAudio(48000) # back to 48KHz samples

For field blended videos:

WhateverSource() # 25i source
Yadif(mode=1, order=n) # or QTGMC
SRestore(frate=23.976)

There are other possibilities too.

[johnmeyer]

In this country, until "All in the Family" in the early 1970s, almost all TV shows were produced on movie film (usually 35mm) and then telecined. This is because none of them were live, and editing video was done on linear editors (i.e., deck to deck) and even "cuts-only" was difficult.

Actually, until digital HD became common 10-15 years ago, even shows up to the end of the 90s were often done on film.

Film was always 24 fps, and as everyone here knows, that was usually just sped up to 25 fps to show on PAL systems.

[koberulz]

Well as an example, episodes of Psych list at 43 minutes on IMDB and are 41 minutes on my hard drive. MediaInfo lists them as Progressive and Top Field First, which...makes no sense?

[manono]

It makes perfect sense They have PAL speedup. 2 minutes shorter or thereabouts.

[koberulz]

That makes sense, it's the progressive top field first that doesn't. Why would progressive footage have a field order?

What do I do to slow these down, bearing in mind they're MKV files created via MakeMKV? Bulk editing in AviSynth isn't something I know anything about. I'm guessing a CRF x264 encode into a new MKV file is the way to go? Something to drop each episode into a "Fixed Episodes" folder with the exact same file name as the original would be ideal.

[koberulz]

Actually...would FFMPEG be a better option? I'm not aware of bulk AviSynth editing being a thing, whereas that runs from the command line so a BAT file can handle it in theory (not that I know anything about BAT files). Have to worry about output formats and input formats and such too.

I know y'all are against the idea of bulk converting but a) the chances of one episode of season one being done differently to any other episode from season one are remote, even if there's a greater chance of the seasons differing from each other and b) even if I spot-check every episode it's still easier to put together the ones that require the same treatment, point a BAT file at them, and hit "go" rather than individually running hundreds of files through manually.

[jagabo]

That makes sense, it's the progressive top field first that doesn't. Why would progressive footage have a field order?

It makes perfect sense. The field order flag in an MPEG 2 stream is a single bit so it has to be either 1 or 0, TFF or BFF, regardless of how the video is encoded. MediaInfo is just reporting the value of the flag.

What do I do to slow these down, bearing in mind they're MKV files created via MakeMKV? Bulk editing in AviSynth isn't something I know anything about. I'm guessing a CRF x264 encode into a new MKV file is the way to go? Something to drop each episode into a "Fixed Episodes" folder with the exact same file name as the original would be ideal.

You use a batch file with a FOR loop to create the AviSynth scripts and execute ffmpeg to encode them. Here's an example of such a batch file:

for %%F in (*.ts) do (
echo LWLibavVideoSource("%%F"^) >script.avs
echo Yadif(order=1^) >>script.avs
echo crop(2, 0, 0, 0^) >>script.avs
echo Spline36Resize(720,404^) >>script.avs
ffmpeg -i script.avs "%%F.mp4"
del "%%F.lwi"
del script.avs
)

pause

It builds scripts like:

LWLibavVideoSource("ramp.ts") 
Yadif(order=1) 
crop(2, 0, 0, 0) 
Spline36Resize(720,404)

You'll have to change the batch file to build the scripts you need (see post #19) and specify the ffmpeg encoder settings you want.

[koberulz]

Is there an explanation of how all that works? I've been given batch scripts on here before but it's always just a copy-and-past-this-exact-code thing, and googling "%%F" and the like...doesn't return hugely helpful results.

I shouldn't need to resize should I?

[DB83]

^^ If you did any programing even 'basic' you would have come across a 'For Next Loop' and basically this is all it is.


The >> are 'pipes' which send the content created before the '>>' to the, in this case the script, file named after the '>>'

[koberulz]

Well, sure. But it starts with > and then goes to >> and I have no idea what "%%F" means, specifically, or how that bit of it works. It's clearly looping for every .ts file, but I'm after a more specific understanding.

[DB83]

We I think the first '>' is a typo and as you already state the %%F just means 'every file'.


I really do not know how much more specific one can get. Or just access the 'help' screen in the Windows Command Box.

[jagabo]

> creates the file and puts the text in it
>> appends the text to the existing file

Remove the "del script.avs" line to see the resulting AVS file -- for debugging your batch file.

Here's an more relevant example I came up with since then:

for %%F in (*.mkv) do (
echo LWLibavVideoSource("%%F"^) >script.avs
echo AssumeFPS(24000, 1001, sync_audio=true^) >>script.avs
echo ResampleAudio(48000^) >>script.avs
ffmpeg -i script.avs "%%F.new.mkv"
del "%%F.lwi"
del script.avs
)

pause

I'm not sure how well LWlibavVideoSource() works with MPEG 2 in MKV. I usually demux to an mpg/m2v file and use DgIndex/Mpeg2Source() when dealing with MPEG 2 video.