MPEG-2 HDTV to SD-DVD - "best" method

[Killer3737]

Ok I know this has been covered several times, and there isn't really such thing as best when it comes to this, that's why I put it in quotes. What I want to try and get though, is a good template I can work from for all of my encodes.

At the moment, I do the following:

DGIndex (default settings) to extract audio file from .TS and generate D2V file
Create an AVISynth Scipt that I then feed...
Into Cinema Craft Encoder SP2

That is the basic method, and as far as i've found the "best" way to do it. Now where the variables come in, and what I'd like advice on.

- DGIndex - Are there any settings I should be adjusting in this before just creating the D2V file)
- The AviSnyth scripts - The big one really. What I should be putting into these scripts.
- Cinema Craft Encoder - - Is there a more up to date CCE guide than this one from 2005?

I also don't understand what in the AVISynth script is overpowering the settings in CCE. I've tried what seems like hundreds of different scripts that I've been given or advised to use, but I'd like to try and get a most optimum one that I can change with each video.

So for AVISynth scripts, all of my input videos are HDTV 1920x1080i in either PAL or NTSC, and either TFF or BFF. I don't think there are any other variables? Or does whether the source is film or video matter? I really don't know.

The most recent scripts I've used are these:

MPEG2Source("source", cpu=0)
ColorMatrix(mode="Rec.709->Rec.601")
assumetff().Bob(height=480)
LanczosResize(720,480)
assumetff().separateFields()
SelectEvery(4,0,3) #TFF source - use (4,1,2) for BFF
Weave()

MPEG2Source("source", cpu=0)
ColorMatrix ()
SmoothDeinterlace(lacethresh=10, tff=true, doublerate=true)
MSharpen ()
LanczosResize (720, 480)
Trim (1,0)
SeparateFields
SelectEvery (4,0,3)
Weave()

But I really don't know if they are good for what I want, which is basically the highest quality DVD I can get.

[poisondeathray]

So for AVISynth scripts, all of my input videos are HDTV 1920x1080i in either PAL or NTSC, and either TFF or BFF. I don't think there are any other variables? Or does whether the source is film or video matter? I really don't know.

Yes it depends on what the source is . For example, you wouldn't deinterlace film material

[Killer3737]

Ok obvious question, what program can I use to be sure of the source? I know DGIndex tells you as you create the .D2V file, but it seems to be just giving a rough guess with a percentage. Also you say you wouldn't deinterlace a film souce, do you mean in the script you wouldn't? I'd still output an interlaced file in the end?

[poisondeathray]

No tool does it perfectly, or can tell you what the source is; they all make mistakes. You have to examine it manually to be sure

Here is a mini-guide that explains how to do it
http://neuron2.net/faq.html#analysis

If it was a film source, and it's telecined, you would inverse telecine it (IVTC) , and your script would reflect that. A film source is progressive, so you would output progressive (encode a progressive DVD)

For interlaced sources, the only reason you are deinterlacing then reinterlacing, is that you can't resize an interlaced source with a straight resize (your fields get messed up and you get artifacts). Deinterlacing always reduces quality, but this is unavoidable if you want to resize an interlaced source

[Killer3737]

Ok right thanks, I'm beginning to understand why some of these operations are in the script now. Ok here is one example of video that I'm working with right now:

Video
ID : 224 (0xE0)
Menu ID : 1 (0x1)
Format : MPEG Video
Format version : Version 2
Format profile : Main@High
Format settings, BVOP : Yes
Format settings, Matrix : Default
Duration : 43mn 30s
Bit rate mode : Variable
Bit rate : 14.2 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 29.970 fps
Standard : Component
Resolution : 8 bits
Colorimetry : 4:2:0
Scan type : Interlaced
Scan order : Top Field First
Bits/(Pixel*Frame) : 0.229
Stream size : 4.32 GiB (91%)

Now i'll check it properly tomorrow with that link you've given, but DGIdex says it's 90% film, and let's assume for the purpose of me understanding it is, how would my script roughly look? Are you saying I'd output a progressive DVD with this, even though the source HD file from the film is Interlaced?

I guess I'm really just trying to understand what the things I'm changing in programs and adding in scripts are. I think I know most of it, but some is still a mystery, and I just do as i'm told basically.

[poisondeathray]

If it's 90% film , chances are it's film, but parts of it may be interlaced or cadence might be broken. Note you still can't be certain, until you examine the content manually. In that case, I would use DGIndex in "honor flags" mode, and use an adaptive field matcher, and then decimate (e.g. from TIVTC.dll package). This should return the 23.976 film rate (progressive). Since you're in the UK, you might do a PAL speedup, but that's another topic. Most euro spec players should be able to play NTSC content anyway. The colormatrix is for hd=>sd color coefficients

MPEG2Source("source.d2v")
TFM()
TDecimate()
LanczosResize(720,480)
ColorMatrix(mode="Rec.709->Rec.601", clamp=0)

Have a look at that link above, and all the docmentation and instructions that come with all the packages. Most of them have examples as well

[Killer3737]

Ok I've just created the script mentioned on that page for identifying the video, and loaded it into VirtualDub, but then I'm stuck, I don't know what I'm supposed to be looking at. What does this mean:

We will label a new picture with a new letter below. If you see a repeating field sequence.....

[poisondeathray]

when you step through the separated fields, each new letter represents a different picture

true interlaced would have a different picture each step, because each field is taken in a separate moment in time

true progressive would have repeats for that procedure, because each frame is taken from a separate moment in time; when you separate the fields, they both belong to the same frame. Note it won't be exactly the same - you'll notice a slight up & down motion between pairs.

The best way is to distinguish is to find a sequence with some horizontal panning or motion

The combinations listed there are the most common, but there are some bizarre combinations suchh as field blending, phase shifting. If you're not sure what you have, then post a sample

[Killer3737]

Hmm I don't know maybe I did something wrong, in VirtualDub with the script loaded all I see at the bottom is:

Frame 146650 (0:40:46.611) [K]

The K is present all the way through.

I tried to extract a 30 second sample with VideoReDo, but I got a 'Transport Muxer Buffer Underflow' error when I try to extract it. That usually happens when I have a bad .TS for whatever reason. A friend sent me this on a DVDR (without an MD5 or anything) so maybe I got a bad rip from the disc, i'll try again.

[poisondeathray]

Yes it sounds like you have errors. Did DGIndex throw some errors when you indexed it? You could also try with tsmuxer to cut segments

You could try projectx , tsdoctor, or pvastrumento to fix it , or even videoredo's quickstream fix

The "k" in vdub refers to keyframe , it has nothing to do with the "abc" field counting method in that link. Those letters are arbitrary letters assigned to distinct fields. You could have called them "dog" or "cat" or "goat".... I'm sure you get the idea.. the point is to distinguish if the fields are derived from different moments in time, or same moment in time.

[Killer3737]

Yes it sounds like you have errors. Did DGIndex throw some errors when you indexed it? You could also try with tsmuxer to cut segments

It indexed ok, but at the end it gave this error:

You could try projectx , tsdoctor, or pvastrumento to fix it , or even videoredo's quickstream fix

Thanks I'll give them a try.

Is there any 'secure' way to rip a data disc? I just drag and dropped the TS in explorer, then tried Recovery Toolbox for CD, and it stopped extracting (no error, just stopped as if it was complete) after about 700mb of the 5gb file, which must be where the problem part is?

The "k" in vdub refers to keyframe , it has nothing to do with the "abc" field counting method in that link. Those letters are arbitrary letters assigned to distinct fields. You could have called them "dog" or "cat" or "goat".... I'm sure you get the idea.. the point is to distinguish if the fields are derived from different moments in time, or same moment in time.

Yeah I'm misunderstanding what I'm doing here then obviously, I know the a b c was arbritary and just used to differentiate the different fields, It's how to distinguish those different fields I'm not understanding. I'll try and fix the file and post a sample.

Thanks for all the help so far by the way, much appreciaited. I've never been properly tought about anything like this, just tried to learn as I go along. obviously I could just do what most people do and stick the .TS into HDTV2DVD or something and be happy with that, but I want to do it properly, with the best quality.

[Killer3737]

I ran the file through MPEG2Repair, this is what it said:

MPEG2Repair: F:\HD VIDEO\FFStorytellers.ts


Sequence Frame 22600(3-P) / Time 0:15:08 :
VideoWarning: Unexpected TFF/RFF flag change in previous GOP.

Sequence Frame 28284(11-P) / Time 0:18:54 :
Error: Packet 11422763 has no TS Sync Byte.
VideoWarning: Discontinuity of (15+) packet(s). First packet ending at offset 2147480124
Additional error(s) detected. Increase VerboseLogLevel in INI file for details.
FileInfo: Last video errors span 629 bytes at file offset 2147479551
VideoWarning: Discontinuity of (8+) packet(s). First packet ending at offset 2147497608
AudioWarning: Timestamp gap of -0.224000 sec. ending at file offset 2147453520

Sequence Frame 28308(12-B) / Time 0:18:55 :
VideoWarning: Timestamp gap of 0.033367 sec. ending at file offset 2147089276
VideoWarning: Timestamp gap of 0.033367 sec. ending at file offset 2147648316
VideoWarning: Timestamp gap of 0.050044 sec. ending at file offset 2147568635
VideoWarning: Unexpected TFF/RFF flag change in previous GOP.
VideoWarning: Timestamp gap of 0.033367 sec. ending at file offset 2147389324
VideoWarning: Timestamp gap of 0.050044 sec. ending at file offset 2147303252
VideoWarning: Unexpected TFF/RFF flag change in previous GOP.
VideoWarning: Timestamp gap of 0.050044 sec. ending at file offset 2147455343
VideoWarning: Unexpected TFF/RFF flag change in previous GOP.

Sequence Frame 65768(6-B) / Time 0:43:30 :
VideoError: Invalid Huffman code in non-intra MPEG2 block. MBA=7315(1840,960)
Additional error(s) detected. Increase VerboseLogLevel in INI file for details.
FileInfo: Last video errors span 19 bytes at file offset 5124022441

Sequence Frame 65769(6-B) / Time 0:43:30 :
Info: End of MPEG2 sequence

Sequence Summary:

File Size Processed: 4.77 GB, Play Time: 00h:43m:30s
1920 x 1080, 29.97 fps (25.20 fps Telecine), 20.00 Mbps (14.72 Mbps Average).
Average Video Quality: 71.32 KB/Frame, 0.28 Bits/Pixel.
AC3 Audio: 3/2 Channels (L, C, R, SL, SR) + LFE, 48.0 kHz, 384 kbps.
Dialog Normalization: -23.0 dB, Center Mix Level: -3.0 dB, Surround Mix Level: -3.0 dB
2 of 65769 video frames found with errors.
0 of 81572 audio frames found with errors.
648 corrupted video bytes in file.
0.250233 seconds of video timestamp gaps.
0.224000 seconds of audio timestamp gaps.

End of Log

A lot of that is like another language to me although the bit i've bolded stands out, what does that indicate?

[netmask56]

ProjectX does a very good job of repairing TS files but it only does so in Demux mode.

[Killer3737]

Ok I went a little off topic with the last few posts, about one specific video. The intentions of my thread is deciding a 'best' way to encode generally, video sourced TFF/BFF HD files, and getting the best script (or atleast one that is easily modified for certain files). I just encoded one video with this script:

MPEG2Source("F:\Austin City Limits S35E14 Them Crooked Vultures 1080i HDTV DD5.1-CtrlHD\ThemCrookedVultures.d2v")
ColorMatrix(mode="Rec.709->Rec.601")
assumetff().Bob(height=480)
LanczosResize(720,480)
assumetff().separateFields()
SelectEvery(4,0,3) #TFF source - use (4,1,2) for BFF
Weave()
ConvertToYUY2(Interlaced=True)#if needed for your encoder

But the results were not very impressive to me, and there seemed to be an interlacing problem. So, imagining all my input is good quality MPEG2 1080i files, video sourced, and wanting to go to 480/576i, what would you guys be suggesting?

Obviously the following are needed no matter what:

MPEG2Source("F:\Austin City Limits S35E14 Them Crooked Vultures 1080i HDTV DD5.1-CtrlHD\ThemCrookedVultures.d2v")
ColorMatrix(mode="Rec.709->Rec.601")
LanczosResize(720,480)

But what else would you be adding to get the best possible output? Is CCE still the best thing to feed them into? I'm thinking about trying HCEnc.

Thanks