Something Is Wrong with My .ts File, But I Don't Know What (Interlacing?)

[koberulz]

When I watch the Blu-ray back it just doesn't quite look right. Couldn't put my finger on why though...just a shimmery sort of look to things that shouldn't be shimmering I guess? I had a look and it turns out Premiere had flagged it as progressive, even though it isn't, but fixing that didn't seem to improve things.

So, my process: Recorded on a DVR. The .ts file on the DVR's hard drive transferred to my PC. Import into Premiere. Remove commercials. Output as UtVideo YUV422 BT.709 VCM AVI. Convert to x264 via MeGUI. Author Blu-ray disc.

Super simple, works fine for other recordings that have to come via my Intensity Shuttle because cable isn't a fan of the whole being able to transfer the file thing.

Buuuuut...not here. And I don't know why. I suspect interlacing might be involved because it just doesn't look right in Premiere. Usually when I select "first field" or "second field" in the display, it interpolates to get to full-height so the result looks progressive. Not on this one. Here, we have massive amounts of combing in both fields. Which promptly goes away if I open the TS file in AviSynth, but that's complicated to do as a comparison because I have frame 35624 open in Premiere and trimmed to frame 35624 in my AviSynth script, and instead got frame 35607. Which is weird. One of the programs is clearly getting the frame numbers wrong. This strikes me as a bad sign.

I don't know how to cut a lossless sample of a TS file, so...help? Or, here's some MediaInfo stuff:

General
ID                                       : 800 (0x320)
Complete name                            : L:\g1.ts
Format                                   : BDAV
Format/Info                              : Blu-ray Video
File size                                : 13.1 GiB
Duration                                 : 5 h 39 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 5 520 kb/s
FileExtension_Invalid                    : m2ts ssif

Video
ID                                       : 166 (0xA6)
Menu ID                                  : 806 (0x326)
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Codec ID                                 : 27
Duration                                 : 5 h 39 min
Bit rate                                 : 5 099 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 25.000 FPS
Standard                                 : Component
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : MBAFF
Scan type, store method                  : Separated fields
Scan order                               : Top Field First
Bits/(Pixel*Frame)                       : 0.098
Stream size                              : 12.1 GiB (92%)
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 86 (0x56)
Menu ID                                  : 806 (0x326)
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 2
Codec ID                                 : 3
Duration                                 : 5 h 39 min
Bit rate mode                            : Constant
Bit rate                                 : 192 kb/s
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 kHz
Compression mode                         : Lossy
Delay relative to video                  : -496 ms
Stream size                              : 467 MiB (3%)

Text
ID                                       : 46 (0x2E)-801
Menu ID                                  : 806 (0x326)
Format                                   : Teletext Subtitle
Language                                 : English

Menu
ID                                       : 1036 (0x40C)
Menu ID                                  : 806 (0x326)
Duration                                 : 5 h 39 min
List                                     : 166 (0xA6) (AVC) / 86 (0x56) (MPEG Audio) / 46 (0x2E)-801 (Teletext Subtitle, en) / 7806 (0x1E7E) () / 7000 (0x1B58) ()
Language                                 :  /  / English
UTC 2019-08-16 00:00:41                  : en:Bosnian News / en:News via satellite from Radio-Televizija Bosne i Hercegovine, Bosnia. / news/current affairs /  / 00:29:53 / 
UTC 2019-08-16 00:30:34                  : en:Macedonian News / en:News via satellite from public broadcaster MRT in Skopje, in Macedonian, no subtitles. / news/current affairs /  / 00:30:53 /

I have no idea what the hell all that is at the end. Weird. It's significantly longer than that...so significantly that I had to trim it in order to fit under the character limit for this post, which is 50,000 characters. I was only over by 22,294. So...fun.

[sneaker]

I don't know how to cut a lossless sample of a TS file, so...help?

Does the problem show in the beginning? Then cut the first e.g. 100 MB using DGSplit.

[koberulz]

Not sure about playback but I just checked and the combing weirdness in Premiere doesn't happen at the start, no. And that's the tail end of the previous show, so it might be the broadcast rather than the file.

EDIT: Checked further along in the show I'm working with and that's fine too. So it's a temporary thing. Still have no idea why it's happening though.

It also takes (*&^*&^ing forever to load a preview in Premiere. It's fine at the start of the file and takes longer and longer as I get deeper into the show. Eventually takes literally minutes for the display to change.

[Sharc]

Where does this come from:

FileExtension_Invalid                    : m2ts ssif

ssif is usally found in 3D MVC Blu-rays

[koberulz]

What do you mean where does it come from?

[badyu17]

I have no idea what the hell all that is at the end. Weird. It's significantly longer than that...so significantly that I had to trim it in order to fit under the character limit for this post, which is 50,000 characters. I was only over by 22,294. So...fun.

Your DVR kept EPG data in your recorded file. It was part of your satellite/cable signal. Probably also wrongly remuxed original TS into BDAV M2TS which is what these cheap set top boxes do.

As for the file itself I always had problems with MBAFF with separated fields flavor. For example remuxing it with mkvtoolnix results in weird frame rate. I suggest that you remux TS into mkv/mp4 with ffmpeg and from then work with that file.

[Sharc]

I am just wondering why MediaInfo reports these extensions as invalid and where it found the ssif (normally used in 3D blu-ray structures).

[smrpix]

It also takes (*&^*&^ing forever to load a preview in Premiere. It's fine at the start of the file and takes longer and longer as I get deeper into the show. Eventually takes literally minutes for the display to change.

Your program is more than five and a half hours long. Premiere has to create the audio waveform for preview, so it's not unexpected it will be slow until it's finished doing that.

Does the shimmering go away if you reverse the field order?

[koberulz]

That process finished literally days ago. And it wouldn't explain why the deeper I go the slower it gets, nor would it explain multi-minute load times for frames. I'm usually able to work just fine while it's processing audio.

The shimmering effect is only visible when watching on TV, I'm not sure what it looks like elsewhere, so I'd have to burn a whole new disc with reversed fields. The fact that there's combing in both fields in Premiere, but neither field in VDub, seems like a more obvious starting point.

[koberulz]

Tried uploading a UtVideo sample, but the server timed out. Pretty sure that happened to me last time I tried to upload a sample here as well. Is something broken?

[poisondeathray]

Tried uploading a UtVideo sample, but the server timed out. Pretty sure that happened to me last time I tried to upload a sample here as well. Is something broken?

I notice this happens with firefox but not chrome . Chrome seems to upload ok to this site

[koberulz]

Weird. Does indeed work fine in Chrome.

Disabling either field in Premiere actually does seem to make a lot of the combing go away, but it sticks around near the shoulders in the first frame.

[poisondeathray]

The Y is interlaced 50 fields/s ;

But the U,V looks like 25 fields/s (I.e it's progressive chroma content , but also with some combing). Basically you're missing 1/2 the temporal color information

You might be able to interpolate / reconstruct those missing chroma fields


But check your process first, can you upload a TS sample, or as close to original source as possible from the DVR ? Maybe PP mishandled it

[koberulz]

So why is it showing up in some areas but not others? Because it seems to go away later in the broadcast, but I can't be 100% sure that it isn't just one of the non-problem areas.

I loaded the TS file into AviSynth, then VDub, then bobbed it and got the same result so between that and it not being an issue after the first few minutes or in the tail-end of the previous show also captured on this recording it does seem like a source issue.

[poisondeathray]

So why is it showing up in some areas but not others? Because it seems to go away later in the broadcast, but I can't be 100% sure that it isn't just one of the non-problem areas.

No idea

Could be broadcast issue, equipment issue, overheating etc.... just wild guesses

[koberulz]

Right, but why do the players' legs look fine but their shoulders are combed, for example? Is there a way I could look at some frames from later on and know whether it was a good frame, or just that the issue wasn't visible?

[poisondeathray]

Right, but why do the players' legs look fine but their shoulders are combed, for example? Is there a way I could look at some frames from later on and know whether it was a good frame, or just that the issue wasn't visible?

probably because of "red" and saturation

chroma issues are more easily seen with "red" and higher saturation

[jagabo]

There's combing in the chroma channels but it's not normal. It's pairs of lines rather than every other line. But I suspect this is from error in the TS to AVI conversion. Stop f'ing around and just upload a segment of the TS file.

[koberulz]

I don't know how to.

[jagabo]

AviDemux, H264TS_Cutter, ffmpeg... Transport streams are designed to be picked up any time during a broadcast. Cuts don't even need to be at header or frame boundaries. So they can be cut with any blind file splitter like a HJSplit or a hex editor.

An ffmpeg example:

ffmpeg.exe -y -i source.ts -ss 00:00:30.00 -t 00:00:10.00 -codec copy -y cut.ts

That will extract a 10 second long segment starting at 30 seconds into the video. It does not cut on a keyframe so the first few seconds may not play properly. That doesn't matter here -- just make sure the segment is long enough to span several GOPs.

[koberulz]

Here.

[poisondeathray]

But this section looks ok in terms of that issue . Post a "bad" section.

[koberulz]

Weird. That's the same segment. Here's the bit I posted the AVI from.

[jagabo]

Both clips look fine. Deinterlacer's are going to have problems with that large scrolling video display in the background of the clip in post #21. But that's not an encoding error in the source, it's just what happens when you have thin horizontal lines in interlaced video. One field (SeparateFields() in AviSynth):


[Attachment 50485 - Click to enlarge]

[poisondeathray]

Yes, both clips are ok in terms of that issue

Possibly your version of PP is mishandling it because it works ok here

If you couldn't get avisynth to work properly in that segment, the culprit is usually the source filter

(I think you're just trying to make Team Canada look bad ... not that they need any help looking bad ...)

[jagabo]

FYI, I used LWLibavVideoSource().

[poisondeathray]

FYI, I used LWLibavVideoSource().

Same here. LSmash in general is much better at handling transport streams (with the exception of DGSource)

He's using an older version of PP IIRC , that's the likely culprit for that botched stream

[netmask56]

I ran the news bulletin through TSDoctor and came out the other end OK! I then used Vidcoder to make a mp4. That worked as well. You could just load the TS file as is into Vidcoder for conversion to MP4 if that is the preferred format. TSDoctor has the advantage of cleaning up transmission errors if any..

[koberulz]

Yeah, I've been using this:

LWLibavVideoSource("g1.ts")
AssumeTFF()
Trim(35624,35624)

Which gets me this:



If I then apply the VDub bob doubler filter, set to "TFF" and "Bob", I get the combing issue:



However if I use this, and remove the VDub filter:

LWLibavVideoSource("g1.ts")
AssumeTFF()
Trim(35624,35624)
Bob()

I get this, which is fine:

[jagabo]

VirtualDub doesn't handle interlaced YV12 properly. Convert to YUY2 (which VirtualDub does handle properly) and you'll see the video is fine with VirtualDub's Bob Doubler.

LWLibavVideoSource("g1.ts")
AssumeTFF()
Trim(35624,35624)
ConvertToYUY2(interlaced=true)

This is a longstanding bug in VirtualDub. Avery Lee refused to fix it, arguing instead that interlaced YV12 shouldn't use the same fourcc as progressive YV12.