HDTV recordings: 1920x1088 vs. 1920x1080 lines.. [SOLVED]

[vhelp]

[SOLVED] -- I was using vfapi when I should have been using dgdecode/mpeg2source to process MPEG's.


I don't know, but after working with these HDTV sources put out by my Pinnacle PCTV Pro stick and its recording software, I"ve been trying to deal with an issue that I thought I raised a question on in previous discusisons elsewhere on this forum, but my memory is not at its best considering how swampped I've been over these 14 days of the Olympic games. Anyway.

The issue is with the decoding of the obtained PS MPEG-2 's from the PCTV Pro stick. I am using DGIndex (various vers) and after processing to a .d2v project, I am getting reported outputs of 1920x1088 even though (in some versions) the info details report 1920x1080 during processing. And when I export out to VFAPI or AVISynth script using the info() function, they each report with 1920x1088 pixel resolution. So it would seem that I do have those 8 extra lines of pixels.

I think my memory is trying to come back and tell me that someone did hint what the issue might be. But, after working with the these MPEG sources, I've been noticing that there's this gray bar at the bottom of the mpegs, but I dismissed it on account of being too swamped with deadlines and things. So, this evening, I decided to put some time into this problem and consult google, and this is what I found out:

--> grayline with mpeg2source by lexor; Dec 10, 2005

The hint in that discusion seems to lean toward it being the PS MPEG-2 decoder engine -- the one installed by the Pinnacle PCTV Pro stick, I think. If that be true then there should be an update from Pinnacle or else perhaps another mpeg decoder (hacked or other) might be available to fix this. However, in the link above, there is a tool posted that hacks the mpegs to fix this problem. I haven't actually tested it, nor have I read that discussion completely, (I will, after I finish this post) but I think it might work for the time being.

Download hack tool: fix1088.exe 132kb -- a command line tool.

Other HDTV cards might also suffer from this grayline at the bottom thing, and this tool might help those, as well.

-vhelp 4839

[jagabo]

Yes, a lot of devices produce 1088 lines (because it's mod16). Some decoders (CoreAVC for example) have the option of forcing them to 1080 lines.

[vhelp]

Yes, a lot of devices produce 1088 lines (because it's mod16). Some decoders (CoreAVC for example) have the option of forcing them to 1080 lines.

I did a quick what-if scenario and dropped one of the mpeg's (a small one) into virtualdub and it did report a 1920x1080 pixel resolution and no graybar in sight. So, it seems to be a problem with the software tools as they decode/process using mod16 algorthms, as would seem the case with the version dgindex I was using.

I ran a few calculations to see what common resolutions (height) would suffer from this issue (with these mod16-based tools) in the listing below:

30.00 --> 1.33AR --> 720x480 div 16
17.00 --> 2.35AR --> 640x272 div 16
22.50 --> 1.78AR -> 640x360 div 16
33.75 --> 1.78AR -> 960x540 div 16
45.00 --> 1.78AR --> 1280x720 div 16
67.50 --> 1.78AR -> 1920x1080 div 16

In an effort to test this mod16 issue, I threw in a few of my recently uploaded 640x360 mpeg clips of the games and I found that they too would suffer the mod16 error when I loaded it into dgindex. I have several versions but not the latest. So I'll assume that this issue has been fixed in later versions. I don't know the links to those so I couldn't test to see if it was fixed. However, loading them into virtualdub decodes the clips without the mod16 error. So, its good to know that this issue can be fixed in those tools.

-vhelp 4844

[jman98]

I record HDTV signals directly from my cable box via firewire and bypass all capture cards. Any 1080i video I have recorded is actually 1920x1088. The point is that it might not be the card doing this but it might be simply how broadcasters are sending the signal.

[Nelson37]

Also dealing with memory issues but I recall that this was a variable from the original broadcast. The patch solution was much like the "header trick" for resolution issues, some softwares ignore the 1088 value and some choke on it.

The main problem was that progs that did not like the number would re-encode when authoring, which was basically not needed.

[vhelp]

[UN-RESOLVED] -- you have to manually crop the pixels.

Hi guys.

I will search google to see if there is an update to dgindex and see if this issue has been cleared up. When it comes to mpeg videos, I much prefer to work inside dgindex on them. I suppose I could crop off the 8 pixels but, as simple a step that is, that would be one more step to the chain of events, and I already have too many at this point.

If I find an update, (and it resolves the issue) I'll post it up here for consistancy sake. Until then, see you around.

EDIT: Well, unfortunately, there were no fix.. cropping seems to be the only way of aleaviating the problem,
though that feature in dgindex was disabled, so you have to do it in post -- avisynth or virtualdub.

-vhelp 4846

[edDV]

I record HDTV signals directly from my cable box via firewire and bypass all capture cards. Any 1080i video I have recorded is actually 1920x1088. The point is that it might not be the card doing this but it might be simply how broadcasters are sending the signal.

Everything I use shows 1080 lines. How are you measuring?

[jman98]

Everything I use shows 1080 lines. How are you measuring?

Both the Teco Bit Rate Viewer and Ulead DVD MovieFactory 6 report my video as being 1920x1088. In fact, I first noticed the problem because Ulead kept insisting on re-encoding my video when trying to produce HD DVD even though I checked the option that said "Don't encode compliant sources". I suppose it could be a header issue where the header reports 1088 but the video is actually 1080. Not sure.

[MozartMan]

I record HDTV signals directly from my cable box via firewire and bypass all capture cards. Any 1080i video I have recorded is actually 1920x1088.

jman98,

I don't know what cable box you use, but I get 1920x1080i (and 1280x720 on some channels) files from Comcast Motorola DCT-3416. Something wrong with your setup.

Everything I use shows 1080 lines. How are you measuring?

Both the Teco Bit Rate Viewer and Ulead DVD MovieFactory 6 report my video as being 1920x1088.

Bit Reate Viewer doesn't work on HD .TS files.

Here is the guide on how to transfer video from cable box to PC in original quality via Firewire:

http://home.comcast.net/~exdeus/stbfirewire

[jman98]

I record HDTV signals directly from my cable box via firewire and bypass all capture cards. Any 1080i video I have recorded is actually 1920x1088.

jman98,

I don't know what cable box you use, but I get 1920x1080i (and 1280x720 on some channels) files from Comcast Motorola DCT-3416. Something wrong with your setup.

Everything I use shows 1080 lines. How are you measuring?

Both the Teco Bit Rate Viewer and Ulead DVD MovieFactory 6 report my video as being 1920x1088.

Bit Reate Viewer doesn't work on HD .TS files.

Here is the guide on how to transfer video from cable box to PC in original quality via Firewire:

http://home.comcast.net/~exdeus/stbfirewire

OK, first of all, I really don't care if you think something is "wrong in my setup". I'm just using standard drivers to capture via firewire, but you know what? I really don't care, not at all, that I have this problem because I can work around it.

Second, I never said I was feeing TS files into Teco. I edit with VideoReDo to remove commercials and save as MPEG-2 program streams (Ulead barfs on TS and frankly, it's just easier for me to work with PS).

Guys, I'm just reporting what I'm getting in my captures. That's all I'm doing. I'm not looking for "you don't know what you're doing" type posts. If you don't like what I said, that's YOUR problem. I'm just reporting what I'm getting and I can work around it and I'm not looking for help. The link is how I did my captures but again, I can work around it so I don't really care, not even a little, that I have this issue.

[MozartMan]

OK, first of all, I really don't care if you think something is "wrong in my setup". I'm just using standard drivers to capture via firewire, but you know what? I really don't care, not at all, that I have this problem because I can work around it.

Second, I never said I was feeing TS files into Teco. I edit with VideoReDo to remove commercials and save as MPEG-2 program streams (Ulead barfs on TS and frankly, it's just easier for me to work with PS).

Guys, I'm just reporting what I'm getting in my captures. That's all I'm doing. I'm not looking for "you don't know what you're doing" type posts. If you don't like what I said, that's YOUR problem. I'm just reporting what I'm getting and I can work around it and I'm not looking for help. The link is how I did my captures but again, I can work around it so I don't really care, not even a little, that I have this issue.

Oh man!

1. Nobody said "you don't know what you're doing".

2. If you really don't care why the hell do you post here?

3. Good luck to you man!

Adios.

[vhelp]

Actually, to settle this mod16 vs. whatever thing, one could review the source code to dgindex and see where it utilizes mod16 algorithms, but good luck in that, especially if it uses pre calculated table or equations as they are harder to follow, else worse case, the routine could be in libraries that dgindex calls. But, for those who are c++ savy, it might be worth a look-sees and settle this 1088 vs. 1080 problem.

-vhelp 4854

[Guest34343]

The latest version of DGIndex should handle this. It should pop up a window asking if you want to treat the video as 1080 instead of 1088.

If that is not happening, please post a link to an unprocessed source stream sample.

I know the DGIndex code pretty well, so I can give you a definitive answer if you give me a source stream.

[vhelp]

Been using various versions in the 1.4.x's range, then later searched google and found v1.5.2 which is what I've been using this past week, from this link:
--> http://neuron2.net/dgmpgdec/dgmpgdec.html

I must still have an older version. Thanks for looking into this problem over here

-vhelp 4857

[Guest34343]

1.5.2 is the latest version. Is it OK with 1.5.2?

If not, where is the sample?

[vhelp]

Good morning..

Sorry I haven't responded sooner. I"ve been away from my home computer, and my work computer is constantly being blocked for outside usage, including being able to upload -- rapidshare; megaupload; and a few others are no-no's for me, and they are even blocked from my view. Our employer is just too vigalent about these things. And, I can't find anything that is 10 MB or less in size that I might be willing to upload over dialup. So I'm in a pickel as to how to get you a test sample. For now, I'll try and answer your question in further details.

And I just wanted you to know that there is no urgency to have a resolution right away. Its just that I spotted this problem and thought I'd share it and see if there was a solution to it already. Thanks.

1.5.2 is the latest version. Is it OK with 1.5.2?

I have tried versions in the 1.4.8 range and 1.4.9x ranages, and the same problem. And after D/L'ing the 1.5.2 version, the same problem occurs, whether 640x360* or 1920x1080, they all output the same issue.
In dgindex, there are two ways I open mpegs.. by the clunky way of File\Open\.. or, by dragging them to the window. I prefer the drag way because of the way I work.

* the 640x360 are my own encodes from TMPGenc, not from the Pinnacles recording softare. So, it would seem that it is not tied to Pinnacle as I was getting around to. Maybe the two recording (mpeg) types of these are similar in attributes and are missing something in the specs or format that other mpegs (by other tools, etc) have and that DGIndex have no problems with.

The other thing I noticed -- maybe it will help you figure out whats going on. I find that whenever I do work with the 1080 sources from my pinnacle recording software, the bottom 8 pxel line is gray. But, when I work with my own encoded mpegs by TMPGenc, the botton 8 pixel line is black. Hmm...

Thanks for all the help on this matter, plus the enhancements you've made on dgindex

-vhelp 4858

[Guest34343]

I can't do anything without a sample. I just need a single GOP. 1-2MB should be fine. As long as it's enough to show a picture in DGIndex.

[jagabo]

I have version 1.5.0 RC5 of DgIndex and I don't see any "extra" data around non-Mod16 frames. I just tested a 480x360 encode with TMPGEnc Plus and DgIndex showed exactly 480x360 pixels. Does DgIndex rely on any external filters like MPEG splitters and decoders? Could those be a source of vhelp's problem?

I know that some image compression schemes have an encoded frame size and a "valid" frame size. So an image may be encoded with a frame size larger than the valid image data (so as not to have to special case the edges). Does MPEG have such a concept?

[Guest34343]

DGIndex is totally self contained.

Yes, MPEG can specify the coded size and the display size.

If vhelp would simply post a sample stream, we can easily figure out what is happening.

[vhelp]

wow.. I jumped up out of my bed (was napping on the problem) with an idea

@ jagabo I think we can all help each other out, actually. Since I can't upload files over dialup, would be willing to consider encoded (with TMPGenc) an video in 640x360 pixels and then upload it. You can test your's out with DGIndex, too, if you want. But if you upload this video, I can test it out on both my pc's and see if I get the same problem. If I do (and you don't) then we are both correct with our theory that it is a codec and/or libirary thing that is causing the confusion.

This would help me out because I can at least download a 10MB or so file in almost no time. My upload speeds is in the order 1.2kb times, and I would have white hair by then. I would be greatful, thank you.

. . .

Otherwise, here's what I have so far in my own trials..

Testing with DGIndex 1.5.2 using 640x360 pixel videos

I had been using 1.5.2 on my Windows XP Home edition computer and thought that maybe the issue is with that computer. But after several test encodes on my WIN98 computer, that is not the case there, either.
I tested with 1.4.3 and 1.4.9 etc., and they all produce the same thing on the win98 computer, too.

The following were based on a quick encode in TMPGenc as the 720x480 test source encoded back a few years ago from my Polaroid DRM-2001G dvd recorder over a 480i component output. So don't worry about imperfections you might see. Thanks.


* BEFORE: virtualdub output -- Crl+1 -- screen output capture



* AFTER: TMPGenc -> TMPGenc -> DGIndex 1.5.2 -> VFAPI -> virtualdub Ctrl+1 -> output -> .jpg

Thanks everyone

-vhelp 4863

[vhelp]

@ neuron2

I'm sorry I can't satisfy your requirements. I did try encoding a short clip, and thought great, I can finally get some answers, but the clips I fed into 1.5.2 would not completely encode to a .d2v file because of some GOP issue -- but it would process perfectly in older versions, even with 8 frames, and still demostrate the x8 pixel problem, using only a 210kb mpeg. Sounds pretty stupid, I know, but it worked. But you are only interested in the 1.5.2 aspects. Oh well. AT least I tried.

-vhelp 4864

[Guest34343]

Aha, seems to be limited to VFAPI.

I've duplicated it with VFAPI serving but not with Avisynth serving.

I'll see what I can do about it, but out of interest, why are you using VFAPI instead of an Avisynth script?

[Guest34343]

I did try encoding a short clip, and thought great, I can finally get some answers, but the clips I fed into 1.5.2 would not completely encode to a .d2v file because of some GOP issue

Can you give me more details about this? I can't fix things if you don't tell me about them!

[jagabo]

Here's a little 640x360 MPG file from TMPGEnc Plus:

macbeth.mpg

[vhelp]

Oh. Thanks jagabo.. for trying.


* BEFORE: macbeth.mpg -> DGIndex 1.5.2 -> VFAPI -> virtualdub Ctrl+1 -> output -> .jpg



* AFTER: DGindex -> macbeth.d2v -> avisynth -> virtualdub Ctrl+1 -> output -> .jpg

hmm.. it looks like my huntch was correct, (that the problem would still be there) and I was right.
Anyway. It seem that neuron2 hit on the problem right away -- vfapi.

Moving forward I will definately be using avisynth via "dgdecode.dll" / mpegsource() on these mpegs.

Still, I use vfapi for various projects and I would like to see a resolution. But I put no pressure on the developer to rush anything with respect to vfapi. I'm just glad that we solved the mystery behind the problem.

My thanks to everyone, I'm so happy again

-vhelp 4867

[jagabo]

I thought VFAPI had been abandoned. It looks like it hasn't been updated in six years.

[Guest34343]

Still, I use vfapi for various projects

You don't seem to like to answer direct questions.

Here, I'll try again. Why do you need to use VFAPI for these projects?

[jagabo]

Yes, MPEG can specify the coded size and the display size.

Is the coded size always mod16?

[Guest34343]

Yes, you always have to have an integral number of macroblocks.

[vhelp]

I'll see what I can do about it, but out of interest, why are you using VFAPI instead of an Avisynth script?

You know, I'm not gonna bullshit around the bush on this.. old habbits die hard

..and I'm still gonna use it. I'm not concirned with the impact in speed it has over avisynth. But given all the nonsense I went through with these HD sources, I have to admit that its time for a little bit of change in some of my workflow habbits. And I've already started on it, by including the two script functions for dealing with mpegs.

But don't get me wrong, I've been using avisynth for quite some time and in all kinds of video projects. Its just that with the recent broadcast of the Olympic games in HD, the old habbit thing got stompped on, and the rest you know is history

I hope I've (finally) answered your question, otherwise, thanks for putting up with me and my stuberness,
-vhelp 4868