Crop VP6 video using frame header?

[KeyMs92]

Hi,

I've been looking for ways to crop my VP6 video without loss of quality. The bottomline on almost every thread is that this can't be done. However, I found one thread which stated it was possible in some cases using the header:

Open windows calc, enter the height of the current file, and the set it to hex. Open virtualdub, and open tools->hex editor. Load your file in. Click on "strf" which will highlight the important chunk. (Not strl or strh.) Now look in the highlighted area of the left pane for the height, it'll probably start right underneath strf (the bytes will be backwards, but you should recognize it from what calc showed). Subtract 8 from the number (use calc if you aren't handy with hex) and replace the first byte - you probably won't need to touch the others. Save, and see if it worked.

http://forum.doom9.org/archive/index.php/t-119748.html

Unfortunately this didn't work for me. On most players the video looked the same after editing and on a few players only the aspect ratio was changed. However, in Vdub the video did look cropped.

This gave me a sparkle of hope it might be possible. I then found this in the VP6 documentation:

OutputVFragments. The vertical decoded height of the frame as it should be scaled on output in 8x8 block units. See definition of ScalingMode below.

ScalingMode. Internally a frame may be encoded at a different resolution to the eventual size that it is presented on output from the decoder. There are four ways to scale the frame on output MAINTAIN_ASPECT_RATIO, SCALE_TO_FIT, CENTER, OTHER.

http://multimedia.cx/mirror/vp6_format.pdf

This looks pretty interesting, although strangely I cannot find any of these properties in the Vdub Hex editor. These properties should be part of the I-frames, in the 'intra frame header'. Unfortunately I haven't been able to find such a header... Any idea?

[El Heggunte]

Hmmm, that's interesting.

However, in Vdub the video did look cropped

Possibly it's because VirtualDub uses the VfW version of the VP6 codec.

As for the "other" players you tried... are they using the "official" DirectShow decoder by On2 (vp6dec.ax) ?
If "yes", then the fault is on the filter itself --- if "no", then the fault is on the "alternative decoder" that's being used.

[KeyMs92]

Possibly it's because VirtualDub uses the VfW version of the VP6 codec.

That's what I thought! No, the other players are not using the official decoder. I actually have no idea how to get them to work with the official one. I believe they use the 'unofficial' libavcodec. I've had a decoder issue before, which is mentioned in this thread:

http://forum.doom9.org/archive/index.php/t-151123.html

In a way, it doesn't matter because I upload most of them to Youtube, which would decode it incorrectly anyway... I'm hoping there's a way to crop the video that is picked up by the unofficial decoders out there.

[El Heggunte]

FWIW, MPlayer supports both the VfW and the DirectShow VP6 codecs.

Not sure if this will be helpful though.

Hummm, I didn't know that the reverse-engineering of the On2 codecs by the FFMPEG/LIBAV teams is a half-assed job
Just for the record, their RealVideo decoders still suck a lot , if I am not mistaken.

[KeyMs92]

Hummm, I didn't know that the reverse-engineering of the On2 codecs by the FFMPEG/LIBAV teams is a half-assed job

It seems like it yeah :P

But getting back to the second point in my first post: Do you know if and where I can found the properties listed in the documentation?

[El Heggunte]

You'd better read again the Section #9 (page 23) of the file vp6_format.pdf

And probably you'll need an ACTUAL hex-editor (XVI32 for example), so that:

a) you could view and edit the bits inside each byte, AND ...

b) ... perform a precise "Replace All" operation

C) Possibly you will find out that the open-sourced VP6 decompressors are even more b0rked than you thought :–/

[KeyMs92]

You'd better read again the Section #9 (page 23) of the file vp6_format.pdf

Right, I've played around with different hex editors for quite a while but I haven't been able to find something. Maybe I'm missing something but I've no idea to be sure where each property is located. At least the names of the properties and values are not mentioned in te bitstream.

[El Heggunte]

You've got to look for some, let's say, "repetition patterns"

( apologies if this sounds strange to your ears and eyes, my English is both non-native and somewhat weak )

It would be easier to identify them if you were dealing with a "well-marked" format like, for example, DTS:

http://wiki.multimedia.cx/index.php?title=DTS#Frame_format

However, according to what I managed to understand from the 'VP6 PDF', the "constant bytes" for each encoded frame may vary too much

I mean, the "magic byte sequence" for VP6-file-#1 will differ from the "m.b.s." for VP6-file-#2

So the zeroth problem to be solved in this case is, one has to determine what is the file-specific "fixed byte sequences" which identify the beginning of every VP6 video frame inside the container (AVI, FLV, whatever).

[KeyMs92]

You've got to look for some, let's say, "repetition patterns"

Easier said than done! :P

Well, I've attached the byte sequence file for ya, so you can have a look. I only took the first 125 frames (until the second key frame) with Vdub (direct stream copy). You can open it with XVI32. To me, it looks like one complete MESS of bytes...