MemoriesOnTV - Problem #1

[terrypin]

This picks up directly from my last post in the the thread 'Ffdshow - two separate configurators?', and arises from jagabo's suggestion that I describe the specific problem(s) I'm having with the CodeJam program 'MemoriesOnTV' (MoTV).

I actually have two problems with MoTV that are presumably codec-related. They may be separate or independent. For now, while I have ffdshow temporarily uninstalled, I'll try to describe exclusively just the first of these, Problem #1. It arises when I use MoTV without ffdshow installed. (As already described, Problem #2, which arises when ffdshow is installed, is even more serious.)

I aim to decribe this in enough initial detail to allow others to help isolate and fix the problem. Codejam too say they are working on it, although I have not yet had confirmation they've even reproduced it.

The problem in brief is that MoTV won't work with my usual compressed files. I mainly use DivX. I have recently updated that, and my current configuration, from the 'Update' utility is:
DivX Player 6.4
DivX Codec 6.4.0
DivX Converter 6.2.1

The test source file I've used here was made by capturing about 1 second of screen activity in Google Earth with a program called FRAPS. I've used it in both uncompressed and compressed forms in the past in many programs. I compress it using VirtualDub. In this example I've done so with DivX (my usual choice) and with XviD (in case that offers any clues). Here are the full details of those 3 files.

File Name: TestUncompressed.avi
================================
AVICodec
--------
File : 3.83 MB (3.83 MB), duration: 0:00:00, type: AVI, 0 audio stream(s), quality: 65 %
Video : 3.83 MB, 0 Kbps, 25.0 fps, 728*480 (4:3), FPS1 = Unknown, Supported

GSpot
-----
Name.........: TestUncompressed.avi
Filesize.....: 3.81 MB (or 3,911 KB or 4,005,144 bytes)
Runtime......: 00:00:01 (18 fr)
Video Codec..: <unknown>
Video Bitrate: 44476 kb/s
Audio Codec..: No Audio Channels
Audio Bitrate: n.a.
Frame Size...: 728x480 (1.52:1) [~44:29]
FPS..........: 25.000

The following codec should be capable of processing this format:
--> Fraps Video Decompressor

DirectShow claims to be able to play the file. The following combination of filters were
used:
D:\My Pictures\MOVIES\Misc\TestUncompressed.avi (Video Source)
Avi Splitter (Video Pre-processor)
[VFW: Fraps Video Decompressor] (Video Decoder)
Color Space Converter (Video Post-processor)
Video Renderer (Video Renderer)


File Name: TestCompressedDivX.avi
==================================
AVICodec
--------
File : 426 KB (426 KB), duration: 0:00:00, type: AVI, 0 audio stream(s), quality: 65 %
Video : 418 KB, 0 Kbps, 25.0 fps, 728*480 (4:3), DX50 = DivXNetworks Divx v5, Supported


GSpot
-----
Name.........: TestCompressedDivX.avi
Filesize.....: 426 KB (or 436,676 bytes)
Runtime......: 00:00:01 (18 fr)
Video Codec..: DivX 5.0
Video Bitrate: 4758 kb/s
Audio Codec..: No Audio Channels
Audio Bitrate: n.a.
Frame Size...: 728x480 (1.52:1) [~44:29]
FPS..........: 25.000

The following codecs should both capable of processing this format:
DivX Decoder Filter
XviD MPEG-4 Video Decoder
DirectShow is apparently able to play the file as well (see "DirectShow Render",
below).

DirectShow claims to be able to play the file. The following combination of filters were
used:
D:\My Pictures\MOVIES..-..estCompressedDivX.avi (Video Source)
Avi Splitter (Video Pre-processor)
DivX Decoder Filter (Video Decoder)
Video Renderer (Video Renderer)

File Name: TestCompressedXviD.avi
==================================
AVICodec
--------
File : 334 KB (334 KB), duration: 0:00:00, type: AVI, 0 audio stream(s), quality: 65 %
Video : 326 KB, 0 Kbps, 25.0 fps, 728*480 (4:3), XVID = XVID Mpeg-4, Supported

GSpot
-----
Name.........: TestCompressedXviD.avi
Filesize.....: 334 KB (or 342,076 bytes)
Runtime......: 00:00:01 (18 fr)
Video Codec..: XviD
Video Bitrate: 3706 kb/s
Audio Codec..: No Audio Channels
Audio Bitrate: n.a.
Frame Size...: 728x480 (1.52:1) [~44:29]
FPS..........: 25.000

The following codecs all claim to be capable of processing this format:
--> XviD MPEG-4 Codec
--> DivX Decoder Filter
--> XviD MPEG-4 Video Decoder

DirectShow claims to be able to play the file. The following combination of filters were
used:
D:\My Pictures\MOVIES..-..estCompressedXviD.avi (Video Source)
Avi Splitter (Video Pre-processor)
DivX Decoder Filter (Video Decoder)
Video Renderer (Video Renderer)

====================

Note that in practice, in a typical MoTV project, I import many JPG photos, a smaller number of AVI digicam clips (with sound), and maybe 3 or 4 of these large 'capture', no-sound, AVI files that we're exclusively focusing on here.

MoTV works OK with the uncompressed file. That's what I meant about a work-around. But that file is normally huge, typically 300-700 MB, so obviously that in itself has adverse impacts on space and performance.

MoTV will not work with the DivX or XviD compressed files. Those clips will not preview. I just get a static image displayed throughout the duration the clip is 'playing'. Not surprisingly, if I go ahead regardless and render this 1-clip project to DVD, that too just displays a static image when played.

This DivX compressed file runs OK in all my usual players, such as PowerDVD, WMP10, Media Player Classic, NeroShowTime, etc. And I can load it into Movie Maker or NeroVision Express and it works properly - I can preview it correctly.

So... any help in isolating the problem would be greatly appreciated please. If I hear anything from CodeJam on it I will report back.

--
Terry, West Sussex, UK

[guns1inger]

300 - 700 MBs isn't huge unless you have 4 GB HDD. I work with DV files of 20 30 GB size, and many around here work with lossless or uncompressed video that is 2 - 4 times that size.

That said, you have found your answer, whether you like it or not - MOTV doesn't seem to like Divx - so use a different format. If it will read losslessly encoded footage (lagarith, huffyuv) then try one of those instead. Otherwise, use the uncompressed version.

As the Stones said - you can't always get what you want.

[jagabo]

Try disabling the Divx decoder's "output YUV 4:2:0 when supported" option.

You should also try using Xvid instead. If you do, Xvid has more options for output colorspaces. Try each of them.

[terrypin]

Thanks, appreciate the fast response.

Re space, 4 or 5 clips of that size, total maybe 2-3 GB within MoTV, while working on creating the movie seems a bit daft. Typically, the DivX version is 30 or 40 times smaller. And I may have half a dozen similar projects around in one stage or another - total 10-15 GB if uncompressed. Plus backups. Apart from space, performance is also affected. Every operation on the files takes so much longer, particularly previewing them.

As described, XviD didn't work either. So far the only type I've found that does is Cinepak, but that gave only about a 50% size reduction. Those two you suggest, lagarith and huffyuv are new to me. (Like most codecs!) They don't appear in the following list that VirtualDub offers:

(Uncompressed RGB/YCbCr)
Cinepak Codec by Radius
Converts YV12, I420, YUY2, UYVY to RGB24
Converts YV12, I420, YUY2, UYVY to RGB24
DivX® 6.4.0 Codec (1 Logical CPU) <--- My usual choice
DivX® 6.4.0 YV12 Decoder
FHelix I420 YUV Codec
Intel Indeo(R) Video R3.2
Intel Indeo® Video 4.5
Intel IYUV codec
Ligos Indeo® Video 5.11
MainConcept DV Codec 2.0.4
Microsoft H.261 Video Codec
Microsoft H.263 Video Codec
Microsoft MPEG-4 Video Codec V1
Microsoft MPEG-4 Video Codec V2
Microsoft MPEG-4 Video Codec V3
Microsoft RLE
Microsoft Video 1
Microsoft Windows Media Video 9
XviD MPEG-4 Codec

So presumably you're recommending I first find and install them?

--
Terry, West Sussex, UK

[guns1inger]

You don't have to find them, as Videohelp has done the hard yards for you. Either click on the names in your post - see how they are blue links - go to the tools emnu (on your left), then the codecs section, and you will find links.

[terrypin]

OK, installed both of those thanks. HuufYUV seemed to work in MoTV, although preview was a bit jerky. About to test on much larger AVI. However, compression only seemed to be to about 45% of original, so clearly not as effectve as DiVX.

Jury still out on Lagarith, which previewed but very jerkily.

I'm also wondering whether some of the iffy previewing I get in MoTV may be because my 4-year old PC is borderline adequate on performance (Athlon 1800, 512 MB).

--
Terry, West Sussex, UK

[jagabo]

HuffYUV and Lagarith are lossless codecs and therefore don't compress by a lot. The jerky preview is because the low compression makes big files and your computer is having trouble reading the files off the hard drive fast enough, especially if you are using an external drive. They are not meant as playback codecs but for editing (high quality, low compression).

[terrypin]

Thanks for the follow-up. Pleased to report that I successfully compressed an original of 2.3 GB to 970 MB with HuffYUV, imported it into MoTV (where it gave an acceptable preview), and the resultant DVD (on HD) played fine.

So that's a good working compromise until maybe we can find some way to get the smaller DivX (50 MB) or XviD (31 MB) files working in MoTV.

Thanks for that lead, guns1inger

--
Terry, West Sussex, UK

[jagabo]

If I was you I'd skip the Divx encode anyway. Each compression to a lossy codec will reduce quality. Stick with the lossless HuffYUV or Lagarith. Another intermediate you might consider is MJPEG. It's a lossy codec but you can set it to high quality settings and get reasonable results. Not as clean as HuffYUV but smaller.

[guns1inger]

Got to agree with jagabo - Divx or Xvid are great for low size high compression viewing, but are awful for intermediate files if quality means anything to you. Like MP3, then do damage, even at lower levels of compression, and when you have to compress again for mpeg2, the damage compounds.

Put a 200+ GB HDD into your machine so these little (and they are little, in the video scheme of things) don't cause you so much concern.

[terrypin]

OK, thanks both. I decided to adopt that approach.

But I've hit a snag. I just tried my second large AVI - and the compressed versions with both HuffYUV and Lagarith are larger than the original!

Orig.avi 543 428 KB
Huf.avi 1,006,917 KB
Lag.avi 678,277 KB

--
Terry, West Sussex, UK

[guns1inger]

Yes. You are going from something heavily compressed to something that is losslessly compressed. In most (if not all) cases the results will be bigger when using huffyuv or lagarith.

I don't understand why this is a snag, or why you see it as a problem. These are intermediate files. When this is encoded to mpeg2 it will be much smaller again.

I am working on a video at the moment. It started life as a VHS to DVD transfer of almost 27gb. After almost 12 hours of processing tonight, it will become a 70+gb lagarith encoded avi. Tomorrow night it will get encoded to mpeg2 and be just over 4.2GB. The 70gb file will then get deleted. This is how it works if you want to play with video.

You could fit all three of those files on 2 floppy disks. If space is that tight, buy a coupleof 512mb USB drives and use those for temporary video storage. Those files won't make a dent in them.

[terrypin]

Space here is not really all that tight. I have 3 HDs in this PC: 2 x 200 GB, 1 x 47 GB, although one of the large ones is used exclusively for nightly backups, including all video files. So it's a fair question, as to why I call it a 'snag'.

I suppose it's mainly because I've never really used the larger files other than as 'sources'. And I've got into the habit of working with DivX with previous programs. MovieMaker, NeroVision Express, etc, never had a problem with them. And I took their compactness for granted. And for me the quality seems fine (bearing in mind this type of source is already inherently of limited sharpness).

So it's a matter of adjusting my approach and expectations. (And admittedly, I'm also curious why MoTV has this problem while other programs do not.)

There's an alternative, of course. As explained in my opening post and in the thread that prompted it, if I get MoTV Problem #2 fixed, then I could re-install ffdshow, which apparently allows MoTV to work with DivX.

--
Terry, West Sussex, UK

[guns1inger]

I guess the most compressed I ever get prior to the final output is DV. It is not unusally to have a couple of GB disappear into a title sequence or logo without any trouble at all. That's not to say HDD isn't a concern, but I know that most of what is created is transitive, and not for permanent storage

[terrypin]

I've now exhaustively tested all the codecs I have (as offered in VirtualDub>Compression). The majority failed, notably DivX and XviD as discussed earlier. These are the ones that worked, i.e. they previewed OK in MoTV.

Type Size
------------------ ----------
Original (FRAPS) 317,709 KB

Cine2k 107,674 KB
MPEG4-1 15,159 KB
MPEG4-2 15,141 KB
MPEG4-3 15,055 KB
WMV9 9,840 KB

--
Terry, West Sussex, UK