Announcing ZPEG Demonstration Site - Works with h264, h265!

[raymondwestwater]

ZPEG is a new video pre-processing technology that reduces your compressed file size by removing visual redundancies before feeding the video to the compressor. The ZPEG site (www <dot> zpeg <dot> com) has a "demo" page that allows upload of a video file to be transcoded to a lower bandwidth. Once processing is complete, you can download and compare the quality of the raw and pre-processed versions of the content. You will see about a 20% reduction in file size without degradation in visual quality. I hope you enjoy the site! - Raymond

[-Habanero-]

I don't get it, 20% compared to what? The raw original? x264 and x265 can compress a video 200:1 and retain high quality.

[raymondwestwater]

+Hi Habanero, Sorry I left this unclear. The way we arrived at this measure (20%) was by:
- compressing each of the Xiph Derf demonstration videos to 6Mbs
- Determining the mean square error of the compression
- Pre-Processing the file using our technology to make it more compressible
- Compressing the pre-processed file until it reached the same mean square error
- Measuring the resulting bandwith
The results of these experiments are documented on the demo page following the interective demo section.
Thanks again for your question
- Raymond

[-Habanero-]

You are still being unclear. And what was the format of the "Xiph Derf demonstration videos"? Were they already compressed or not?

[raymondwestwater]

Those videos are uncompressed.

[-Habanero-]

Before we go on I'd like to state for the record that moderator johns0 infracted me for my above posts. Baldrick should seriously screen his mentally ill staff.

Anyway... if the videos are uncompressed and you can only compress them by 20% then your codec is worse than MPEG-1. Why should anyone use it?

[pandy]

Temporal smooth, contra-sharpening and perhaps few other things... suddenly it become to be a new, patented technology with some marketing blahblah behind.
Later people are curious why image is so artificial and plastic (but! but! but! it has same PSNR!!! )

[raymondwestwater]

This technology adds another 20% to MPEG-1, MPEG-2, AVC, HEVC, VP9, (your codec here). That is why everybody should use it (jk)!

[raymondwestwater]

pandy, I understand your skepticism! There is a lot of noise being made out there about applying a human visual model to conventional filtering techniques. Or even using convention techniques, as you suggest.

For what it's worth, This technology is based on a transform-domain compression codec, and uses no filtering techniques whatsoever. It operates in the decorrelated 3-D transform domain, applying the human visual model to the decorrelated basis vectors. Thus the resulting visual artifacts can be precisely tuned to the viewing distance and frame rate. That coupled with a new multiresolution entropy removal algorithm allows video "push" from the server, instantly adapting to changing channel conditions.

As ZPEG has no prayer of ever commercializing this codec, they extracted the visual processing segment and used it to pre-process video sequences. The results were pretty good, and so we're hoping people will like our approach.

[raymondwestwater]

habanero, I'm sorry you were reprimanded, I really don't see why!

[sophisticles]

pandy, I understand your skepticism! There is a lot of noise being made out there about applying a human visual model to conventional filtering techniques. Or even using convention techniques, as you suggest.

For what it's worth, This technology is based on a transform-domain compression codec, and uses no filtering techniques whatsoever. It operates in the decorrelated 3-D transform domain, applying the human visual model to the decorrelated basis vectors. Thus the resulting visual artifacts can be precisely tuned to the viewing distance and frame rate. That coupled with a new multiresolution entropy removal algorithm allows video "push" from the server, instantly adapting to changing channel conditions.

As ZPEG has no prayer of ever commercializing this codec, they extracted the visual processing segment and used it to pre-process video sequences. The results were pretty good, and so we're hoping people will like our approach.

This "explanation" sounds like a lot of purposely constructed jibber-jabber.

To be clear, are you claiming that if one where to start with a source file of any origin, say an DVD sourced mpeg-2 and then proceeded to encode it twice, once by feeding straight to an encoder, such as x265, DivX, whatever and once by processing it with this "new" technology and then feeding it to the same encoder, that the final encode that had been pre-processed with ZPEG would require 20% less bit rate to achieve the same quality or put another way that at the same bit rate the ZPEG processed file would be of 20% higher quality at the same bit rate? How is this being measured, by PSNR, SSIM or some other way? And are you saying that this process is independent of the codec used?

There's a major problem with the claims being made about ZPEG and it's this, all encoders work by looking for redundancies, both spatially and temporally, that can be discarded. One of the simplest such schemes exists when going from an Intra only codec to a gop based codec, that's how P frames and B frames work, the encoder keeps only the parts that changed and discards the rest.

With gop based encoders, the process is much harder and decisions have to be made about what can be permanently discarded (unlike the above scanrio which is lossless and can be reversed 100%) and what should be kept to maintain visual fidelity.

I'm betting that if ZPEG does in fact compress a file by 20% then that comes at the expense of the encoder as it leaves 20% less data that can be safely discarded by the encoder, in other words it's "stealing" 20% worth of compression from the encoder the end user wants to use.

I would think that if in fact this technology was capable of what doing what you claim, the smart move would not be to come to a forum such as this and sell us on the benefits but rather to contact a deep pocket player in codecs like Google or Intel and license the technology to them for millions.

Google would pay a fortune to gain another 20% compression efficiency for Youtube as bandwidth costs are exorbitant and Intel would likewise pay through the nose to improve Quick Sync compression by another 20% and make their hardware encoder the undisputed king.

The fact that your company hasn't done so, or if the have that neither of them had any interest tells me all I need to know about ZPEG.

[raymondwestwater]

Your first paragraph concisely sums up our proposition:

To be clear, are you claiming that if one where to start with a source file of any origin, say an DVD sourced mpeg-2 and then proceeded to encode it twice, once by feeding straight to an encoder, such as x265, DivX, whatever and once by processing it with this "new" technology and then feeding it to the same encoder, that the final encode that had been pre-processed with ZPEG would require 20% less bit rate to achieve the same quality or put another way that at the same bit rate the ZPEG processed file would be of 20% higher quality at the same bit rate? How is this being measured, by PSNR, SSIM or some other way? And are you saying that this process is independent of the codec used?

The point is that we are able to find and remove redundancies that motion-estimation compressors (H.264, H.265, VP9) inherently cannot. This is due to the way that they extract redundancy, which is to find a motion compensation vector then record the difference or error term. But there is no good body of theory to quantize the difference between two video macroblocks, and that is where the efficiency reaches its limit.

We do measure our gain when we reach equal mean square error as compared to the compressor-only version. Our expectation is that the bandwidth at which visual equivalence is reached will be even lower, as our human visual model is quite non-linear and should always outperform a simple error measure.

While I cannot possibly do the theory justice here, its underpinnings were described in my PhD thesis, which has been bound by Springer and is available (Real-Time Video Compression: Techniques and Algorithms, Raymond Westwater). So there is an available reference if you are interested in exploring the underpinnings in more depth.

But the real proof is in the seeing! That's why we have launched an interactive demonstration on the ZPEG site, and welcome feedback, comments, and, yes, flamings. We'd love to have you give it a try and tell us what you think (I'm not afraid you will be too coy to speak your mind).

As far as selling the technology for millions - it's not so easy to go from garage to Google! I am exploring the possibility of sharing the ideas and technology with a wider audience as a step towards eventual fame and monetization (spoken somewhat tongue-in-cheekly). It seems the big leap can only be taken as a series of small steps...

[raymondwestwater]

P.S. sophisticles, love your handle!

[sophisticles]

I would be happy to try your technology, unfortunately you seem to be purposely obfuscating usage of your preprocessor; you do not allow the download of a binary file that one can use to process a source, you force a tester to upload a sample but you put an artificial limit of 65mb per file and you do not allow to download the processed file.

If you are really serious about this technology and really believe in it alter your site so that one or more of the following is allowed:

1) Offer a cli binary that one can use to process a source prior to encoding with the encoder of their choice

or

2) Allow a user to upload a sample and then be able to download the supposedly processed file so that he may then feed it into the encoder of their choice.

Unless and until one is able to start out with a source file, encode one of them with the encoder of their choice and process another copy with ZPEG and then encode it with the same encoder and use 20% less bit rate and compare the 2 encodes using PSNR, SSIM and one's own eyes, I have to consider your site a scam.

The way you have it configured now you control all the parameters and the end user is supposed to trust that you are not pulling a fast one on the back end.

No one, except for the greenest of video enthusiasts, is going to buy what you're selling until you make the process more transparent.

[raymondwestwater]

"You seem to be purposely obfuscating usage of your preprocessor" - or am I trying to make it accessible to the widest possible audience?

I do indeed have a cli version available for use. I can make it available to you if you would like to make a real test, or I can run a file through the preprocessor for you and send you the result.

Why don't you contact me through the contact page? I would be happy to create you an account and show you the usage.

[_Al_]

Those lossless files after using your ZPEG method could be large, so why not to provide a following x264 encoding with known settings, like x264.exe --crf 18 --tune grain etc. Using latest x264 version. In that case, everyone can have a rough idea what your method gets rid of within video and how it looks while watching on screen. Comparing yours H264 and encoded original using same setting.

[Brad]

The ABC clip on the demo page has some sort of ugly crosshatching artifact throughout (both unfiltered and filtered).

[raymondwestwater]

_AI_, thanks for your feedback. We originally used equal-QP to develop our bandwidth savings model (which gives excellent results), but have evolved our testing methodology to be more adaptable to the real world. So we compress both raw and processed YUV files to the fixed "target" bandwidth, and extract the average QP as a measure of injected MSE. We then re-compress the pre-processed content to the bandwidth that results in the same average QP.
For those of you following along at home, we are using an Ubuntu 16.04 server with the x264 and x265 versions for that release.
x264:
x264 0.148.2643 5c65704
(libswscale 3.1.101)
(libavformat 56.40.101)
(ffmpegsource 2.22.0.0)
built on Jan 18 2016, gcc: 5.3.1 20160114
x264 configuration: --bit-depth=8 --chroma-format=all
libx264 configuration: --bit-depth=8 --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat/ffmpegsource license: GPL version 2 or later


x265:
x265 [info]: HEVC encoder version 1.9
x265 [info]: build info [Linux][GCC 5.3.1][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX

Command line arguments:
${X264} --no-progress --preset ${SPEEDTEXT} --fps ${RATE} --input-res ${WIDTH}x${HEIGHT} --vbv-bufsize ${X264BUFFER} --bitrate ${X264BITRATE} -o ${FILE}.m${VB}.${X264BITRATE}.${X264EXT} ${FILE}.m${VB}.yuv

Where
X264 is x264 or x265
SPEEDTEXT is fast, medium, or slow
X264BITRATE is the target bitrate
X264BUFFER is twice the target bitrate (2 seconds)
X264EXT is mp4 or mkv

[raymondwestwater]

vaporeon800, I don't know to which video you refer, nor at what resolution you are playing it. But I can say that the videos are sourced from the Xiph Derf site without change, and so if the source contains artifacts, we should correctly reproduce them! However, if there is a problem with the compression process, I'd love to know more!

[sophisticles]

"You seem to be purposely obfuscating usage of your preprocessor" - or am I trying to make it accessible to the widest possible audience?

I do indeed have a cli version available for use. I can make it available to you if you would like to make a real test, or I can run a file through the preprocessor for you and send you the result.

Why don't you contact me through the contact page? I would be happy to create you an account and show you the usage.

I accept, I have sent you an email via the contact page with details, in summary I do not use Windows, I use Ubuntu Mate, if you have a variant that runs on Linux I will accept that, if not I ask that you download the file found on this page:

http://demo-uhd3d.com/fiche.php?cat=uhd&id=17

Process it with your application with the highest quality settings it offers, no filtering or resizing, then give me the processed output. I will then encode the original using the medium presets for both x264 and x265 via handbrake and I will encode the one you processed with the same settings and encoders only with 20% lower bit rate.

I will then calculate PSNR using ffmpeg of encodes and I expect to see that the values are within a small margin of error of each other. I will also post all encodes here for every to see and judge for themselves.

If this is acceptable to you I have provided my email address to you for your reply.

Maybe you really have discovered something new and if so I would think that you would be a millionaire very soon.

Honestly, I am very skeptical.

I do have some follow up questions regarding the claims you are making concerning your application:

What type of processing speeds do you claim?

Is the code easily integrable into existing encoder designs or would it require a complete rewrite of an encoder, i.e. could developers like the x265 team take your code, integrate it into their software and receive the same benefit? Based on what you have said it doesn't seem to me that the methods you claim to use are complimentary with traditional MPEG based encoders?

Edit, I think I may be thinking about this the wrong way, in theory I should be able to take any file I want, run it through your application, get an output that is 20% smaller, with the same quality (i.e. PSNR greater than 50 decibels and SSIM greater than .99) and then be able to run both your version and the original version through a traditional encoder and get the expected compression for both files.

If you're willing to provide a binary I'll be happy to run a litany of tests.

[raymondwestwater]

sophisticles,

I'd like to start with a sanity check - clearly PSNR is not a good measure to compare the raw and pre-processed YUV files, right? Because our intention is to add as much error as possible to improve compressibility without affecting the visual experience? So we would validate that difference by SSIM, VMAF or golden eye viewing (we use all three).

Then the error introduced by the compression should indeed be equal when compared to the source from which the compression comes when the bandwidth savings are as advertised. In this case, I estimate bandwidth savings for your file, x264 compression, medium speed, 16Mbs (Netflix quality), 2 second buffer to be 16%.

The raw file QP as reported by x264 is:
x264 [info]: frame I:16 Avg QP:18.20 size:1511247
x264 [info]: frame P:1303 Avg QP:21.44 size:137573
x264 [info]: frame B:2668 Avg QP:24.94 size: 10435

The average raw file QP is calculated as:
((16*18.20) + (1303*21.44) + (2668*24.95))/(16+1303+2668)
=23.78

The processed file QP as reported by x264 is:

x264 [info]: frame I:16 Avg QP:17.44 size:1170226
x264 [info]: frame P:1307 Avg QP:20.61 size:142855
x264 [info]: frame B:2664 Avg QP:25.08 size: 11117

The average processed file QP is calculated as:
((16*17.44)+(1307*20.61)+(2664*25.08)/(16+1307+2664)
=23.58

The expected bandwidth savings can then be calculated as:
1 - 2^((23.58-23.78)/6) = 2.3%

Unfortunately, this clip generates little advantage. However the two calculations that will have equal error will be:
<raw yuv>.yuv - <raw@16Mbs>.yuv = <proc yuv>.yuv - <proc@15.6Mbs>.yuv

[raymondwestwater]

Well, I have the results, and they're not great. The overall savings for this clip is only 2.3%, double sad as we normally save an average of 30% on UHD clips. The only anomaly I can see is that the clip is compressed with a variable frame rate, and apparently virtually all frames have been dropped. This causes two problems for my algorithm:
1. The reconstructed frames are perfect digital copies, perfect fodder for the x.264 engine
2. The discontinuity in motion does not serve the three-dimensional model well.
Anyway, you got me!

[Brad]

vaporeon800, I don't know to which video you refer

The only one that is downloadable on the demo page, unless I'm missing something. "ABC clip" = 720p60 red carpet footage with ABC bug in corner.

But I can say that the videos are sourced from the Xiph Derf site without change, and so if the source contains artifacts, we should correctly reproduce them!

I don't see this one on their site, but I agree regarding the artifacts. I just think it would make more sense to add a video demo that isn't flawed to start with, such as one of the Xiph sequences you quote the numbers from.

[raymondwestwater]

Thanks, vaperon, for your comment. I will point out that some of the Derf video suffer from the same problem, especially the controlled burn video. All of the Derf videos have been rendered and are available for review at www <dot> zpeg <dot> com <slash> netflix <dot> shtml. These videos are mastered at Neflix rates. Perhaps I will add links to these files into the demo page...

[jagabo]

The only one that is downloadable on the demo page, unless I'm missing something. "ABC clip" = 720p60 red carpet footage with ABC bug in corner.

I downloaded this video and didn't see such artifacts. It has some milder artifacts though:


[Attachment 38791 - Click to enlarge]
http://www.zpeg.com/videos/leftraw.mp4

[Brad]

The compression of the side-by-side has smoothed over most of the picture. Plenty of blocks remain where the artifacts are still present. I've cropped the left side of your JPEG and highlighted some of them.



The ones I watched were the higher-quality individual downloads.

[jagabo]

Ah, I see the files you downloaded. They do have much heavier artifacts, like in your sample image. The side-by-side video must have blurred away lots of those artifacts during re-encoding.

In any case, about all they are doing is applying a little 3d noise reduction with edge protection. Anyone who encodes video knows that will help compression.

[lordsmurf]

In any case, about all they are doing is applying a little 3d noise reduction with edge protection. Anyone who encodes video knows that will help compression.

That's what I'm reading here as well.

[raymondwestwater]

jagabo, lordsmurf, of course the idea is similar in principle to noise reduction, except that the "noise" to be removed is based on a human visual model operating in the decorrelated 3-D transform domain. So the removal is far more targeted and effective than is possible with filtering, which cannot provide the same level of discrete access to the individual basis vectors.

[raymondwestwater]

vaperon800, thank you for your clarifications. To create the side-by-side rendering, I separately made the two compressions, combined them and re-compressed at a higher bit rate, as you suggest. I use a similar technique to make the side-by-side renderings in the online demo. It's a compromise solution...