the demise of codecs as we know them...

[deadrats]

as the title suggests, something occurred to me recently, as i was playing around with a half dozen blu-ray and hddvd rips that i have sitting on my hdd's.

i was looking at the various audio and video streams, looking at the codecs, the bitrates they are encoded at and while trying to figure out the fastest way, with the maximum quality, to encode a 32 gig 1080p m2ts that i have, i started thinking about how much faster a nehelem would be than my quad core 9500, and i started thinking about the evolution of codecs, and compression in general, and i realized something, that i always knew in the back of my mind, but never really thought too much about:

the only reason codecs where invented was because storage was so limited.

now at first this may seem like a "no sh*t", but think about it for a few minutes: back when napster was all the rage, and more people still use 56k modems to connect to the internet, 128mb of ram was a lot, P3's where king, no one had a dvd burner, encoding an mp3 took forever and 20 gig hard drives where considered huge (not to mention hdd's cost an arm and a leg), software audio and video encoders were the "killer apps" that enticed end users to buy faster and faster cpu's and more and more ram.

because of limited hdd capacities, an entire cottage industry sprung up around encoding, and various codecs vied for market share: mp3, rm, ac3, ogg, wma/wmv/ mpeg-1, mpeg-2, mpeg-4, divx, xvid, the list is almost endless.

along with the codecs, a seperate, yet connected "sub culture" sprung up which revolved around converting cd and dvd rips into smaller sizes while maintaining quality as close to original as possible.

since a codec is a program that COmpresses and DECompresses video or audio, with the decompression occurring on the fly are the stream is played back, as codecs improved and where able to compress streams into smaller and smaller sizes with better and better quality, the demands on out hardware increased which spurred demand for faster and faster cpu's, and as video cards jumped on the band wagon and started including the capability to handle the decompression of video streams, the demand for faster video cards also increased. along with that the need and demand for faster/more ram, and faster chipsets also increased.

at the same time hdd capacities didn't keep up with the same rate of advancement as other parts of our pc's.

now consider this: if hard drive capacities had increased at a rate faster than other parts of our pc's, would the same trend happened? in other words, if back when napster was still king, and kazaa hadn't been sued yet, we had 500 gig hdd's, would any audio codec gained popularity?

i don't think so, i think people would have instead just ripped their cd's to uncompressed wave files. with out the need to encode mp3's, cpu demand would have decreased, as would the demand for faster/more ram.

how do i know? because i have been playing around with dvd's for years and now with hddvd and blu ray, i have noticed a rather interesting trend:

the dvd spec has always allowed for 3 different audio formats: mpeg layer 2, ac3 and lcpm, which is uncompressed audio. the interesting thing is that due to limited dvd capacities, even with dual layer dvd's, i have never seen a commercial dvd with uncompressed lcpm audio.

that has all changed with hddvd and blu-ray, i currently have a number of rips sitting on my hdd's, at sizes between 20 gig and 32 gig and looking at the streams i have noticed something very interesting: no matter which video codec is used, mpeg 2, vc-1 or h264, the m2ts or evo file, half the time, is using uncompressed lcpm audio, often time it will have a few ac3 streams as well and the video stream is using very high bit rates, even when encoded with vc-1 or h264 it's not uncommon to see 20+ Mb/s bitrates.

now trying to transcode a 32 gig 1080p 20+ Mb/s m2ts to anything else, requires a ton of computing horsepower, i don't even think a dual nehelem based system will be able to do it in real time, but the real question becomes why would i want to transcode it. the quality of the file as it is absolutely incredible and spending $1000 to upgrade to a pc that can transcode it to a lesser quality is absolutely silly. considering a 500 gig hdd can be had for $100, it's much smarter to just add another 500 gig hdd (i currently have 3) and keep the file as it is.

and that's when i realized: 1.5 terabyte hdd's are already here and if one were so inclined you could easily put together system with 3 terabytes of storage space for about $600. as hdd capacity increases, the need to transcode video into smaller, lower quality decreases, which consequently reduces the need for improvements in codecs as well as the demand for faster cpu's and ram.

at some point storage capacity will reach the level where the need to compress video at all will be eliminated, i can see in 10 years time 1 petabyte hdd's being common place, with possible optical media being in the hundreds of terabyte range.

the interesting thing is that current cpu's are more than fast enough to handle uncompressed video and audio, which makes me wonder if the reason intel is so eager to move us to solid state drives is because the capacity of ssd are where we where 8 years ago. if intel can convince end users that ssd's are better due to supposed better performance, people may be willing to trade capacity for speed. and if capacity can be reduced to the levels of 8 years ago, or at least slowed down significantly, the demand, and thus the market for faster and faster cpu's will continue increasing.

sidenote: intel maybe a bit too clever for their own good: larrabee may end up biting them in the ass. larrabee will be composed of 16 cores or more and it's widely speculated that it won't be as fast even current video cards from nvidia and amd. i don't think intel cares all that much, intel has been pushing ray tracing quite heavily, and for those that don't know ray tracing doesn't rely on DX or OGL, it's for all practical purposes software rendering and intel has already shown a demo at hi resolution running on a dual nehelem system in nearly real time. considering larrabee will have at least 16 cores, and each core will support hyper threading, it's not to big a stretch to expect such a demo running in faster than real time on larrabee. add in comments from several prominent video game developers that DX11 will be the last significant graphics api as well as an move towards ray tracing in general, larrabee may end up being the only game in town. i wonder though whether or not larrabee will end up cannibalizing their cpu sales when people realize that larrabee can run any x86 application, including OSes, with just minor modifications.

at any rate, back to codecs, as you can see i personally don't think we will be seeing compressed video for too many more years, which basically means that i don't expect the demand for faster cpu's to continue at the present rate, at least at the consumer level.

well i'm done "ranting", thoughts anyone?

[Micker_C]

Interesting story and made for some light reading Except when you got into hardware, that's not your strong suit. I have built systems since the 80's and although I can see a point with respect to proprietary systems, there are very few bottle necks in a well planned Core build. Software on the other hand is the tricky animal, though even that can get a good tweaking.

[yoda313]

i have never seen a commercial dvd with uncompressed lcpm audio.

You don't have too many concert or music video dvds do you? That is where the application comes into play. This is for audio PURISTS. It's almost a hybrid of the dvd audio format even though its STEREO. I guess its there way to say "hey regular dvds can be just as cool as dvd audio to audiophiles". Granted I usually leave it on 5.1 dd for concert dvds.....

There are a few constraints on your otherwise solid interpretation of what may eventually come to pass.

First off is the commercial model. Until a physical disc is made that can hold pure uncompressed video at 100gigs a minute or whatever the real world equivalent of all that 4k4k movie quality resolution is codecs will still be needed. You won't see SONY or other movie distributors selling a copy of a action movie on a harddrive. Its just not gonna happen. First of all the average joe won't like paying harddrive prices for one movie even if its the best quality you'll ever see it on. So hence the need for SOME compression of some type to fit on a cheaper, more marketable media type.

The other reason we'll never truly get rid of codecs is two fold - BACKWARDS COMPATIBLITY and - INTERNET SPEED.

Backwards compatibility is going to be the reason you'll still have needs for mpeg or wmv, .mov etc....

Internet Speed is the other reason some form of compression will be around for quite some time. Your argument over the need for mp3 versus wav is valid for the harddrive and 56k dial up speed issues. However even with 1mb and faster broadband speeds more common these days how many people want to wait to download an album the size of a full cd? Not to mention not everyone has a high capacity mp3 player in the 20gb or up range where you could use wavs realisticly.

That creates a third point - PORTABLE PLAYERS. Portable audio and video players are a big success these days largley due to COMPRESSION. Even though there are 100+gb plus mega players now available from archos, apple, and microsft and others these are more for high end technophobes that want their whole collection with them. I'm sure the 4-12gb range sells better than the 30gb and up monsters....



HOwever I do agree with you that on the home side compression will be less and less necessary. BUT one other snag occured to me. Not everyone uses a HTPC configuration. This means that if you want to watch a movie you captured or listen to a song you downloaded you have to move it from one form to another so you can watch it or listen to it more comfortably.

So I agree with you to an extent but there are certain technological and societal changes that need to be met for compression is no longer a necessity.

[tmw]

For audio, I think you may be right. There is some thought that the current WAV digital audio is already compressed (due to just the sampling frequency) and going to higher frequencies would require higher bitrate. But, we can't hear those higher frequencies, so it may not really matter.

On video, we see the resolution expanding to occupy the available space. I'm going to make up a metric to compare the efficiency of DVD to BR--the metric is the data density, measure by pixels of resolution per MB of storage capacity.

DVD capacity is ~8 GB (DL) and had 0.35 Mpixels (720x480) for ~43 pixels per MB of storage.

BR has capacity of 50 BG (DL) and 2.0 Mpixels (1920x1080) for ~41 pixels per MB of storage.

Thus, the advantage in storage capacity is offset by having more video information (more pixels) on the file. Thus, having a codec is just as important for BR than DVD.

Until we reach the point of not being able to see any visible benefits of having higher resolution (like wav files for audio), I think codecs will exist. They may go away, but it will be a lot further than 10 years down the road.

Okay, now I have to get back to a paper on learning organizations....

[Cornucopia]

Bitrate compression works in 2 areas, STORAGE SIZE and PIPELINE BANDWIDTH (throughput).

Have you ever heard the axiom, "as storage size increases, so does our ability to fill it up with newer and larger data" (or something similar)? Then you already know why the old Bill Gates quote of "all we'll ever need is 64k" is ludicrous. And to extrapolate, so it your assertion.

Let's do some math...

for example (assuming no overhead, etc):
2 hours of UNCOMPRESSED stereo audio @ 16bit, 44kHz is:
2 * 60 * 60 * 2 * 16 * 44100 = 1.18GB with a bitrate of 1.35Mbps

Sure, that might be easy on any recent/modern computer, but even today 1.35Mbps isn't available everywhere, and that certainly doesn't work on all types of devices beyond desktops/laptops.

Well, continuing with audio, what if you go higher definition with more time...
4 hours of UNCOMPRESSED 5.1surround audio @ 24bit, 96kHz:
4 * 60 * 60 * 5.1 * 24 * 96000 = 19.7GB with a bitrate of 11.21Mbps.

Now, that's already too large and too fast for DVDs and CDs, as well as many other current devices and storage mediums (let's not even talk about the internet).

Well, extrapolating some more, let's assume you want top/studio/concert quality in "multitrack" form so you can adjust your own balance (trade that for alternate languages, commentary, etc)...
8 hours of UNCOMPRESSED 10.2surround audio @ 32bit(floating point), 192kHz:
8 * 60 * 60 * 10.2 * 32 * 192000 = 210.11GB with a bitrate of 59.77Mbps

Now we're at a level that's even too big for BD and some current off-the-shelf current harddrives (though not the biggest ones). And the bandwidth is pushing the limit on many peripherals (at least for sustained rates).

And so far I've only talked about audio...

2 hours of 601 (YUV 4:2:2 SD UNCOMPRESSED Video) is 160Mbps bitrate with storage size of 140GB. Even while you might be able to fit the size, the sustained bitrate is gonna hurt everywhere but the MOBO backplane and highest serial protocols, plus you'll need RAID systems.

RGB is a little bit more, going from SD to HD quadruples the requirements, going to 4k quadruples again, highspeed framerates woud double or triple that at least, stereo3d would double that, HDR would double or quadruple that. Holography anyone?

It's a moving bar...what's good enough? (and for what applications?).

Personally, I wouldn't worry about it. Codecs will be with us for the unforseeable future, just get what's good for your immediate and near-term needs and upgrade when the time comes.

Scott

[jagabo]

Larrabee on hold: http://arstechnica.com/hardware/news/2009/12/intels-larrabee-gpu-put-on-ice-more-news-...me-in-2010.ars

[edDV]

Home users use commercial media as is or rip and store for convenience. Reompression is needed to fit local storage media capacity or bit rate limitations.

For audio, mp3 allowed practical internet downloads and storage of ~10 albums per CDR.

For video, divx/xvid allowed storage of a complete movie on a CDR or several on a DVDR. Also, divx/xvid bit rates could be handled on a wifi home network.

The reason most recompress HD video is to fit a DVDR-5/9 or to allow SD DVD playback. Quality is sacraficed in order to fit affordable media. Also, bit rates need to be reduced to allow wireless transmission on home networks or playback on portable devices.

Now precompressed HD video is coming into the home from broadcast or internet sources. You can fit ~44 min of broadcast 720p to a DVDR-5 blank or more to dual layer DVDR. Or you can stream 10-16Mb/s over local wireless networks directly to a net enabled TV. If you wire your home for gigabit Ethernet, centralized servers free you from optical media or wireless limitations and the need to recompress. Still, recompression is needed for portable devices or internet transmission.

So, I agree the trend is to less recompression on a PC and to asset storage on centralized home media servers. Soon, affordable hardware re-compression devices will be available to wirelessly stream from a central server.

Cable/sat media distribution companies would rather you centralize assets on their servers so they can control distribution to individual displays. Other media distribution companies see internet distribution as a way to offer and control assets.

Some users will be satisfied with cable or internet VOD. Others will want to run their own home servers with local storage.

[JohnnyMalaria]

All media are crude approximations of our true sensory experiences. Technology advances to make the illusions more believable and the industry uses those advances to tease people into upgrading existing equipment. Nevertheless, we watch 2D video that fills just a small part of our visual field accompanied by contrived audio intended to give the impression of 3D. Paraphrasing what was stated earlier, we will always never have just enough storage space. It's the same with the new software needs new hardware because new hardware can run new software cycle.