Do we really need codecs?

[android]

Do we really need codecs to create any digital media?
for example can't we create images with using binary?
Different codecs doesn't do the same? , don't they simplify the work? Just want to know in details

[_Al_]

Codecs make raw data many, many times smaller. It compresses original arrays of three r,g,b planes (or YUV planes or non planar with point sequences).

Imagine one 20x20 video frame with black color only. In raw version you'd need to store below data for r,g and b planes:
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Instead you define some "code" and define that frame like: numpy.zeros((20,20,3), numpy.uint8)
Thats three 20x20 planes using numpy in python. That is not a codec, but you get the picture. Codecs introduce some algorithms, lossy or not, and simplify what is possible to reduce data. Raw data are just too much to store.

[android]

I guess both have their own advantages not sure but raw data is faster in processing? And compressed will take more time then raw

And also I guess if some one code 20x20 black picture then it will only work in his system not in everyone's system ,so codecs will remove this limitations?

[android]

By the way ,what are some examples of some simplest audio video codecs ,and they written in which programing languages

[_Al_]

Processing is always done with decompressed raw data. Imagine darkening a video frame, you must have an access to every pixel and change all data, but when exporting it always is compressed using a particular codec again

That example of mine was a programming language where things work one way only, a code syntax to a video that has a low entropy (uniform video), not the other way. You cannot have a video transformed to a code syntax (programming language) fully, there is too much entropy. Codecs must do it differently, I am no expert in this. It can register a changes to an I frame, so if frames are the same , it just references that frame fully. Like encoding an image to an 5min audio, if setting x264 codec with infinite GOP length, all is stored is one frame and compressed audio. Also there are compressions for blocks from within a frame. The whole blocks moving in a vector thru-out frames could be referenced etc.

[_Al_]

Also, like zip algorithms you can just simple reference blocks of the same numbers. Not sure if codecs use that or not, but perhaps that is also in their arsenal. I could be wrong here but compressed videos as a data cannot be compressed using zip algorithms much at all, the entropy (complexity) is already too high so there is a little to compress something. Perhaps yes but with longer time and effort, which becomes counterproductive. Video would need a lot of computing power for compression and also decompression or have a huge latency during playback.

[Cornucopia]

@_Al_, most codecs already incorporate lossless algorithms like Zip, with the "lossy" portion being the entropy residual that is thrown away (but not, in "lossless" forms).

@android, YES we really need codecs for most video & audio & photo uses. Uncompressed/RAW is very high bandwidth, especially with video, and ESPECIALLY with HD, UHD video.
Codecs reduce the bitrate by 1/2, 1/3, 1/5, 1/10, 1/50, 1/100, 1/300, etc. The smarter & more complex the algorithm, the more it can compress, but the more delay there is in the processing, and the more CPU processing power needed. The best balance is in personal/corporate priorities, and that is often how the given codec is chosen.
Example: 1920x1080 @60p Fps in 24bit RGB color (8bit/pixel/color) = 1920 * 1080 * 60 * 24 = ~3Gbps! An decent quality h265-compressed version of that can be 5Mbps, or ~1/600 of the size.

Scott

[android]

Uncompressed contents won't delay or latency while playback?
for example 3gbps video will not lag while playback?

And codecs are written in programming languages?

https://www.codespeedy.com/create-a-single-colored-blank-image-in-cpp-using-opencv/

Other programing languages don't use h264 etc?

[Cornucopia]

Uncompressed/RAW doesn't have any inherent delay in decoding, but I think what you are really wondering is how is that possible at those bandwidths? There could easily be lags, delays if the playback system is not designed to easily handle the throughput necessary for that much data. That is not a deficiency in the code/data, but a weakness in the system. And that is another reason why codecs are used - to keep data throughput in a reasonable range.

All codecs are algorithms (encoders, decoders), and all algorithms are programs written in programming languages.
Don't confuse the actor (verb), aka the codec algorithm, with the action (object noun) of the data which is formatted according to the codec's rules.
An h264-encoded video stream has no programming code built into it, but it is formatted in a very strict structure so that any compliant decoder should be able to properly decode it in the same way. We speak often of codecs, but what we are usually referring to are the data files formatted to a codec's rules. Calling it by the codec name is just an easy way to recognize which algorithms were used/are needed.
Al's python example might have confused you because their example did include the programming code in the bitstream. They did mention that as being a nonstandard example, but maybe that wasn't clear.

Scott

[android]

Uncompressed/RAW doesn't have any inherent delay in decoding

When videos is uncompressed 10mbps then I think ,if system read write speed below 10mbps then it won't play properly,not sure about this

[android]

By the way h264 codec is written in which language?

[Cornucopia]

That is what I said - it is a weakness of that particular system. I am sure about it. If the system can/could handle (your example of) 10Mbps, then there is basically no delay in decoding, because the processing is simply copying bits from stored memory to screen/GPU memory.

And again, that is one of the main points in favor of using codecs: to allow the targeted playback systems to be able to sufficiently handle the throughput requirements.
IRL, 10Mbps is easy, but if a device can not sufficiently, consistently handle that bitrate, it would not matter what format the source was, compressed or not.

To get to the heart of your OP question: why do you want to know those details? It seems you are having a bit of difficulty understanding the basic overview of codecs, so I am concerned that these details will be out of your league.

Here is an analogy : someone wants to send you a poster in the mail but your mail slot is only 6" wide x 1" high, but the poster is 20"x30". How can they get it into the slot?
Maybe by rolling it up. Maybe by cutting it up (different algorithms), or maybe by folding it enough so that it fits in the slot. They each have their benefits and detractors and risks, and amount of time needed to prepare (both in the squeezing and in the unsqueezing.
Your last question is like asking "what if the 6" x 1" slot is not big enough?" Or, "what if I don't have a 20"x30" opening available?"

Scott

[Cornucopia]

The h264 "codec" is a format standard. The standard is executed by various implementations of it created by different companies, but all are designed to be compatible with the standard. (This includes x264, mainconcept's version, Sony or Panasonic's version, etc). It can be designed in many ways using many programming languages (although I am pretty sure the vast majority of codecs use C and Assembly language, for speed).

This line of questioning is beginning to sound like a high schooler needing to turn in a report.

Scott

[pandy]

Uncompressed contents won't delay or latency while playback?
for example 3gbps video will not lag while playback?

Allow me to do simple math for you - 3Gbps video over 10 Mbps link will give 3000/10=300 sec latency and also frame speed will be around 0.0033fps.

And codecs are written in programming languages?

Not necessarily - if your codec is implemented in HW then it can be some HDL or even schematic (depends on codec complexity).

[android]

why do you want to know those details?

Am thinking to open my youtube channel ,before creating videos I want to learn everything about video and audio
And also this knowledge will help me in many things like recording ,editing etc

The h264 "codec" is a format standard

So it's just a design ,how video should be compressed
And x264 is library who encodes videos according to h264 design

HDL

hdl full form?

[pandy]

HDL

hdl full form?

Hardware Description Language

[Cornucopia]

why do you want to know those details?

Am thinking to open my youtube channel ,before creating videos I want to learn everything about video and audio
And also this knowledge will help me in many things like recording ,editing etc

The h264 "codec" is a format standard

So it's just a design ,how video should be compressed
And x264 is library who encodes videos according to h264 design

HDL

hdl full form?

Hdl is basically a type of machine-specific or machine family-specific programming language.

Yes, the compression format's algorithms are the designs for how the media should be compressed.
Yes, x264 is a specific implementation library used to execute the h264 format's designs. One of many similar competing libraries (some of which cost money or need licensing, some like x264 which do not cost money).

Knowing compression might help you be efficient with creating quality media, but is tangential or supplemental to the core process of creating, editing and publishing media in general. For that, it is more important to know how cameras & microphones & lights work; what consists of a good image, sound, scene composition, story; how to create smooth & intuitive edits into an engaging narrative.

Scott