View Profile
View Forum Posts
Has anyone done or seen any tests comparing Apple ProRes 422 with high-bitrate H.264 or MPEG2?
We use 422 as a delivery format to go to DCP for theatrical versions of trailers. The bitrate is around 150mbps.
We are wondering if it is possible to reach adequate quality and smaller sizes with another codec. Say, with bitrates of 50-100mbps, and tweaking the GOP size/structure. (E.g., only using "I" frames.) Assuming H264 is a more "efficient" codec, we are hoping to get smaller file sizes.
Or does H.264 just not have the color depth to compete?
It would be great to see if someone has done some real-world testing.
I just did a quick test using 50mbps, GOP size 1, I-frames only ... and "by eye" I see no difference whatsoever. Same level of noise, same colors. The ProRes is 1.92GB, the H264 is 566MB. How would one "test" the difference technically? Or measure the color information?
Some more testing ... with Adobe Media Encoder, using MPEG2, and the "4:2:2" profile, the ProRes file went from 1,920MB to 290MB! And I would say, subjectively, that is is 99% as good. This is remarkable to me. (With H264 I was down to 500-700MB)
At first I used the default GOP Settings of 3 M Frames, 15 N frames, but with GOP Size of 1. Then I tried setting M and N to 1, but, unexpectedly, the result was worse, especially for graphics. This doesn't make sense to me.
Noticed a lot of noise in a scene that was a hard cut to mostly white; tweaked "Intra DC Precision" to 10 bits ... and the result was much better, graphics too.
There are also many options in AME which are beyond my knowledge; I would like to understand them and maybe get even better results. Is there a guide to these terms? I could find nothing on Google.
- Macroblock Quantization?
- VBV Buffer Size?
- Noise Control (Sensitivity/Reduction) ?
- Write SDE?
- Intra DC Precision (8/9/10 bits) ?
- Ignore Frame Interval?
What do these mean? And how do we tweak them to get as close as possible to ProRes?
Thanks!
+ Reply to Thread
Results 1 to 25 of 25
-
-
Aha! Found explanation of 2 of those:
When in the Bitrate/Quality choice is "Specify Quality", the next control is a slider that controls the quality. This is actually the MPEG macroblock quantization level. Valid values are from 2 to 31. Low values of this variable mean higher quality and higher bitrate, so 2 is the highest quality and 31 is the lowest quality.
When the Bitrate/Quality choice is either "Specify Bitrate" or "Specify Quality", the next control is a slider that controls the MPEG VBV buffer size in units of 16k bits. The meaning of this parameter is confusing because it is used to help predict if the MPEG decoder might suffer buffer overflows or underflows. VBV buffer over/under flows are more commonly a problem when in CBR mode. If significant artifacts are present in the generated movie, try increasing this parameter.
From http://vis.lbl.gov/NERSC/Software/express/help6.2/help/relnotes/mpeg.htm -
I would stay with ProRes422 for compatibility reasonsOriginally Posted by d0g
AVC (as the standard) does offer 422, 10bit profiles, even 444 profiles . Some AVC encoders even offer lossless encoding (x264) - but it's not as widely compatible with all applications. ProRes is entrenched as the standard in the industry
for example, "AVC-Intra", the Panasonic format is standardized 10bit AVC, 422, intra - but still not accepted by some programs. Sony has it's own XAVC implementation with different profiles, but again , not as widely accepted
You technically test it by using PSNR, SSIM testing for objective measures, difference/subtract testing. You can test PSNR-Y, PSNR-U and PNSR-V for each channel (same with SSIM) or combined . There are several other metrics used as well, but PSNR and SSIM is the most common for objective testing . Signal engineers almost exclusively use PSNR
I just did a quick test using 50mbps, GOP size 1, I-frames only ... and "by eye" I see no difference whatsoever. Same level of noise, same colors. The ProRes is 1.92GB, the H264 is 566MB. How would one "test" the difference technically? Or measure the color information?
Some more testing ... with Adobe Media Encoder, using MPEG2, and the "4:2:2" profile, the ProRes file went from 1,920MB to 290MB! And I would say, subjectively, that is is 99% as good. This is remarkable to me. (With H264 I was down to 500-700MB)
At first I used the default GOP Settings of 3 M Frames, 15 N frames, but with GOP Size of 1. Then I tried setting M and N to 1, but, unexpectedly, the result was worse, especially for graphics. This doesn't make sense to me.
But you were using long GOP, not intra for the first tests. To GOP size of 1 is intra (all I frame). The quality is worse - that is expected - because there is no temporal compression. Each frame is encoded and self contained, not deriving any information from other frames. In order to use all Intra formats, you need higher bitrates (thus filesizes) for comparable quality. When using very high bitrates, the quality will be better (because no IBP fluctuations)
The whole idea of using intra formats is they are less compressed, easier to edit, no I-B-P temporal quality fluctuations. That's why they not used as end delivery formats, only "master" or intermediate formats
For your MPEG2 tests - there will be significant differences in quality when if you grade it , or show it to a colorist or compressionist. It will fall apart, and not survive multiple generations. Look especially at dark areas, you will see fine macroblocks. Your eyes might not be trained for it, but it's easy to see when you know what to look for
Basically, the "strongest" compression technique in the last 20 years (and in the future with HEVC) is using temporal compression. That alone will reduce filesizes ~50%. But the negatives are IBP fluctuations, more difficulty editing. Long GOP formats are typically not considered "high quality" masters , even when the I:B:P ratios are set close together
I wouldn't bother, because MPEG2 will be 8bit only. The compression won't be any better when using I-frame and will be worse than ProRes. It's all negatives
What do these mean? And how do we tweak them to get as close as possible to ProRes?
For the other explanations you haven't found:
Noise Control (Sensitivity/Reduction) - sensitivity adjustment for noise. It doesn't denoise. It adjusts the motion threshold that encoder will search for matching blocks between frames. Lower values = higher sensitivity (slower search, better quality) . With noisy source, that will make it much slower to encode.
Write SDE? Sequence Display Extension will be written for each GOP. Basically some applications expect the flag to be written at the end of each GOP, it won't affect quality.
Intra DC Precision (8/9/10 bits) - This is the precision of the DC coefficients for the Intra macroblocks. You usually want to set it to 10 for highest quality
Basically - yes you can get better compression using Long GOP or even AVC - but you make some trade offs with compatibilty. You have to be more specific on what scenarios this is for , and if submitting ask if it will be acceptedLast edited by poisondeathray; 29th Aug 2014 at 09:35.
-
Wow, thanks for all that info! (Obviously I came to the right place.)
What are your thoughts about using XDCAM?
I just converted to XDCAM HD422 1080p30 50mbps and (again, to my naked eye) it looks basically identical to the ProRes, and is 66% smaller (as expected: 50 vs 150 mbps)
When I worked at a cable channel, we used XDCAM for the entire workflow, from ingest, to edit, to color, to delivery, to broadcast, to archive.
Our scenario/workflow ...
1. Clients send us feature films as ProRes ... 422, 422 HQ, or 4444 - anywhere from 100GB to 200GB.
2. We send drives to editors - sometimes on the other coast. This is expensive. If we could reduce file size, we could transfer features via internet.
3. Editors do their own coloring, fx, etc ... then send final edit back to us as ProRes upload. These are usually abou 1.5 to 3GB. This is time consuming ... often client is waiting, and ProRes upload of trailer could take an hour or more. Smaller files would save time.
4. Final ProRes goes out to either (a) conversion for web, or (b) conversion for DCP for theatrical release.
5. Archive ... obviously we keep a copy ... and our 20TB NAS is full. It would be great to be able to reduce archive size. -
Originally Posted by d0g
XDCAM HD422 is very standard stuff for broadcast, but quality is not acceptible for full features. It's same thing as 50Mb/s CBR Long GOP, MPEG2. Keep in mind, cinema typically has a higher standard than "broadcast quality"
On one hand, quality is lower (maybe not to average viewer, but it is and won't survive multiple generations), but on the other hand it's a very standardized format and compatible format across different applications, lower bitrate. Bitrate is everything here : Filesize = Bitrate * Running Time
For web, sure it's fine, even overkill. But for DCP/ theatrical release, no client would want you to degrade their footage with XDCAM. If word got out that you use XDCAM for your intermediate format, you will lose abut 99% of your theatrical clients - but I guess it depends on who your clients were. Perhaps some "Indy" types won't know the difference
The term "Archive" implies an original or lossless copy. It implies high quality. Re-encoding to XDCAM definitely isn't "archival" quality in theatrical circles, but usual practice in broadcast circles
I understand your dilemma, but you're going to have to make some tradeoffs here , either in quality, costs/storage space. Something has go to give... you have to decide what
Editors , colorists hate long GOP. Sure you can use it, but intra formats make things easier/faster to edit. You'll get complaints, some subtle , some not so subtle from the editors
Also there are platform issues. Mac/FCP based editors tend to want Prores only. FCP is still very entrenched in the industry . FCPX can import many native formats, but in the past FCP7 editors had to convert everything to prores anyways (takes time) . If you have some "old school" editors on the team, they will complain unless they get prores . Conversely , PC based editors typically don't like prores. It doesn't run as well on PC
If you were to use a MPEG2 422 long GOP derivative, you would need about 100Mb/s for it to be comparible in quality, and not all software can handle MPEG2 422 @ 100Mb/s . It's also 8bit, so any heavy grading will suffer. 10bit is the standard for theatrical intermediates. The problem with any MPEG2 derivative is macroblocking, noisy artifacts - again typical person might not see it, but anyone involved in video post production will see it. -
OK, thanks for the feedback. I think you're right on all counts.
Better solution may be to upgrade our internet connection ... and just expand our RAID. -
What about finding local editors for the team ? Coast to coast for courier/Fedex really isn't cost effective
For your proposed internet/cloud solution - bandwith isn't always that great in certain areas . What if some of your editors were rurally located ? -
Finding new editors isn't really an option; we have our team in place. Thanks tho
-
I don't get it: Fedex is too expensive? Not cost effective?
If you calculate 200GB needed to go cross-country Same-day or Next-day, you burn the ProRes as data files to BD-R-25s/50s. I just checked, and a small box of ~4-8 (50 vs. 25) BDs is <2lbs.
That's ~$125-150 USD one way. Much less on the way back where only 1 BD-25 would need to be sent (according to post #4). Say $100?
So that's $250 for next day back & forth - why is that not cost effective on a feature? Unless you are going to MULTIPLE editor sites (why would that be?).
And it would only add a few hours of burn time total.
Am I missing something?
Scott -
Originally Posted by Cornucopia
Not as cost effective or having faster turnaround has having local editors
HDD / SSD is better investment IMO, because it sounds like an ongoing service, not a few of one offs. He's going to be reusing these hopefully. Can't really reuse BD-R. Larger capital outlay upfront, but will payoff in ROI
The other issue implied is time. He said... "often client is waiting" . Probably not going to want to waste time burning on both ends, and editor doesn't want to transfer files files before editing for the same reason he doesn't want to transcode everything .
Maybe you already do this, d0g, but for quick feedback - you usually want a lower quality web preview, before the final edits are made. That's where web is useful for IMO, not for transferring source files. Maybe client doesn't like the grading. Or sometimes some bozo changes their mind and wants something completely different. Having to redo or make major edits again is a real PITA. That's where the web flexibility comes into play . And showing the client at least something may get them to temporarily stop breathing down your neck -
Ha, of course we do all approvals via low-res h264s! Many revisions before we lock picture.
But we spend $1000 or more per month shipping HDs from NY to LA and back. So if we had a way to 'beam' the features it would save us $10-15K/yr.
Blurays? That's way too much time and hassle.
150GB at 50mbps is ~ 6 hours (http://bitrate.info/) -- that would work great and save money. -
BD-XL-REs? (4x) write at 144mbps = 142min. burn time. for 150GB. And being re-writable, you can use them over and over again.
What's killing you is the shipping cost, because of the weight of the HDDs (and probably insurance, too). And shipping is never a good idea for HDDs, even with cushioning. Also, read speed probably equal to or greater than 144mbps, so could just read the files straight off the drive rather than transfer. 200GB=2discs=2drives, both online, no waiting?
Just a thought.
Scott -
Is there any reason why no one has suggested DVCPRO-50 or DVCPRO HD?
-
Quality is (relatively) lower, Not a full raster format (so will produce softer results) e.g. 1920x1080 is actually stored as 1440x1080Originally Posted by sophisticles
Basically it's impossible to get 1/3 of the size of Prores HQ using Intra compression without some serious damage . And if he's willing to use long GOP, it potentially causes a whole bunch of other problems with editing speed , possibly compatibilitly. Headaches from complaining editors -
Latency and random access times are way too slow for using it as editing storage mediumOriginally Posted by Cornucopia
-
It's an interesting idea. Not for editing, but for transport. Never thought of it. Will chew it over. Thanks!Originally Posted by Cornucopia
-
Member
- Sep 2014
- Jacksonville, FL
I know this topic is about an "alternative to ProRes", but honestly ProRes is a stellar codec in so many ways. It's 10 bit, 422 chroma sub-sampling, i-frame, etc. It's data rate vs quality is really unmatched in my opinion.
I have many networks that allow direct digital uploads of it as well, not to mention how universally acceptable it is across every major editing platform. All that said, my opinion is to stick with it because it's so strong.
BTW, I made a cool little ProRes Data Rate Calculator that's super easy to use and can help you assess your storage needs as well as data rate for all of the common frame sizes and frame rates or ProRes.
Anyways, that's my 2 cents.
Micha -
Two thoughts. First ProRes IS the low-bandwidth standard for DCP.
Second. If you have matching editing systems set up on the east and west coast, (Avid, Premiere or FCP) you can send low res proxies to your remote editors and have them email only the completed project files back to you for (automatic and almost instantaneous) re-conforming to the full rez material. A lot less strain on the ol' internet, and a greatly reduced FedEx bill. -
AVC and HEVC* have all these features (and more) as well while offering better compression (same quality at lower bitrate or better quality at same bitrate compared to ProRes). Reasons to favor ProRes are mainly compatibility/acceptance.Originally Posted by Micha McLain
*(HEVC 4:2:2 in draft status atm)Last edited by sneaker; 28th Sep 2014 at 01:12.
-
They have MOST of those features. They do not have the alpha channel of ProRes4444. For editing/compositing, that is extremely important.
Also, and more importantly in this case, AVC & HEVC would not give major gains in bitrate compared to ProRes, when using I-frame-only and 4:2:2 mode. Those 2 features are mandatory for high-quality, professional editing. And even the modest gains of using AVC-Intra are not worth the added complexity of the encoding type and its higher performance requirements. Again, this is meant for pro editors who are going to be doing multiple layers of HD/2k/4k, etc. The bandwidth requirements for multiple simultaneous streams of HD ProRes are very reasonable (I can do it with my RAID workstation at home) and the CPU/speed requirements are much less. pdr already mentioned the failings of MPEG2 LongGOP encoding for high-quality finishing, and with these codecs, that analogy is just extended.
I won't even grace the HEVC option with a comparison, we all know it is WAY TOO EARLY yet to be using that codec through-and-through for such a mission critical position as one's editing/finishing codec. To even mention it at this time shows you don't fully understand all the elements involved. Maybe in 3-5 years. Or more, if you were to want to use capture hardware-direct-to-edit-codec options (like those available for DNxHD, ProRes, Cineform, AVC-Intra and R3D). Wishful thinking, but we just aren't there yet.
@smrpix, I was under the impression from the OP that the point of the original question dealt specifically with needing the remote editors to have Hi-Quality versions to work with, not proxies. Sure, proxies make sense from a workflow standpoint (and one could use ProRes LT/proxy for this, or AVC, or SD DV, etc - lots of options), but the problem was (I thought) with maintaining MATCHING HQ assets at multiple locations.
I'm curious to get another post from d0g re: clarification of this...
Scott -
He only mentioned 10 bit, 4:2:2 and I frames, they have all of those.Originally Posted by Cornucopia
Any proof for that claim?Originally Posted by Cornucopia
-
True. You got me on that first bit. Plus, there is a new profile for AVC-Intra that includes 4:4:4, I see. Touché.
Proof? You mean like, am I going to do my own encode and post stuff? No. But I don't need to either. Not counting using custom solutions like tailored ffmpeg templates, one would likely be using AVC-Intra from a hardware-related encode (such as a Panny camera) or a standardized NLE export profile, and those follow expected, standardized guidelines. I'm not the one claiming, the manufacturers are. And they say: 100Mbps for 10bit 4:2:2 Hi quality. That is versus Prores' ~147Mbps. That is a savings, yes, but a modest savings, particularly compared to some LongGOP format that might be 25Mbps (which we've already said should be ruled out). So again, this modest savings is probably not worth the additional performance hit.
Scott -
I like and use ProRes a lot actually. Not as a delivery format, just intermediate/archival format. It's light, fast and has good quality. I think it's comparable to CineForm.
I figured out how to export to ProRes from Virtualdub just a few minutes ago actually. It works well after using DeShaker filter for example.......Got my retirement plans all set. Looks like I only have to work another 5 years after I die........ -
You can ask anyone that deals with these codecs regularly - In reality, you need ~150Mb/s AVC intra to be roughly equivalent to ProRes HQ or DNxHD's ~176Mbs . Yes it's better, but not a huge savings. Usually not worth the compatibility issues. The official Panasonic 100Mb/s AVC-Intra is notably noisy and way overhyped marketing. It's not even close. I've done quite a few subjective and objective tests on this.
Believe it or not, prores HQ is actually considered "noisy" by high end DP's . The 4444 variant is more forgiving, but also requires a lot more bitrate. Typically you will see noise, especially around dark, shadow areas. But the noise pattern is organic and forgiving, maybe even "filmic". 43dB PSNR is the usually cutoff for high quality broadcast, but you need >50dB PSNR for high budget, high end "masters", which the Prores HQ 422 variant struggles to achieve.
You can encode AVC intra with approx an additional 10-20% more efficiency by enabling CABAC and deblocking, but typically these are disabled for editing, and they are disabled as well in the official Panasonic AVC-Intra implementation. Even with those disabled, AVC Intra isn't as smooth for editing (decoding latency is higher) than the likes of Prores, Cineform and DNxHD. And with CABAC and deblocking everything feels like it grinds to a halt.
There are AVC implementation differences as well x264 usually does slightly better at intra encoding than Mainconcept for example, but sometimes has more compatibility issues
If anyone is really interested I can post a few tests / results, but these are pretty much known facts -
Please do. Not really using AVC Intra, but I always like to learn, especially about known facts I don't know since I'm not that familiar with that field.If anyone is really interested I can post a few tests / results, but these are pretty much known facts
----
In general I agree that compatibility is the main issue and I would probably suggest to stick with ProRes,...users currently on my ignore list: deadrats, Stears555, marcorocchini
Quote
Visit Homepage
