VideoHelp Forum
+ Reply to Thread
Results 1 to 24 of 24
Thread
  1. Member
    Join Date: Aug 2014
    Location: United States
    Search PM
    Has anyone done or seen any tests comparing Apple ProRes 422 with high-bitrate H.264 or MPEG2?

    We use 422 as a delivery format to go to DCP for theatrical versions of trailers. The bitrate is around 150mbps.

    We are wondering if it is possible to reach adequate quality and smaller sizes with another codec. Say, with bitrates of 50-100mbps, and tweaking the GOP size/structure. (E.g., only using "I" frames.) Assuming H264 is a more "efficient" codec, we are hoping to get smaller file sizes.

    Or does H.264 just not have the color depth to compete?

    It would be great to see if someone has done some real-world testing.

    I just did a quick test using 50mbps, GOP size 1, I-frames only ... and "by eye" I see no difference whatsoever. Same level of noise, same colors. The ProRes is 1.92GB, the H264 is 566MB. How would one "test" the difference technically? Or measure the color information?

    Some more testing ... with Adobe Media Encoder, using MPEG2, and the "4:2:2" profile, the ProRes file went from 1,920MB to 290MB! And I would say, subjectively, that is is 99% as good. This is remarkable to me. (With H264 I was down to 500-700MB)

    At first I used the default GOP Settings of 3 M Frames, 15 N frames, but with GOP Size of 1. Then I tried setting M and N to 1, but, unexpectedly, the result was worse, especially for graphics. This doesn't make sense to me.

    Noticed a lot of noise in a scene that was a hard cut to mostly white; tweaked "Intra DC Precision" to 10 bits ... and the result was much better, graphics too.

    There are also many options in AME which are beyond my knowledge; I would like to understand them and maybe get even better results. Is there a guide to these terms? I could find nothing on Google.
    • Macroblock Quantization?
    • VBV Buffer Size?
    • Noise Control (Sensitivity/Reduction) ?
    • Write SDE?
    • Intra DC Precision (8/9/10 bits) ?
    • Ignore Frame Interval?

    What do these mean? And how do we tweak them to get as close as possible to ProRes?

    Thanks!
    Quote Quote  
  2. Member
    Join Date: Aug 2014
    Location: United States
    Search PM
    Aha! Found explanation of 2 of those:

    When in the Bitrate/Quality choice is "Specify Quality", the next control is a slider that controls the quality. This is actually the MPEG macroblock quantization level. Valid values are from 2 to 31. Low values of this variable mean higher quality and higher bitrate, so 2 is the highest quality and 31 is the lowest quality.

    When the Bitrate/Quality choice is either "Specify Bitrate" or "Specify Quality", the next control is a slider that controls the MPEG VBV buffer size in units of 16k bits. The meaning of this parameter is confusing because it is used to help predict if the MPEG decoder might suffer buffer overflows or underflows. VBV buffer over/under flows are more commonly a problem when in CBR mode. If significant artifacts are present in the generated movie, try increasing this parameter.

    From http://vis.lbl.gov/NERSC/Software/express/help6.2/help/relnotes/mpeg.htm
    Quote Quote  
  3. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    Originally Posted by d0g View Post
    Has anyone done or seen any tests comparing Apple ProRes 422 with high-bitrate H.264 or MPEG2?

    We use 422 as a delivery format to go to DCP for theatrical versions of trailers. The bitrate is around 150mbps.

    We are wondering if it is possible to reach adequate quality and smaller sizes with another codec. Say, with bitrates of 50-100mbps, and tweaking the GOP size/structure. (E.g., only using "I" frames.) Assuming H264 is a more "efficient" codec, we are hoping to get smaller file sizes.

    Or does H.264 just not have the color depth to compete?
    I would stay with ProRes422 for compatibility reasons

    AVC (as the standard) does offer 422, 10bit profiles, even 444 profiles . Some AVC encoders even offer lossless encoding (x264) - but it's not as widely compatible with all applications. ProRes is entrenched as the standard in the industry

    for example, "AVC-Intra", the Panasonic format is standardized 10bit AVC, 422, intra - but still not accepted by some programs. Sony has it's own XAVC implementation with different profiles, but again , not as widely accepted


    I just did a quick test using 50mbps, GOP size 1, I-frames only ... and "by eye" I see no difference whatsoever. Same level of noise, same colors. The ProRes is 1.92GB, the H264 is 566MB. How would one "test" the difference technically? Or measure the color information?
    You technically test it by using PSNR, SSIM testing for objective measures, difference/subtract testing. You can test PSNR-Y, PSNR-U and PNSR-V for each channel (same with SSIM) or combined . There are several other metrics used as well, but PSNR and SSIM is the most common for objective testing . Signal engineers almost exclusively use PSNR

    Some more testing ... with Adobe Media Encoder, using MPEG2, and the "4:2:2" profile, the ProRes file went from 1,920MB to 290MB! And I would say, subjectively, that is is 99% as good. This is remarkable to me. (With H264 I was down to 500-700MB)

    At first I used the default GOP Settings of 3 M Frames, 15 N frames, but with GOP Size of 1. Then I tried setting M and N to 1, but, unexpectedly, the result was worse, especially for graphics. This doesn't make sense to me.

    But you were using long GOP, not intra for the first tests. To GOP size of 1 is intra (all I frame). The quality is worse - that is expected - because there is no temporal compression. Each frame is encoded and self contained, not deriving any information from other frames. In order to use all Intra formats, you need higher bitrates (thus filesizes) for comparable quality. When using very high bitrates, the quality will be better (because no IBP fluctuations)

    The whole idea of using intra formats is they are less compressed, easier to edit, no I-B-P temporal quality fluctuations. That's why they not used as end delivery formats, only "master" or intermediate formats

    For your MPEG2 tests - there will be significant differences in quality when if you grade it , or show it to a colorist or compressionist. It will fall apart, and not survive multiple generations. Look especially at dark areas, you will see fine macroblocks. Your eyes might not be trained for it, but it's easy to see when you know what to look for

    Basically, the "strongest" compression technique in the last 20 years (and in the future with HEVC) is using temporal compression. That alone will reduce filesizes ~50%. But the negatives are IBP fluctuations, more difficulty editing. Long GOP formats are typically not considered "high quality" masters , even when the I:B:P ratios are set close together




    What do these mean? And how do we tweak them to get as close as possible to ProRes?
    I wouldn't bother, because MPEG2 will be 8bit only. The compression won't be any better when using I-frame and will be worse than ProRes. It's all negatives

    For the other explanations you haven't found:

    Noise Control (Sensitivity/Reduction) - sensitivity adjustment for noise. It doesn't denoise. It adjusts the motion threshold that encoder will search for matching blocks between frames. Lower values = higher sensitivity (slower search, better quality) . With noisy source, that will make it much slower to encode.

    Write SDE? Sequence Display Extension will be written for each GOP. Basically some applications expect the flag to be written at the end of each GOP, it won't affect quality.

    Intra DC Precision (8/9/10 bits) - This is the precision of the DC coefficients for the Intra macroblocks. You usually want to set it to 10 for highest quality





    Basically - yes you can get better compression using Long GOP or even AVC - but you make some trade offs with compatibilty. You have to be more specific on what scenarios this is for , and if submitting ask if it will be accepted
    Last edited by poisondeathray; 29th Aug 2014 at 10:35.
    Quote Quote  
  4. Member
    Join Date: Aug 2014
    Location: United States
    Search PM
    Wow, thanks for all that info! (Obviously I came to the right place.)

    What are your thoughts about using XDCAM?
    I just converted to XDCAM HD422 1080p30 50mbps and (again, to my naked eye) it looks basically identical to the ProRes, and is 66% smaller (as expected: 50 vs 150 mbps)

    When I worked at a cable channel, we used XDCAM for the entire workflow, from ingest, to edit, to color, to delivery, to broadcast, to archive.

    Our scenario/workflow ...

    1. Clients send us feature films as ProRes ... 422, 422 HQ, or 4444 - anywhere from 100GB to 200GB.
    2. We send drives to editors - sometimes on the other coast. This is expensive. If we could reduce file size, we could transfer features via internet.
    3. Editors do their own coloring, fx, etc ... then send final edit back to us as ProRes upload. These are usually abou 1.5 to 3GB. This is time consuming ... often client is waiting, and ProRes upload of trailer could take an hour or more. Smaller files would save time.
    4. Final ProRes goes out to either (a) conversion for web, or (b) conversion for DCP for theatrical release.
    5. Archive ... obviously we keep a copy ... and our 20TB NAS is full. It would be great to be able to reduce archive size.
    Quote Quote  
  5. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    Originally Posted by d0g View Post
    Wow, thanks for all that info! (Obviously I came to the right place.)

    What are your thoughts about using XDCAM?
    I just converted to XDCAM HD422 1080p30 50mbps and (again, to my naked eye) it looks basically identical to the ProRes, and is 66% smaller (as expected: 50 vs 150 mbps)

    When I worked at a cable channel, we used XDCAM for the entire workflow, from ingest, to edit, to color, to delivery, to broadcast, to archive.

    Our scenario/workflow ...

    1. Clients send us feature films as ProRes ... 422, 422 HQ, or 4444 - anywhere from 100GB to 200GB.
    2. We send drives to editors - sometimes on the other coast. This is expensive. If we could reduce file size, we could transfer features via internet.
    3. Editors do their own coloring, fx, etc ... then send final edit back to us as ProRes upload. These are usually abou 1.5 to 3GB. This is time consuming ... often client is waiting, and ProRes upload of trailer could take an hour or more. Smaller files would save time.
    4. Final ProRes goes out to either (a) conversion for web, or (b) conversion for DCP for theatrical release.
    5. Archive ... obviously we keep a copy ... and our 20TB NAS is full. It would be great to be able to reduce archive size.




    XDCAM HD422 is very standard stuff for broadcast, but quality is not acceptible for full features. It's same thing as 50Mb/s CBR Long GOP, MPEG2. Keep in mind, cinema typically has a higher standard than "broadcast quality"

    On one hand, quality is lower (maybe not to average viewer, but it is and won't survive multiple generations), but on the other hand it's a very standardized format and compatible format across different applications, lower bitrate. Bitrate is everything here : Filesize = Bitrate * Running Time

    For web, sure it's fine, even overkill. But for DCP/ theatrical release, no client would want you to degrade their footage with XDCAM. If word got out that you use XDCAM for your intermediate format, you will lose abut 99% of your theatrical clients - but I guess it depends on who your clients were. Perhaps some "Indy" types won't know the difference

    The term "Archive" implies an original or lossless copy. It implies high quality. Re-encoding to XDCAM definitely isn't "archival" quality in theatrical circles, but usual practice in broadcast circles



    I understand your dilemma, but you're going to have to make some tradeoffs here , either in quality, costs/storage space. Something has go to give... you have to decide what

    Editors , colorists hate long GOP. Sure you can use it, but intra formats make things easier/faster to edit. You'll get complaints, some subtle , some not so subtle from the editors

    Also there are platform issues. Mac/FCP based editors tend to want Prores only. FCP is still very entrenched in the industry . FCPX can import many native formats, but in the past FCP7 editors had to convert everything to prores anyways (takes time) . If you have some "old school" editors on the team, they will complain unless they get prores . Conversely , PC based editors typically don't like prores. It doesn't run as well on PC

    If you were to use a MPEG2 422 long GOP derivative, you would need about 100Mb/s for it to be comparible in quality, and not all software can handle MPEG2 422 @ 100Mb/s . It's also 8bit, so any heavy grading will suffer. 10bit is the standard for theatrical intermediates. The problem with any MPEG2 derivative is macroblocking, noisy artifacts - again typical person might not see it, but anyone involved in video post production will see it.
    Quote Quote  
  6. Member
    Join Date: Aug 2014
    Location: United States
    Search PM
    OK, thanks for the feedback. I think you're right on all counts.
    Better solution may be to upgrade our internet connection ... and just expand our RAID.
    Quote Quote  
  7. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    What about finding local editors for the team ? Coast to coast for courier/Fedex really isn't cost effective

    For your proposed internet/cloud solution - bandwith isn't always that great in certain areas . What if some of your editors were rurally located ?
    Quote Quote  
  8. Member
    Join Date: Aug 2014
    Location: United States
    Search PM
    Finding new editors isn't really an option; we have our team in place. Thanks tho
    Quote Quote  
  9. Member Cornucopia's Avatar
    Join Date: Oct 2001
    Location: Deep in the Heart of Texas
    Search Comp PM
    I don't get it: Fedex is too expensive? Not cost effective?

    If you calculate 200GB needed to go cross-country Same-day or Next-day, you burn the ProRes as data files to BD-R-25s/50s. I just checked, and a small box of ~4-8 (50 vs. 25) BDs is <2lbs.
    That's ~$125-150 USD one way. Much less on the way back where only 1 BD-25 would need to be sent (according to post #4). Say $100?

    So that's $250 for next day back & forth - why is that not cost effective on a feature? Unless you are going to MULTIPLE editor sites (why would that be?).
    And it would only add a few hours of burn time total.

    Am I missing something?

    Scott
    "When will the rhetorical questions end?!" - George Carlin
    Quote Quote  
  10. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    Originally Posted by Cornucopia View Post
    I don't get it: Fedex is too expensive? Not cost effective?

    If you calculate 200GB needed to go cross-country Same-day or Next-day, you burn the ProRes as data files to BD-R-25s/50s. I just checked, and a small box of ~4-8 (50 vs. 25) BDs is <2lbs.
    That's ~$125-150 USD one way. Much less on the way back where only 1 BD-25 would need to be sent (according to post #4). Say $100?

    So that's $250 for next day back & forth - why is that not cost effective on a feature? Unless you are going to MULTIPLE editor sites (why would that be?).
    And it would only add a few hours of burn time total.

    Am I missing something?

    Scott



    Not as cost effective or having faster turnaround has having local editors

    HDD / SSD is better investment IMO, because it sounds like an ongoing service, not a few of one offs. He's going to be reusing these hopefully. Can't really reuse BD-R. Larger capital outlay upfront, but will payoff in ROI

    The other issue implied is time. He said... "often client is waiting" . Probably not going to want to waste time burning on both ends, and editor doesn't want to transfer files files before editing for the same reason he doesn't want to transcode everything .






    Maybe you already do this, d0g, but for quick feedback - you usually want a lower quality web preview, before the final edits are made. That's where web is useful for IMO, not for transferring source files. Maybe client doesn't like the grading. Or sometimes some bozo changes their mind and wants something completely different. Having to redo or make major edits again is a real PITA. That's where the web flexibility comes into play . And showing the client at least something may get them to temporarily stop breathing down your neck
    Quote Quote  
  11. Member
    Join Date: Aug 2014
    Location: United States
    Search PM
    Ha, of course we do all approvals via low-res h264s! Many revisions before we lock picture.

    But we spend $1000 or more per month shipping HDs from NY to LA and back. So if we had a way to 'beam' the features it would save us $10-15K/yr.

    Blurays? That's way too much time and hassle.

    150GB at 50mbps is ~ 6 hours (http://bitrate.info/) -- that would work great and save money.
    Quote Quote  
  12. Member Cornucopia's Avatar
    Join Date: Oct 2001
    Location: Deep in the Heart of Texas
    Search Comp PM
    BD-XL-REs? (4x) write at 144mbps = 142min. burn time. for 150GB. And being re-writable, you can use them over and over again.

    What's killing you is the shipping cost, because of the weight of the HDDs (and probably insurance, too). And shipping is never a good idea for HDDs, even with cushioning. Also, read speed probably equal to or greater than 144mbps, so could just read the files straight off the drive rather than transfer. 200GB=2discs=2drives, both online, no waiting?

    Just a thought.

    Scott
    "When will the rhetorical questions end?!" - George Carlin
    Quote Quote  
  13. Is there any reason why no one has suggested DVCPRO-50 or DVCPRO HD?
    Quote Quote  
  14. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    Originally Posted by sophisticles View Post
    Is there any reason why no one has suggested DVCPRO-50 or DVCPRO HD?
    Quality is (relatively) lower, Not a full raster format (so will produce softer results) e.g. 1920x1080 is actually stored as 1440x1080





    Basically it's impossible to get 1/3 of the size of Prores HQ using Intra compression without some serious damage . And if he's willing to use long GOP, it potentially causes a whole bunch of other problems with editing speed , possibly compatibilitly. Headaches from complaining editors
    Quote Quote  
  15. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    Originally Posted by Cornucopia View Post
    BD-XL-REs? (4x) write at 144mbps = 142min. burn time. for 150GB. And being re-writable, you can use them over and over again.

    What's killing you is the shipping cost, because of the weight of the HDDs (and probably insurance, too). And shipping is never a good idea for HDDs, even with cushioning. Also, read speed probably equal to or greater than 144mbps, so could just read the files straight off the drive rather than transfer. 200GB=2discs=2drives, both online, no waiting?

    Just a thought.
    Latency and random access times are way too slow for using it as editing storage medium
    Quote Quote  
  16. Member
    Join Date: Aug 2014
    Location: United States
    Search PM
    Originally Posted by Cornucopia View Post
    BD-XL-REs? (4x) write at 144mbps = 142min. burn time. for 150GB. And being re-writable, you can use them over and over again.

    What's killing you is the shipping cost, because of the weight of the HDDs (and probably insurance, too). And shipping is never a good idea for HDDs, even with cushioning.
    Scott
    It's an interesting idea. Not for editing, but for transport. Never thought of it. Will chew it over. Thanks!
    Quote Quote  
  17. Member
    Join Date: Sep 2014
    Location: Jacksonville, FL
    Search Comp PM
    I know this topic is about an "alternative to ProRes", but honestly ProRes is a stellar codec in so many ways. It's 10 bit, 422 chroma sub-sampling, i-frame, etc. It's data rate vs quality is really unmatched in my opinion.

    I have many networks that allow direct digital uploads of it as well, not to mention how universally acceptable it is across every major editing platform. All that said, my opinion is to stick with it because it's so strong.

    BTW, I made a cool little ProRes Data Rate Calculator that's super easy to use and can help you assess your storage needs as well as data rate for all of the common frame sizes and frame rates or ProRes.

    Anyways, that's my 2 cents.

    Micha
    Quote Quote  
  18. Member
    Join Date: Jun 2012
    Location: USA
    Search Comp PM
    Two thoughts. First ProRes IS the low-bandwidth standard for DCP.

    Second. If you have matching editing systems set up on the east and west coast, (Avid, Premiere or FCP) you can send low res proxies to your remote editors and have them email only the completed project files back to you for (automatic and almost instantaneous) re-conforming to the full rez material. A lot less strain on the ol' internet, and a greatly reduced FedEx bill.
    Quote Quote  
  19. Originally Posted by Micha McLain View Post
    I know this topic is about an "alternative to ProRes", but honestly ProRes is a stellar codec in so many ways. It's 10 bit, 422 chroma sub-sampling, i-frame, etc. It's data rate vs quality is really unmatched in my opinion.
    AVC and HEVC* have all these features (and more) as well while offering better compression (same quality at lower bitrate or better quality at same bitrate compared to ProRes). Reasons to favor ProRes are mainly compatibility/acceptance.

    *(HEVC 4:2:2 in draft status atm)
    Last edited by sneaker; 28th Sep 2014 at 02:12.
    Quote Quote  
  20. Member Cornucopia's Avatar
    Join Date: Oct 2001
    Location: Deep in the Heart of Texas
    Search Comp PM
    They have MOST of those features. They do not have the alpha channel of ProRes4444. For editing/compositing, that is extremely important.

    Also, and more importantly in this case, AVC & HEVC would not give major gains in bitrate compared to ProRes, when using I-frame-only and 4:2:2 mode. Those 2 features are mandatory for high-quality, professional editing. And even the modest gains of using AVC-Intra are not worth the added complexity of the encoding type and its higher performance requirements. Again, this is meant for pro editors who are going to be doing multiple layers of HD/2k/4k, etc. The bandwidth requirements for multiple simultaneous streams of HD ProRes are very reasonable (I can do it with my RAID workstation at home) and the CPU/speed requirements are much less. pdr already mentioned the failings of MPEG2 LongGOP encoding for high-quality finishing, and with these codecs, that analogy is just extended.

    I won't even grace the HEVC option with a comparison, we all know it is WAY TOO EARLY yet to be using that codec through-and-through for such a mission critical position as one's editing/finishing codec. To even mention it at this time shows you don't fully understand all the elements involved. Maybe in 3-5 years. Or more, if you were to want to use capture hardware-direct-to-edit-codec options (like those available for DNxHD, ProRes, Cineform, AVC-Intra and R3D). Wishful thinking, but we just aren't there yet.

    @smrpix, I was under the impression from the OP that the point of the original question dealt specifically with needing the remote editors to have Hi-Quality versions to work with, not proxies. Sure, proxies make sense from a workflow standpoint (and one could use ProRes LT/proxy for this, or AVC, or SD DV, etc - lots of options), but the problem was (I thought) with maintaining MATCHING HQ assets at multiple locations.

    I'm curious to get another post from d0g re: clarification of this...

    Scott
    "When will the rhetorical questions end?!" - George Carlin
    Quote Quote  
  21. Originally Posted by Cornucopia View Post
    They have MOST of those features.
    He only mentioned 10 bit, 4:2:2 and I frames, they have all of those.

    Originally Posted by Cornucopia View Post
    Also, and more importantly in this case, AVC & HEVC would not give major gains in bitrate compared to ProRes, when using I-frame-only and 4:2:2 mode.
    Any proof for that claim?
    Quote Quote  
  22. Member Cornucopia's Avatar
    Join Date: Oct 2001
    Location: Deep in the Heart of Texas
    Search Comp PM
    True. You got me on that first bit. Plus, there is a new profile for AVC-Intra that includes 4:4:4, I see. Touché.

    Proof? You mean like, am I going to do my own encode and post stuff? No. But I don't need to either. Not counting using custom solutions like tailored ffmpeg templates, one would likely be using AVC-Intra from a hardware-related encode (such as a Panny camera) or a standardized NLE export profile, and those follow expected, standardized guidelines. I'm not the one claiming, the manufacturers are. And they say: 100Mbps for 10bit 4:2:2 Hi quality. That is versus Prores' ~147Mbps. That is a savings, yes, but a modest savings, particularly compared to some LongGOP format that might be 25Mbps (which we've already said should be ruled out). So again, this modest savings is probably not worth the additional performance hit.

    Scott
    "When will the rhetorical questions end?!" - George Carlin
    Quote Quote  
  23. Member racer-x's Avatar
    Join Date: Mar 2003
    Location: 3rd Rock from the Sun
    Search Comp PM
    I like and use ProRes a lot actually. Not as a delivery format, just intermediate/archival format. It's light, fast and has good quality. I think it's comparable to CineForm.

    I figured out how to export to ProRes from Virtualdub just a few minutes ago actually. It works well after using DeShaker filter for example.......
    The memories of a man in his old age, are the deeds of the man in his prime.......
    Quote Quote  
  24. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    You can ask anyone that deals with these codecs regularly - In reality, you need ~150Mb/s AVC intra to be roughly equivalent to ProRes HQ or DNxHD's ~176Mbs . Yes it's better, but not a huge savings. Usually not worth the compatibility issues. The official Panasonic 100Mb/s AVC-Intra is notably noisy and way overhyped marketing. It's not even close. I've done quite a few subjective and objective tests on this.

    Believe it or not, prores HQ is actually considered "noisy" by high end DP's . The 4444 variant is more forgiving, but also requires a lot more bitrate. Typically you will see noise, especially around dark, shadow areas. But the noise pattern is organic and forgiving, maybe even "filmic". 43dB PSNR is the usually cutoff for high quality broadcast, but you need >50dB PSNR for high budget, high end "masters", which the Prores HQ 422 variant struggles to achieve.

    You can encode AVC intra with approx an additional 10-20% more efficiency by enabling CABAC and deblocking, but typically these are disabled for editing, and they are disabled as well in the official Panasonic AVC-Intra implementation. Even with those disabled, AVC Intra isn't as smooth for editing (decoding latency is higher) than the likes of Prores, Cineform and DNxHD. And with CABAC and deblocking everything feels like it grinds to a halt.

    There are AVC implementation differences as well x264 usually does slightly better at intra encoding than Mainconcept for example, but sometimes has more compatibility issues

    If anyone is really interested I can post a few tests / results, but these are pretty much known facts
    Quote Quote