BDMV Authoring Workflow Guide

[SameSelf]

Normally, I encode my videos for the web, and my workflow for that is documented here:

https://forum.videohelp.com/threads/373829

However, I recently authored my first BDMV and had to adjust my workflow. My AE comps have heavy effects and render out at ~0.5 fps. Ordinarily, this is not a problem when authoring for the web because 1) the clips are short and 2) I use one-pass CBR encoding. I just kick off an encode before going to bed, and six or seven hours later when I wake up, it is finished. However, when encoding for Bluray, 2-pass encoding suddenly makes the process of encoding a 90-minute, 60 fps video at 0.5 fps look like a fool's errand. I have seen some people on this forum suggest lossless intermediates. I was loathe in the past to use them because of the sheer size of HD lossless files. But suddenly lossless intermediates looked like the way to go. So below is my modified workflow for authoring BDMV. Feel free to comment.

1. Shoot HD video with my Canon Vixia HV40 in a native 29.97 progressive frames per second.
2. Capture my footage live via the HDMI-out from the Canon using my Atomos Ninja 2 in a ProRes HQ 422 10-bit format as 1920x1080p29.97.
3. Process the footage in AE (grading, NR, etc.).
4. Dynamically link the AE comps in PP to build my timelines for rendering.
5. Export the timelines from PP as lossless AVIs using Lagarith in a YUY2 4:2:2 format (avoid RGB and YV12 4:2:0). Lossless intermediates are also nice because they offer the flexibilty of experimenting with various encode settings without having to re-encode at 0.5 fps.
6a. Downrez the FullHD footage to 1280x720 and 4:2:0 and double the frame rate to ensure bluray compliance using Avisynth.

AVISource("E:\Lagarith.avi", audio=false).AssumeFPS(30000,1001)
ConvertToYV12(interlaced=false, matrix="rec709")
ChangeFPS(60000,1001)
Spline36Resize( 1280, 720 )

6b. Because authoring programs prefer a single video file versus multiple files, stitch together the AVIs in Avisynth before encoding:

video1 = AVISource("E:\video1.avi", audio=false).AssumeFPS(30000,1001).ConvertToYV12(interlaced=false, matrix="rec709")
video2 = AVISource("E:\video2.avi", audio=false).AssumeFPS(30000,1001).ConvertToYV12(interlaced=false, matrix="rec709")
video1 ++ video2
ChangeFPS(60000,1001)
Spline36Resize( 1280, 720 )

7. Encode the avs script using x264. Note: the bitrates are set to just below 28 Mbps to avoid triggering the transcode flag in DVD Architect.

x264 --bitrate 27500 --preset veryslow --tune film --bluray-compat --vbv-maxrate 28000 --vbv-bufsize 27500 --level 4.1 --keyint 60 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 1 -o out.264 in.avs
x264 --bitrate 27500 --preset veryslow --tune film --bluray-compat --vbv-maxrate 28000 --vbv-bufsize 27500 --level 4.1 --keyint 60 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 2 -o out.264 in.avs

8. Once the video streams are encoded using x264, importing into DVD Architect or Encore is a cinch. The transcode flags are not triggered, and it is a trivial exercise to build the Bluray at that point. I won't detail the steps of authoring as this is a very bespoke process.

[ron spencer]

Thanks...but it is odd that you title your thread "BDMV Authoring Workflow Guide" and then later say "I won't detail the steps of authoring as this is a very bespoke process." Bespoke??? Not sure really. Let us know the software you use, etc. You have provided your encoding workflow, not an authoring workflow.

Actually, most authoring programs can use either demuxed files for muxed files...I would not use one that cannot handle either. Why are you not using your 1080 file size or your captured files?

Is this your unit: http://www.atomos.com/ninja/


Another thought...I suppose one could capture and not do any editing...how would the encoding process work? Say you use the ninja to record from the HDMI out to the 10-bit 422 Apple ProRes HQ. If you wanted to use those files directly to bluray, what encoding line would you use for x264? I assume this workflow is better than the files the video recorder unit makes without the ninja?


Merry Christmas nonetheless and thanks for more info!!

[SameSelf]

Thanks...but it is odd that you title your thread "BDMV Authoring Workflow Guide" and then later say "I won't detail the steps of authoring as this is a very bespoke process." Bespoke??? Not sure really. Let us know the software you use, etc. You have provided your encoding workflow, not an authoring workflow.

Haha. Well, IMO, all the heavy lifting occurs in getting the media files prepped/encoded for acceptance into the authoring program without triggering the transcode flag. That is where the workflow is complicated.

Once that is done, it is a fairly simple process of importing your assets, building playlists, and linking to a menu. But then, maybe you want chapter points? Maybe you want a photo album? Maybe you want animated menus, transitions, intros, loop points, submenus, popups, sub-titles, extra content, etc., etc.? Well, go for it. But that is where my hand-holding stops. Sorry.

I have pretty much fully migrated over to DVD Architect.

Actually, most authoring programs can use either demuxed files for muxed files...I would not use one that cannot handle either. Why are you not using your 1080 file size or your captured files?

x264 only creates elementary video streams. I suppose I could mux the .264 files with the audio, but why? That is what the authoring program is for. And I don't save the .264 once the bluray is finished. They are intermediate files as far as I am concerned.

Also, Bluray only supports interlaced footage for 1080 content. My footage is native progressive and I didn't want to use the --fake interlaced flag in x264. That only leaves 1280x720. See the BR White Paper.

Is this your unit: http://www.atomos.com/ninja/

Yes. And Wow! I didn't realize that the price had dropped to about half of what I paid. Of course that was about two years ago. Great little device for capturing edit ready content from any camera that has HDMI out.

Another thought...I suppose one could capture and not do any editing...how would the encoding process work? Say you use the ninja to record from the HDMI out to the 10-bit 422 Apple ProRes HQ. If you wanted to use those files directly to bluray, what encoding line would you use for x264? I assume this workflow is better than the files the video recorder unit makes without the ninja?

The x264 line would not be any different. The challenge would be editing the Avisynth code to successfully load the resultant MOV. According to this link, there is an experimental plug-in for such a task. http://avisynth.nl/index.php/QTSource. I haven't tried it. But to be honest, the whole point of the Ninja is to bypass in-camcorder compression and put the footage in an edit friendly format that holds up to grading and other effects. If you don't plan on doing any of that, then I don't really recommend the Ninja as part of your workflow. My Canon HV40 records to HDV tape. When I know I will not be doing any editing of the footage, I just record straight to tape. But that is an MPEG format easily loaded my Avisynth.

Merry Christmas nonetheless and thanks for more info!!

Merry Christmas to you too! Ho ho ho!

[ron spencer]

A few more questions.

Assuming you are using "clean" HDMI (straight from sensor), it is obviously better to use ninja device to capture in huge bitrate, do editing, and encode later. But have you tested to see if it is better than the in-camera/in-video processing? That is, assuming that no additional editing is needed, is it still better quality to use the ninja and encode later on your own? It is more time consuming that just using your video recorders output, but is it better?

This could be good for AVCHD recorders too, as long as there is "clean" HDMI out.

[SameSelf]

I am not an expert in these matters, but I will try to answer your question from my experience. I am sure others on this forum could do a better job.

The main thing to remember is that whether you are talking about MPEG (which is what my camcorder uses) or AVCHD (which most all modern camcorders now use), these are delivery format encodes meant to be watched, not edited. I am not saying my cam, or any cam, does a bad job in encoding video to MPEG or AVCHD. But:

• If you plan to do serious color correction/grading, noise reduction, sharpening, etc., then you are always better off with a lightly compressed, edit-ready format like ProRes or DNxHD (the two options in the Ninja).
• Editing is easier because scrubbing and playback is smoother in NLEs compared to AVCHD (MPEG is not a problem for me at least).
• If you do a lot of cuts on your video, that is easier with edit-ready formats versus long GOP formats where scenes have to be interpreted if, for example, you cut on a B frame.
• You avoid generational losses in your workflow. For example, I only encode once. If I need to make an intermediate encode, I use a lossless intermediate.
• You have complete control over the final encoder to use with the help of frameservers. Right now, x264 is among the highest quality H.264 encoders out there. So I use that whether I am authoring for the web or BD. It is certainly better than any camera codec unless it is a pro camera that uses a pro codec. It is better than the MainConcept encoder bundled in Adobe or Vegas.

As for testing my camera, given everything above, it is an Apples vs Oranges comparison. I am not sure what I would learn. First, it would be extremely difficult to do properly and draw valid conclusions. Just google the various discussions on this forum about encoder comparisons. Second, the analogy I would draw is using Auto mode on your camera versus having complete control over focus, f-stop, shutter speed, gain, etc. The average user could care less about the things. The pro wouldn't choose a camera without it. Using a Ninja or any capture device is putting the the choice of encoder settings in the hands of the user versus letting the camera choose.

[_Al_]

But have you tested to see if it is better than the in-camera/in-video processing?

Canon HV30, HV40 camcorder footage is good enough, for home user, if edited and then export originals again, except places where GOPS are reconstructed. For home user ,that original HDV camcorder footage , besides archiving it is also final delivery footage.

In extreme situations, like a swimmer in detail approaching camcorder, water is splashing in the whole frame, it might introduce blockiness a bit, not noticeable during casual viewing, but that be not acceptable for a professional that works with that footage further ...

Avchd camcorders are perhaps much better regarding this but color corrections etc., keying, it would start to show up something anyway ...

So it is not just how that 4:2:2 footage compares to original mpeg2 or H.264 during casual viewing but there should be also a room for further processing (colors, keying, if slowing down it should have without any encoding artifacts etc. ).

[SameSelf]

A few more questions.

Assuming you are using "clean" HDMI (straight from sensor), it is obviously better to use ninja device to capture in huge bitrate, do editing, and encode later. But have you tested to see if it is better than the in-camera/in-video processing? That is, assuming that no additional editing is needed, is it still better quality to use the ninja and encode later on your own? It is more time consuming that just using your video recorders output, but is it better?

This could be good for AVCHD recorders too, as long as there is "clean" HDMI out.

I think I finally figured out a definitive answer to some of your questions. I ran some tests and, yes, the Ninja footage from the HDMI-out is better than the HDV footage even when I upsample the HDV footage to 4:2:2. As for which is better, YUV 4:2:2 Ninja footage encoded to YUV 4:2:0 using x264? Or HDV footage encoded in-cam to YUV 4:2:0 MPEG2? It is a moot point in my mind because I rarely leave my Ninja footage untouched. But with that said, since I edit in 4:2:2 and save the chroma subsampling until the Avisynth script, I have learned that I can control the chroma resampling using any resizer, like Spline36. I like having control over the chroma subsampling versus leaving it up to the camera because once it is gone, it is gone forever, there is no getting that 4:2:2 information back as my tests show.

https://forum.videohelp.com/threads/377475-Chroma-Sampling-Question?p=2436852

As for being good for AVCHD recorders, I can't say definitely. It depends on what the camera's HDMI outputs. The HV40 was manufactured long before the Ninja came along. I don't know if camera manufacturers have crippled the HDMI out to avoid cannibalizing their higher end 4:2:2 cameras. The only way to know for sure is to conduct a test like mine to see if you camera bypasses the compression. My advice to anyone considering a Ninja is to buy one on ebay to test. If it doesn't work just sell it back on ebay. The cost to do so is only about $30.

[ron spencer]

What ninja do you have? Their page says ninja 2. What do you mean by:

My advice to anyone considering a Ninja is to buy one on ebay to test. If it doesn't work just sell it back on ebay. The cost to do so is only about $30.

$30???

Thanks for your posts.

[SameSelf]

I have a Ninja 2. You can buy one for $300 on ebay and sell it back for $300 then Paypal will charge you roughly 10% on the sale transaction, so cost of $30.

[SameSelf]

What do you mean by $30???

I should also mention that I have read (never tried) that B&H and Adorama have a 30-day return policy with no re-stocking fee. I highly recommend getting one and testing. Having a Ninja really opened up a world of possibilities that I didn't have before buying one and the price is so cheap now for what it does: ProRes/DNxHD 4:2:2 10-bit FullHD continuous recording only limited by the size of the SSD. I have a 240 GB Sandisk Extreme Pro which can hold over 2 hours of ProRes HQ video. Battery life is long-lived. The only real complaint I have is the screen isn't very good. The Ninja Blade is supposed to be a much better screen, but wasn't available when I got mine. But it is not such a hindrance that I feel the need to upgrade.