Multi-angle 1080p concert video

[warmaker]

Hi everybody!

To the point: how do I create multi-angle 1080p video from short clips (from a concert I went to recently) so that I am able to switch beetween cameras/angles seamlessly?

I have an audio track/bootleg that will be my reference to sync all clips and this I can do, I get a nicely ordered timeline, but where do I go from there?

I am almost a total beginner in video editing, but a quick learner and I enjoy learning new things. I therefore should be fine if you just point me in the right direction and not

necessarily provide step-by-step instructions (althoug that would be awesome!)

[_Al_]

downloaded audio and your concert video can be out of sync with no adjusting formula, it will be all over the place, you should use your audio downloaded at the same time, non stop (audio and your videos are recorded separately)

making multi-angle BD is out of question I guess, so just make multitrack MP4 or MKV and toggle videotracks (using player that can do that like ffplay)

choose a videoeditor with multi-track function, it can sync your videos , or without that function (more tedious work), export each track separately, exactly same audio length, to MP4 , then export audio.

use MP4box.exe to create multitrack MP4,
"MP4box.exe" -add "video1.mp4#video" -add "video2.mp4#video" -add "video3.mp4#video" -add "audio.AAC#audio" new.mp4

or use GUI and MKVmerge to get MKV file

use ffplay to play the file and toggle videotracks, not sure how it is going to behave, it might not be exactly seamless , you press key "V" on your keyboard to toggle between your tracks, you start your video with command line:

"ffplay.exe" "new.MP4"

[warmaker]

Hi _Al_, thanks a lot for all the information.

For syncing I use PluralEyes and it seems to be doing excellent job. So no problem here.

The idea with making multitrack file is nice, but I am afraid it won't work here, as what I have are some shreds and random parts of the concert:

[Attachment 43019 - Click to enlarge]

Sure, I could (and will) arrange them, so that I don't have 15 cameras, but still I would get a lot of blank/black spaces in between, therefore I would prefere to have one main video track and every once in a while an option to switch to a different camera/angle.

Could you help me out more in this direction? Or put me right, if I am mistaken?
Thanks!

[_Al_]

Then make two or three tracks and fill empty space in those with main track (which is happening by default if main track is at the bottom),

you have to have an extra video track or tracks. You cannot create a video container with a video track to exist only in some parts.

[smrpix]

Please clarify something. Are you trying to make a file you can live switch each time you view it, or are you trying to combine the clips into a set show?

PluralEyes can do an FCP xml export that Premiere Pro understands, so you can complete your editing there.

[warmaker]

Please clarify something. Are you trying to make a file you can live switch each time you view it, or are you trying to combine the clips into a set show?

The former. I would like to be able to switch to a different angle/camera each time I view the video.

PluralEyes can do an FCP xml export that Premiere Pro understands, so you can complete your editing there.

Nice! Any other software you could recommend to do the job? Preferably free/cheap/simple, as I most probably won't be doing any advanced editing.

[warmaker]

Then make two or three tracks and fill empty space in those with main track (which is happening by default if main track is at the bottom), you have to have an extra video track or tracks.

I get it. But does it mean that the "main track" parts will be copied to the secondary (and tertiary) track to fill the blanks, therefore almost doubling (tripling) the file size (let's say there's a lot of blanks in the secondary and tertiary track)?
E.g. from what I know some Star Wars DVDs have the opening crawl in different languages, while the rest of the movie is the same, obviously. So it's just a tiny fraction of the total running time.
But maybe DVD is just a different story or this is basically diferent situation?

You cannot create a video container with a video track to exist only in some parts.

That's EXACTLY what I think I wanted to do I just couldn't put it down in words that nicely. Too bad, that would make sense.... I guess... Have the main track and secondary/tertiary with content only at specific times. At other times the change to another track/angle/camera is disabled.
Anyway, if you say this is not possible, then how do they do it? Like you suggested above?

[_Al_]

then how do they do it?

Do you mean DVD or BD? It has the same principal, creating multiple tracks , each track is filled with video, no gaps there, and then just switching between tracks during playback. I would not even try to do this though, creating Blu-Ray like this.
BD's have a feature for selection of different title versions, like directors cut version etc.,it is branched seamlessly, parts added or dropped, but that is not what you want. BY choosing a video title you basically select a playlist consisting of some particular m2ts video files.

There might be other platforms, scripting, that might be other story. There might be some platforms around. Or some HTML5 web video players. But not remembering any. But all the same. You'd need whole video tracks anyway to things work simple.

[warmaker]

then how do they do it?

Do you mean DVD or BD? It has the same principal, creating multiple tracks , each track is filled with video, no gaps there, and then just switching between tracks during playback. I would not even try to do this though, creating Blu-Ray like this.
BD's have a feature for selection of different title versions, like directors cut version etc.,it is branched seamlessly, parts added or dropped, but that is not what you want. BY choosing a video title you basically select a playlist consisting of some particular m2ts video files.

There might be other platforms, scripting, that might be other story. There might be some platforms around. Or some HTML5 web video players. But not remembering any. But all the same. You'd need whole video tracks anyway to things work simple.

Thanks a lot for explaining. It seems I will have to slightly change my approach to this project. I hoped it would be doable the way I imagined.

Just one more question - will the videoeditor normally fill the blanks in secondary/tertiary track by itself or do I have to set it?

If you later remember some another solution (platfroms, scripting, html5, anything) let me know. TIA!

[_Al_]

will the videoeditor normally fill the blanks in secondary/tertiary track by itself or do I have to set it?

That is a default feature.
Videoeditor makes visible only those clips on the top. If there is no clip it will render whatever there on track below.
While exporting your main video track only you have to make the other tracks above temporarily invisible or you move your main/bottom track temporarily on the top.

[johnmeyer]

You can easily do what you want with a DVD, but of course it won't be 1080. I have authored quite a few multi-angle DVDs. The spec was created with that in mind. Most multi-angle DVDs are exactly as you show in your excellent diagram: only portions of the concert have more than one viewpoint. It almost has to be this way when dealing with a medium that has finite storage because, as already pointed out, if you included just three tracks, and included footage for the entire concert for each track, it would triple the size of the project, and you could therefore only fit 1/3 the running time on a disc.

While you might not want to live with the spatial quality compromise in authoring a DVD (i.e., not getting 1920x1080 quality), it does have one other advantage in that the navigation is built into every player and, depending on the DVD player, usually lets you switch to the other angle with one press of a button.

I used DVD Architect to create the few multi-angle DVDs that I authored. Because of the way multiple files are combined, you have to let DVD Architect do the rendering, i.e., the DVD spec doesn't just let you multiplex different rendered files together. This is the quote from the DVD Architect manual:

"Any title that contains multiple video tracks will be recompressed when you prepare your project so the main video and the alternate angles can be combined into a new video file. To avoid recompression artifacts (and unnecessary processing time), use AVI files for your multiangle video titles."

[warmaker]

will the videoeditor normally fill the blanks in secondary/tertiary track by itself or do I have to set it?

That is a default feature.
Videoeditor makes visible only those clips on the top. If there is no clip it will render whatever there on track below.
While exporting your main video track only you have to make the other tracks above temporarily invisible or you move your main/bottom track temporarily on the top.

Thanks _Al_, I will be playing around with it this week probably and see where it gets me. Thanks for your help and comments!

Still, I can't help but wonder that what I thought is possible with much older DVDs and not nowadays...

[warmaker]

You can easily do what you want with a DVD, but of course it won't be 1080. I have authored quite a few multi-angle DVDs. The spec was created with that in mind. Most multi-angle DVDs are exactly as you show in your excellent diagram: only portions of the concert have more than one viewpoint. It almost has to be this way when dealing with a medium that has finite storage because, as already pointed out, if you included just three tracks, and included footage for the entire concert for each track, it would triple the size of the project, and you could therefore only fit 1/3 the running time on a disc.

While you might not want to live with the spatial quality compromise in authoring a DVD (i.e., not getting 1920x1080 quality), it does have one other advantage in that the navigation is built into every player and, depending on the DVD player, usually lets you switch to the other angle with one press of a button.

I used DVD Architect to create the few multi-angle DVDs that I authored. Because of the way multiple files are combined, you have to let DVD Architect do the rendering, i.e., the DVD spec doesn't just let you multiplex different rendered files together. This is the quote from the DVD Architect manual:

"Any title that contains multiple video tracks will be recompressed when you prepare your project so the main video and the alternate angles can be combined into a new video file. To avoid recompression artifacts (and unnecessary processing time), use AVI files for your multiangle video titles."

Hi johnmayer, thanks for stopping by and taking a moment to share your knowledge.

I might be actually doing this too (DVD, I mean) with some of the earlier concerts I've been to back in the days when HD recording was not popular. Therefore my source material is of lower quality, so I won't lose much.

As for the other advantage - it is true, but I guess I forgot to mention that it is not vital for me to be able to play it anywhere with any device/software, as this will be strictly for my own use. But that's a good point anyhow.


It is still beyond me though, that this feature (multi-angle) is not implemented nowadays in the way it is with much older and in many ways inferior DVD format...

[johnmeyer]

Having produce a lot of DVD from events (concerts, ballets, wedding, etc.) where I have multiple cameras, I think I know why multi-angle isn't used more: the best shot is the one that gets used.

Yes, I know that you would like to be able to make that choice, and I am not challenging that idea. All I'm saying is that when a person (like me) looks at all the footage, it is actually pretty easy to see which angle is superior to all the others at any given point in time.

The only DVD I ever watched where multi-angle was really useful was when my kids were studying for their driving tests fifteen years ago. There was a driving instruction video that showed various aspects of a particular driving situation (e.g., approaching an intersection with cars at all four corners), and it was actually quite useful to replay the same scene from several different vantage points.

For me, if multi-angle were available on modern music concerts, I'd choose a shot that didn't cut every two seconds. Nothing disrupts the enjoyment of a ballet or concert than too many cuts.

[poisondeathray]

You can do 1080p multiangle on BD, but only the very expensive authoring suites support it



Some HTML5 examples:

This one looks ok, but doesn't have keyboard shortucts
http://youswitch.tv/content/youswitch_multi_angle_live_video_player

This one has a better implementation , and you can use number keys to toggle views
http://www.omnilive.fr/

But neither are "truncated" where you only have parts of streams (they are full views) . HTML5 can have problems with buffering and pauses when switching, you usually get around that by buffering, less of an issue for local playback



VR / 360 has a small but growing following now, but for something like a concert it's probably not as effective presentation method (and you didn't shoot for VR for this particular concert)

[_Al_]

HTML5 .............., you usually get around that by buffering, less of an issue for local playback

The problem actually arises running it locally (html5 in some browser on computer) because some server needs to be employed like Apache etc. So it is possible to implement it on some computer for presentations. Just an info for op. Without that server being installed that would be nice and really solid solution for today HD videos for multiple cameras. Video tracks could be even encoded with that in mind (shorter GOPs etc.). But as johnmeyer mentioned, if there was a demand for that, there'd perhaps be some simple solution available already (not BD).

[poisondeathray]

HTML5 .............., you usually get around that by buffering, less of an issue for local playback

The problem actually arises running it locally (html5 in some browser on computer) because some server needs to be employed like Apache etc. So it is possible to implement it on some computer for presentations.

I'm saying bandwidth is less of an issue when you have local playback, instead of risking some slow server upload

Did you look at how it's implemented ? It's a single video. The views are a large video grid, and you display the region of interest on demand. For example UHD would have 4x 1080p streams, and you switch to one quadrant. Those specific examples might use apache, but you shouldn't need it for that approach. The benefit is you never get gaps or pauses locally (assuming you can playback UHD, and you should be able to because of HW acceleration), and audio is 1 track so audio never glitches

[_Al_]

The views are a large video grid, and you display the region of interest on demand. For example UHD would have 4x 1080p streams, and you switch to one quadrant.

ok, that's interesting, one stream only

[warmaker]

Having produce a lot of DVD from events (concerts, ballets, wedding, etc.) where I have multiple cameras, I think I know why multi-angle isn't used more: the best shot is the one that gets used.

Yes, I know that you would like to be able to make that choice, and I am not challenging that idea. All I'm saying is that when a person (like me) looks at all the footage, it is actually pretty easy to see which angle is superior to all the others at any given point in time.

The only DVD I ever watched where multi-angle was really useful was when my kids were studying for their driving tests fifteen years ago. There was a driving instruction video that showed various aspects of a particular driving situation (e.g., approaching an intersection with cars at all four corners), and it was actually quite useful to replay the same scene from several different vantage points.

For me, if multi-angle were available on modern music concerts, I'd choose a shot that didn't cut every two seconds. Nothing disrupts the enjoyment of a ballet or concert than too many cuts.

johnmeyer, that's a great point and I think you are right - the best shot gets used.

But my case is slightly different as I am not creating commercial footage, but a souvenir for myself. And as such I indeed would like to have it available with multiple angles.

Also, it's a great tip with too many cuts. I will have to watch out for it.

Thanks!

[warmaker]

poisondeathray, _Al_

Thank you both for sharing your knowledge.
It is all very educative, especially the UHD video divided into 4 HD images. Clever. But like you said, it's still a constant double/triple/quadruple stream which makes the file size grow big and also isn't very handy i.e. having 3 tracks and only 2 input videos (a a given moment) means that on two tracks the image will be identical. If it's 4 tracks then it's either 2+2 or 3+1 and so on.

It would all be fine if I had enough source material to play with, but as you can see on the screenshot I posted, it's not the case unfortunately. And that's where my first idea of multi-angle video came from. To have the possibility to switch to another clip, but only when actually possible.

I will have to review my options now with what I've learned (thanks to you) and see what can I do with it.

Anyway - thanks a lot everybody, feel free to add any ideas that you might still have!

Cheers!

[poisondeathray]

poisondeathray, _Al_

Thank you both for sharing your knowledge.
It is all very educative, especially the UHD video divided into 4 HD images. Clever. But like you said, it's still a constant double/triple/quadruple stream which makes the file size grow big and also isn't very handy i.e. having 3 tracks and only 2 input videos (a a given moment) means that on two tracks the image will be identical. If it's 4 tracks then it's either 2+2 or 3+1 and so on.

It would all be fine if I had enough source material to play with, but as you can see on the screenshot I posted, it's not the case unfortunately. And that's where my first idea of multi-angle video came from. To have the possibility to switch to another clip, but only when actually possible.

I will have to review my options now with what I've learned (thanks to you) and see what can I do with it.

Anyway - thanks a lot everybody, feel free to add any ideas that you might still have!

Cheers!

It doesn't have to be a 2x2 grid - that's only if you had 4 views. You can have 2x1 e.g. 3840x1080, or 1x2 1920x2160.

If the 2nd view only has some valid sections, but mostly "black" or empty ,or even a static image, those areas don't "cost" very much in terms bitrate to encode. It might be 1-2% larger for a similar level of quality between 1920x1080 vs. 3840x1080 with a mostly static half. The reason is only the differences are stored between frames (and macroblocks). If both views had the same amount of action (essentially 2 full camera views, no dead space), then yes, you would need roughly double the size. If the 2nd view had only half valid, half empty, it would be roughly 1.5x - I'm sure you get the idea

If its only 2 views, you can do this quite easily with a software video player. 3d/steroscopic players usually have this option with L/R views and shortcuts already mapped. An example would be bino (F7 to swap L/R views, F for fullscreen). It should also be possible with some other video players, but using crop / pan scan functions and mapping your own hotkeys as custom function