Using VirtualDub and AviSynth to make a split/PiP movie

[WebMaximus]

The last few days I've been trying to find a good freeware software that will let me compress and combine 3 video clips produced by FRAPS into one by dividing the screen into 3 parts.

I did find one freeware video editing software called VSDC Free Video Editor but I didn't manage to get a descent output quality using it and I also had problem arranging the clips the way I wanted them. Using VirtualDub on the other hand and a video codec called x264vfw - H.264/MPEG-4 AVC codec produced very good quality but then I have the problem how to combine the 3 movie clips into one.

After some searching on Internet I found out that you can use VirtualDub together with AviSynth scripts to produce this kind of effect, (Split screen/PiP).

Next step will be trying to learn a little bit more about this but since I'm all new to this I though I should ask you pros in here for any tips on how to accomplish this.

In short my situation is as follows:

I have 3 movie clips made by FRAPS. What I would like to do is to compress them using VirtualDub with the codec mentioned above and to have the 3 movie clips combined into the same screen so the output will show all 3 movie clips playing simultaneously - one at the upper half of the screen from side to side and the other two side-by-side at the bottom half of the screen. Also I would like the clips to be automatically re-scaled to fit and last I only want the output to include the audio from one of the clips at the lower half of the screen.

I made a very simple drawing trying to illustrate this



Thanks in advance for any tips on how to accomplish this using VirtualDub together with AviSynth scrips.

[manono]

Also I would like the clips to be automatically re-scaled to fit and last I only want the output to include the audio from one of the clips at the lower half of the screen.

You can tell just by looking at the picture that the width x height ratios aren't the same for A .vs B and C. Maybe before resizing them you'll be cropping from the top and/or bottom of A? Or from the sides of both B and C?

The audio issue is simple. Just add in the audio you want during reencoding or muxing. As for combining them all, simple as well:

A=AVISource("Clip A.avi",Audio=False)#Disables the audio, if it has any
B=AVISource("Clip B.avi",Audio=False)#Disables the audio, if it has any)
C=AVISource("Clip C.avi")
D=StackHorizontal(B,C)
StackVertical(A,D)
This can all be done in one line (I think), but I've divided it into five to make it easier to understand

You could also add little black borders between the videos with the AddBorders command. Most of what you're asking is pretty simple in AviSynth. The tricky part would be to get the three clips at the right aspect ratio, in my opinion.

[WebMaximus]

Thanks a lot for your input, I'll try what you suggest.

As for the aspect ratio I see what you mean but I was hoping it would be some smart way to have the clips automatically resized pretty much in the same way as when you're playing a movie in WMP for example and resize the WMP window.

[WebMaximus]

Using what you suggested I was able to accomplish what I was asking for by using the following script

A=AVISource("A.avi", false)
B=AVISource("B.avi").BilinearResize(960,1080)
C=AVISource("C.avi", false).BilinearResize(960,1080)
D=StackHorizontal(B,C)
StackHorizontal(B,C)
StackVertical(A,D)

However like you warned me about the AR of clip B and C was messed up and both clip B and C looked very strange and compressed so I have now started to re-think how I could combine all clips into one single movie but at the same time keep the original size and aspect ratio. One idea I came up with and I'm not sure if it's possible to put into practice using a AviSynth script would be to instead of showing all clips at the screen at the same time instead alternate between them every 10 seconds for example.

Say I'm having clip A, B and C like in my original example and when I start the movie I'll see clip A for 10 seconds between 00:00-00:10, then a nice transition effect switches to clip B for 10 seconds before transitioning to clip C at 00:20 and then back to clip A again at 00:30 and so on. Important of course is that all three clips need to be playing "in the background" all the time even when they are not "in focus" sort of speak so the time sync between them is kept and they don't fall behind so in this example when the transition is made back to clip A from clip C at 00:30 clip A now also need to be at 00:30 and not at 00:10 from when the transition was made from A to B.

Do you guys think this would be possible?

[jagabo]

You can use Trim() to make sub clips and glue them together in whatever sequence you want.

A = AviSource(...)
B = AviSource(...)
C = AviSource(...)

# 0-299 of A   +  300to599 of B  +  600-899 of C
Trim(A,0,299) ++ Trim(B,300,599) ++ Trim(C,600,899)

Or you can use ReplaceFramesSimple() (part of the RemapFrames filter) to copy blocks of frames around.

A = AviSource(...)
B = AviSource(...)
C = AviSource(...)

last = A
ReplaceFramesSimple(last, B, mappings="[300 599]") # copy frames 300 to 599 of video B onto video "last"
ReplaceFramesSimple(last, C, mappings="[600 899]") # copy frames 600 to 899 of video C onto video "last"

Note that when you don't specify a video "last" is used. So a sequence like:

Filter()

means

last = Filter(last)

[poisondeathray]

In order to preserve the AR in the 1st approach, A would be pillarboxed (black bars on the left & right), B &C would be letterboxed (black bars on the top & bottom)

What kind of "nice transitions" did you want ? Avisynth has built in simple transitions, like fade, dissolves,
http://avisynth.org/mediawiki/Dissolve
http://avisynth.org/mediawiki/FadeIO

There are some transitions from the transall plugin (but many of them are "cheezy" 80's style transitions IMO)
http://avisynth.org/vcmohan/TransAll/docs/index.html

[WebMaximus]

You can use Trim() to make sub clips and glue them together in whatever sequence you want.

A = AviSource(...)
B = AviSource(...)
C = AviSource(...)
 
# 0-299 of A   +  300to599 of B  +  600-899 of C
Trim(A,0,299) ++ Trim(B,300,599) ++ Trim(C,600,899)

Or you can use ReplaceFramesSimple() (part of the RemapFrames filter) to copy blocks of frames around.

A = AviSource(...)
B = AviSource(...)
C = AviSource(...)
 
last = A
ReplaceFramesSimple(last, B, mappings="[300 599]") # copy frames 300 to 599 of video B onto video "last"
ReplaceFramesSimple(last, C, mappings="[600 899]") # copy frames 600 to 899 of video C onto video "last"

Note that when you don't specify a video "last" is used. So a sequence like:

Filter()

means

last = Filter(last)

Tried what you suggested just to see how it would work but received an error message saying ReplaceFramesSimple didn't exist but guess that's because I need to first get this RemapFrames filter you mention?

Then thinking about this approach I'm not sure how I would be able to put it into practice since my clips will be around 10-15 minutes so if I was to manually move every 300 frames between the clips it would be a huge amount of work. But maybe there's a way to automate this creating some kind of loop that will increase the value automaticallly? I'm in no way used to writing code or working with scripts so forgive me for appearing stupid

After writing my last post in here I came to think of another approach that maybe would be a lot easier but I'm not sure if it's possible outside my dreams...

What I'm thinking is if I could add all three clips like "layers" of the same size where only the upmost layer will be visible hiding the other two layers underneath and then have some function that will alternate every let say 10'th second which layer(clip) is on top hiding the other two layers I would get the same effect only seeing one clip/layer at a time without having to rearrange any frames or nothing.

What do you think, would that be possible and does it even make sense?

As for transitions of course it would be nice and cool looking but that is not highest on my list at this moment but rather finding a smart way alternating between the 3 clips every x seconds.

[WebMaximus]

Just tried this very simple script to check out how Dissolve works and this is exactly what I'm looking for only that I would like to have this Dissolve transition alternating between the three clips every 10 seconds instead of first playing the full length clip A, then B and C.

Hmm...

A=AVISource("A.avi")
B=AVISource("B.avi")
C=AVISource("C.avi")
Dissolve(a, b, c, 30)

[poisondeathray]

ReplaceFramesSimple is part of stickboy's remapframes.dll plugin (not included with standard install)
http://avisynth.org/stickboy/
http://avisynth.org/stickboy/RemapFrames.zip

Did you want it to alternate every 10 seconds (A,B,C, A,B,C, A,B,C.....) , or just once each ? (A,B,C)

If just once each, use trim() to cut each clip
eg. a 30 fps clip would be trim(0,299) for 300 frames = 10seconds . If the fps is different, adjust the trim value

a = avisource("a.avi", audio=false).trim(0,299) 
b = avisource("b.avi", audio=false).trim(300,599)
c = avisource("c.avi", audio=false).trim(600,899)

ab=dissolve(a,b,30)
dissolve(ab,c,30)
vid=last

#audio from clip c
aud = avisource("c.avi").trim(0,899)

audiodub(vid,aud)

[WebMaximus]

I want it exactly as you explained:

Did you want it to alternate every 10 seconds (A,B,C, A,B,C, A,B,C.....)

...and I want it to work regardless of the length of the clips since the length will be different from time to time. Not the length of clip A, B and C because those will always be the same length but what I mean is one time clip A, B and C will be maybe 10 minutes but next time they will be 15 minutes. So for that reason I need a method that works regardless of the time of the clips.

[jagabo]

I want it exactly as you explained:

Did you want it to alternate every 10 seconds (A,B,C, A,B,C, A,B,C.....)

I don't see any way to automate that.

[WebMaximus]

Ok, too bad if you're right because I don't think it's especially advanced and looking for the last couple of days what cool things you can do with Avisynth I thought simply alternating between a couple of clips would be piece of a cake.

Will look more into the possibility using the layer approach tomorrow.

[WebMaximus]

If it's not possible to do this with Avisynth does anyone know of any other software that would let me do this?

I don't buy it's impossible and although I was hoping for a free way I would be willing to look into some payware alternative if that's the only option.

[manono]

Why not go back to your first plan by making multiple screens? Either do it the way pdr suggested, by adding black bars where necessary to keep the aspect ratio, or the way I'd probably do it - by cropping where necessary (depending on the visual content). And then resizing everything.

[jagabo]

Why not:

A=AVISource("A.avi", false)
B=AVISource("B.avi").BilinearResize(960,540)
C=AVISource("C.avi", false).BilinearResize(960,540)
D=StackHorizontal(B,C)
StackHorizontal(B,C)
StackVertical(A,D)

I'm guessing all three videos are 1920x1080. That will give you the two smaller videos with the correct aspect ratio (ie, resized by half in both dimensions).

[jagabo]

I don't buy it's impossible

It may be possible using the runtime features but I'm not familiar enough with those to give you a specific script. Basically you would pick frames based on:

(current_frame / 30) % 3

If the result is 0 pick from video A, if 1 pick from video B, if 2 pick from video C.

[WebMaximus]

Maybe I'll try adding boarders/cropping as suggested just to see what it would look like but what I've learned after posting my first plan is that the immersion factor watching the final output is very much increased watching the clips in their original/full screen size.

For this reason I like my second plan better to have the clips alternating (in full screen) every X seconds rather than have them stacked in one single screen at a much reduced size with or without borders compared to watching the clips full screen.

Just realized I haven't wrote anything about the actual content of the clips and maybe doing that will make everything more clear. What I'm trying to do is to "re-create" a segment of a flight in Microsoft Flight Simulator X where clip A in my drawing above represents the cockpit view and clip B and C represent left and right views over the wings.

Here is a screenshot what it looks like and in this screenshot I used the script below but as you can see (although not as clearly as when actually watching the running movie) is how compressed the wing views look.

A=AVISource("A.avi", 
false)
B=AVISource("B.avi").BilinearResize(960,1080)
C=AVISource("C.avi", 
false).BilinearResize(960,1080)
D=StackHorizontal(B,C)
StackHorizontal(B,C)
StackVertical(A,D)

This is what the wing views look like in their original format, a HUGE difference from the split view above as you can see.

[jagabo]

So do what I suggested -- resize both the width and height of the small frames keeping the AR.

[WebMaximus]

Yeah, just tried what you suggested and it does look a lot better keeping the AR but still I would prefer to use the method having the clips alternating for the reason mentioned in my post above.